Understanding Group Sequential Testing
<p>A/B tests are the golden standard of causal inference because they allow us to make valid causal statements under minimal assumptions, thanks to <strong>randomization</strong>. In fact, by randomly assigning a <strong>treatment</strong> (a drug, ad, product, …), we can compare the <strong>outcome</strong> of interest (a disease, firm revenue, customer satisfaction, …) across <strong>subjects</strong> (patients, users, customers, …) and attribute the average difference in outcomes to the causal effect of the treatment.</p>
<p>The implementation of an A/B test is usually not instantaneous, especially in online settings. Often users are treated <strong>live</strong> or in <strong>batches</strong>. In these settings, one can look at the data before the data collection is completed, one or multiple times. This phenomenon is called <strong>peeking</strong>. While looking is not problematic in itself, using standard testing procedures when peeking can lead to <strong>misleading conclusions</strong>.</p>
<p><a href="https://towardsdatascience.com/understanding-group-sequential-testing-befb35cec07a"><strong>Click Here</strong></a></p>