Delusive Extrapolation and A/B Testing
<p><strong><em>A/B testing is becoming increasingly popular in digital product development. Early feedback on the choice of direction may save you a lot of time and money. However, be careful about drawing too far-reaching conclusions from your test results.</em></strong></p>
<p>Imagine that for one day we let 5% of our retail’s online customers test a new feature. We notice that those who have the new feature (the test group) spend on average 20% more money on our site than the other 95% (the control group). This is an A/B test: we’re arranging context A (the new feature) for one group and context B (no new feature) for another group. We then analyze if behaviors differ between the groups as a consequence. From this analysis, we may extrapolate new theories, for example, that if all users gain access to the new feature tomorrow, we will increase total sales by 20%.</p>
<p><a href="https://medium.com/pragmatic-programmers/delusive-extrapolation-and-a-b-testing-ffebe8582a58"><strong>Learn More</strong></a></p>