Practitioner’s Guide to Statistical Tests

<p>Hi, we are&nbsp;<a href="https://www.linkedin.com/in/marnikitta" rel="noopener ugc nofollow" target="_blank">Nikita</a>&nbsp;and&nbsp;<a href="https://www.linkedin.com/in/daniel-savenkov-098b41149" rel="noopener ugc nofollow" target="_blank">Daniel</a>&nbsp;from the CoreML team at VK. It&rsquo;s our job to design and improve recommender systems for friends, music, videos and the news feed. This involves tons of A/B tests, and our progress at the end of the day relies heavily on how accurate and efficient our A/B tests were.</p> <p>The two most essential things in A/B tests are the design of the experiments and accurate analysis of the experiments&rsquo; results. In this article, we will stick to the most common design and compare various statistical analysis procedures, from the very standard t-test and Mann-Whitney test to state-of-the-art approaches like the reweighted bootstrap. The code to reproduce everything you will see in this post is available here:</p> <p><a href="https://vkteam.medium.com/practitioners-guide-to-statistical-tests-ed2d580ef04f"><strong>Click Here</strong></a></p>