10x faster python test iteration via fork(2)

It’s ideal to get feedback on your code faster — to make a code change and see the result instantly. But, as projects get larger, reload times get longer. Each incremental dependency or bootstrap code block that adds 200ms feels worth it, but 50 of them later and it takes 10 seconds to see the result of a code change. <img alt="" src="https://miro.medium.com/v2/resize:fit:700/1*kXnodzozvuBFupVH2kYfmA.png" style="height:176px; width:700px" /> On the Build team at Benchling, that’s where we found ourselves one day. We used 146 packages which pull in 128 transitive dependencies for a total of 274 packages. We also spent a lot of time waiting for SQLAlchemy models to initialize. The result is our test harness took 10 seconds to set up. After making a code change, you’d start the test runner, wait a few seconds, <code>alt</code>+<code>tab</code> to your browser, get distracted for a few minutes, and then find out you had a typo in your code. This is a common challenge for a growing codebase, but it’s something we knew we needed to fix. Here’s the process we arrived at which allowed the second run of tests to start 10x faster — 90% less waiting. While it’ll work a little differently for your codebase depending on the language, dependencies, etc. you’re using, hopefully this can inspire you on your journey to faster feedback and testing. <h1>importlib.reload()</h1> Since the problem is that we spend so long setting up a bunch of modules just right and then want to see the change in a single file we’re editing, the most obvious solution is to use <a href="https://docs.python.org/3/library/importlib.html#importlib.reload" rel="noopener ugc nofollow" target="_blank">importlib.reload</a> from the standard library. <a href="https://benchling.engineering/10x-faster-python-test-iteration-via-fork-2-3aae52d2f6">Click Here</a>