Run AI Models in CUDA-Enabled WSL2
<p>There will be a time you need to run your models in a Linux environment. However, your most powerful GPU is running a Windows, and you don’t want to replace the whole system with Linux, and those oh, those games.</p>
<p>WSL2 is a good choice that balances the gap between Windows and Linux. However, when I was writing these, there are rare articles or discussions about how to enable both CUDA and PyTorch to run LLM and Stable Diffusion in WSL2. After some struggle and testing, I find out a way to run all those large AI models in WSL2 with CUDA and PyTorch enabled. and the most important part:</p>
<blockquote>
<p>The performance is the same as in metal Windows</p>
</blockquote>
<p>Here are my steps to set up WSL2 with CUDA enabled.</p>
<h1>Prerequisites</h1>
<p>You must have all CUDA drivers installed in your Windows already, and smoothly running AI models with CUDA with all necessary components installed.</p>
<p>The reason behind this prerequisite is that WSL2 will use Windows’ CUDA driver</p>
<p><a href="https://medium.com/codex/run-ai-models-in-cuda-enabled-wsl2-6ae7dc9fe941">Read More</a></p>