Hugging Face has written a new ML framework in Rust, now open-sourced!

Recently, Hugging Face open sourced a heavyweight ML framework, <a href="https://github.com/huggingface/candle" rel="noopener ugc nofollow" target="_blank">Candle</a>, which is a departure from the usual Python approach to machine learning, written in Rust with a focus on performance (including GPU support) and ease of use. According to Hugging Face, Candle’s core goal is to make Serverless inference possible. Full machine learning frameworks like PyTorch are very large, which makes it slow to create instances on a cluster; Candle allows for the deployment of lightweight binaries. Additionally, Candle allows users to remove Python from production workloads. Python overhead can have a serious impact on performance, and <a href="https://www.backblaze.com/blog/the-python-gil-past-present-and-future/" rel="noopener ugc nofollow" target="_blank">GIL</a> is a known headache. <h1>Is Rust really possible?</h1> The Pytorch framework is written in Python, and the API is based on Python, which makes getting started very fast. In addition, Python itself is a simple and easy-to-learn programming language, which makes it suitable for both beginners and professional developers. However, the Python-based Pytorch framework has obvious problems. Python can cause performance problems in some cases compared to some static graph frameworks such as TensorFlow, Python’s Global Interpreter Lock (GIL) can affect performance in multi-threaded situations, especially when it comes to CPU-intensive tasks, and Python’s interpreted nature can also introduce some runtime overhead. Additionally, deploying Python-based PyTorch models into a production environment may require some extra steps that are not as convenient as other compiled languages. Apparently, Hugging Face has been looking for a solution, and its answer is to rewrite an ML framework in Rust, the fastest language available today. “What’s cool is that this is from Hugging Face, not just someone’s hobby project.” One user exclaimed. In fact, many HF ecosystems already use Rust, such as <a href="https://github.com/huggingface/safetensors" rel="noopener ugc nofollow" target="_blank">safetensors</a>, <a href="https://github.com/huggingface/tokenizers" rel="noopener ugc nofollow" target="_blank">tokenizer</a>. However, the difficulty of Rust has also put some developers off, “Writing Rust is a tough thing to do, you have to jump around and spend more time thinking about the abstractions of the programming language than you do thinking about the problem to be solved. So I’m in no hurry to rewrite anything in Python just yet.” <a href="https://medium.com/@Aaron0928/hugging-face-has-written-a-new-ml-framework-in-rust-now-open-sourced-1afea2113410">Website</a>