Instacart’s Item Availability Architecture: Solving for scale and consistency.

This blog post is the last segment in a three-part series that outlines our innovative approach to overcoming inventory challenges, delving into the application of product design, machine learning, and engineering technology. Here are <a href="https://tech.instacart.com/how-instacarts-item-availability-evolved-over-the-pandemic-5ce8e84468a0" rel="noopener ugc nofollow" target="_blank">Part-1</a> and <a href="https://tech.instacart.com/how-instacart-modernized-the-prediction-of-real-time-availability-for-hundreds-of-millions-of-items-59b2a82c89fe" rel="noopener ugc nofollow" target="_blank">Part-2</a> of the series if you want to catch up. In the first part of this series, we explored the dramatic shift in Instacart’s item availability, especially during the pandemic, and how we harnessed real-time availability predictions to keep pace with the rapidly fluctuating in-store inventory. In our second part, we delved into the machine-learning artistry behind real-time predictions for hundreds of millions of items. In this final segment, we share the engineering infrastructure we built to scale real-time predictions and foster faster experimentation. <h1>RTA infrastructure</h1> Building an infrastructure to serve Real-time Availability (RTA) predictions shared in <a href="https://tech.instacart.com/how-instacart-modernized-the-prediction-of-real-time-availability-for-hundreds-of-millions-of-items-59b2a82c89fe" rel="noopener ugc nofollow" target="_blank">Part-2</a> required us to address several challenges. <ol> <li>Support fetching of scores via Remote Procedure Call (RPC): For using real-time predictions, we needed to support fetching of scores through a new RPC created by the Machine Learning (ML) team.</li> <li>Support low latency use cases: Because the scores are used in filtering at the retrieval stage, we needed to support fast and bulk fetching of scores at the retrieval stage. An RPC-based approach would be too slow for these needs, so we needed an alternate approach on top of the Real-Time scoring API.</li> <li>Support high-consistency use cases: Consistency of score and what is available across all surfaces was vital, especially when fetching scores for UI changes. Informing customers of an item’s unavailability while showing it available on another surface would erode customer trust.</li> </ol> To solve these challenges, we implemented two methods for ingesting scores generated by the ML models into our DB storage for fast and bulk retrieval. For both methods, we chose to store the scores in a DB and update them to address the need for low-latency use cases. This approach puts the scores closer to retrieval and hence can be fetched via SQL joins. <a href="https://tech.instacart.com/instacarts-item-availability-architecture-solving-for-scale-and-consistency-f5661acb20a6">Read More</a>