Building a Cloud Data Platform
<p>Building a Cloud data platform is not simple, there are several important design considerations to keep in mind. I got an unique opportunity recently to build a greenfield Cloud Data Platform ground up for my company. For background — my company is the carved out Tea business entity of Unilever where we are building all IT systems from scratch.</p>
<p><strong>Guiding Principles/Requirements</strong></p>
<ol>
<li><strong>Start with a minimum viable product (MVP)</strong>: In a carve out scenario there is a lot of fluidity in terms of requirements and resources so the ask was to start small and build a viable product — so scalability was very important both in terms of performance and cost.</li>
<li><strong>Open source and open standards</strong>: Needless to say the ask was to build something that can be easily integrated or optimized if necessary so proprietary formats and closed ecosystems was not something we were looking at.</li>
<li><strong>Quick turn around</strong>: Business wanted to see a quick turnaround as they are used to certain ways of working in the parent organization. So automation was paramount to meet the goals in a very short time.</li>
<li><strong>Tangible deliverables</strong>: Typically data warehouse projects generally have a long gestation period but the ask was to deliver something tangible while we continue to build the final product.</li>
</ol>
<p><a href="https://medium.com/@rajarsi.g/building-a-cloud-data-platform-3887552bbd38"><strong>Website</strong></a></p>