Deep Dive performance optimization techniques in Azure data bricks.
<p>Microsoft Azure data bricks is one of the trendy topics for dealing with large amounts of data & streaming use cases. Based on the growth of data, it is important to maintain data quality & data redundancy. This article will help with internal of housekeeping jobs, duplicates removable & couple of scripts used in the projects.</p>
<p><strong>Step 1 : —</strong> As the project starts, ideally, it is divided into three layers: Data Ingestion, transformation & analytics. It is important to know what data injection is & data redundancy checks. We received CDC data from an Oracle to Azure storage account using Qlik replicate. After that, with some transformation, we will be appending data to the Silver layer.</p>
<p><a href="https://medium.com/@anujg942541/deep-dive-performance-optimization-techniques-in-azure-data-bricks-55189ec8ce71"><strong>Learn More</strong></a></p>