Data Engineering Project: Twitter Airflow Data Pipeline

<p>Well, nowadays social media is abuzz with the legendary fight between Meta&rsquo;s CEO Mark Zuckerberg and X&rsquo;s owner Elon Musk. It has even escalated to the point of a cage fight between the two tech giants. Well we all know how that could turn out.</p> <p>&nbsp;</p> <p>Musk Vs Zuckerberg Cage Fight</p> <blockquote> <p>Well, their fight doesn&rsquo;t at all change the daily tasks for a data engineer :p We still have to cater to irritating requests like &mdash; &lsquo;can you quickly pull this data for me?&rsquo; at work.</p> </blockquote> <p>So if you want to take up a few extra projects on weekends to make yourself feel like you are a data engineer for real, then you have to follow a few legends who make great project use cases. I personally love the videos made by&nbsp;</p> <p>Darshil Parmar</p> <p>&nbsp;on data engineering. They will take you from a basic to an intermediate level in a matter of hours! I worked on one such project last weekend &mdash; Twitter data pipeline! It&rsquo;s honestly a great pipeline for beginners in data engineering. It&rsquo;s not transformation or code intensive but it helps you understand a basic flow that is required for building great, robust data pipelines. If you want to watch the entire video on youtube you can watch it here &mdash;&nbsp;https://www.youtube.com/watch?v=q8q3OFFfY6c. In this article I&rsquo;ll be going in detail over the steps you need to follow to complete this project.</p> <p><a href="https://medium.com/@nishasreedharan/data-engineering-twitter-x-airflow-data-pipeline-project-25b76ddfbc1a">Click Here</a></p>
Tags: Airflow Data