GCP Dataflow Flex Template Pipeline — Part 2 (Local Testing)
<p>Welcome to the Part 2 of building a data pipeline using the GCP Dataflow service. In this part, we will develop & test our pipeline locally. So, lets jump right in. We will focus on below code for testing and developing the Dataflow pipeline:</p>
<blockquote>
<p><strong><em>Local Testing via Jupyter Notebook: </em></strong><a href="https://github.com/amandeepsaluja/gcp-dataflow-pipeline/blob/master/01-flex-template/local_testing/testing.ipynb" rel="noopener ugc nofollow" target="_blank">https://github.com/amandeepsaluja/gcp-dataflow-pipeline/blob/master/01-flex-template/local_testing/testing.ipynb</a></p>
</blockquote>
<p>Before we start, make sure you have created a new virtual environment and installed the below required packages as mentioned in <a href="https://github.com/amandeepsaluja/gcp-dataflow-pipeline/blob/master/01-flex-template/src/requirements.txt" rel="noopener ugc nofollow" target="_blank">requirements.txt</a></p>
<p><a href="https://medium.com/@saluja.amandeep/gcp-dataflow-flex-template-pipeline-part-2-local-development-5c3bf32923c3"><strong>Visit Now</strong></a></p>