GCP Dataflow Flex Template Pipeline — Part 6 (Troubleshooting)

Well, here we are. The final article on creating end to end pipeline for GCP Data Flex Template. In case you missed the previous 5, I have hyperlinked those towards the end of this article. In this part, I will list down all the issues I came across while developing this pipeline. I hope it saves you some hours :) <blockquote> Before we go ahead, I want to say that nothing is perfect in life and data pipeline ;) If you go through all, you might come across various things that can be optimized or better deleted. Well, there is no one way of doing it. Do I really needed Terraform and GitHub Actions? Hell no. But I got my hands dirty and did it in order to learn. If you have any feedback, please let me know :) I appreciate you going through the process with me. </blockquote> <a href="https://medium.com/@saluja.amandeep/gcp-dataflow-flex-template-pipeline-part-6-troubleshooting-258aca620350">Click Here</a>