I’m the embodiment of the meme in which a developer spends hours automating a relatively simple task. In other words, while much of the world is increasingly apprehensive of replacing processes with AI, I’m still pro-automation.
Image courtesy of starecat.com.
And while I’ve developed some pipelines outside of work to serve my own needs or to help out a friend, I still struggled with one very important aspect of each ETL build.
If you’re reading this, I imagine you might struggle with the same issue.
Deployment.
To be clear, at work in nearly two years I’ve written thousands of lines of code, created probably 50-ish pipelines and written CI/CD processes ranging from cloud function deployment to Docker image updates.
When I wanted to replicate some of these processes with my own builds, I experienced a lot of failure.