Run Adhoc queries in DBT with Jinja

<p>DBT Models are great in separating SQL into their own enclosures which can be summoned anytime by Directed A-cyclic Graph code scripted in Python.</p> <p>I have mostly witnessed plain SQL sprinkled with a few built-in DBT methods such as&nbsp;<strong>ref()</strong>&nbsp;and&nbsp;<strong>source()</strong>&nbsp;within the model files. I did wonder if the logic within these models can be changed during run time like we do with DAG code (obviously via Airflow&rsquo;s variables).</p> <p>With SQL injection option out of the window; Conditional block provided by Jinja was the obvious answer.</p> <h1>Jinja?</h1> <p>For Novice users; Jinja is a templating language that references certain variables; values for which are then rendered during runtime! In short, it is possible to advice DBT on how to handle certain conditions that might occur during runtime.</p> <p><img alt="" src="https://miro.medium.com/v2/resize:fit:700/0*83UIrvaFs7_3qgha" style="height:467px; width:700px" /></p> <p>Photo by&nbsp;<a href="https://unsplash.com/@nathangbingle?utm_source=medium&amp;utm_medium=referral" rel="noopener ugc nofollow" target="_blank">Nathan Bingle</a>&nbsp;on&nbsp;<a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral" rel="noopener ugc nofollow" target="_blank">Unsplash</a></p> <p>This&nbsp;<a href="https://jinja.palletsprojects.com/en/3.0.x/templates/#tests" rel="noopener ugc nofollow" target="_blank">link&nbsp;</a>should provide more enlightenment over Jinja templating.</p> <h1>Where do we use this?</h1> <p>There are specific use cases where this trick comes in handy:</p> <ol> <li><strong>Adhoc code execution</strong>: when DBT is expected to run a specific logic only on a special occasion or for a bug that doesn&rsquo;t occur too often. An if-else block here could house code for both the usual as well as the Adhoc logic.</li> <li><strong>Re-instating only certain parts of a dataset</strong>: This could be useful to re-instate only a specific section or partition of the table to cut down on costs or while dealing with with large datasets.</li> </ol> <p><a href="https://christo-lagali.medium.com/run-adhoc-queries-in-dbt-with-jinja-26e352e6b112">Website</a>&nbsp;</p>
Tags: Adhoc Jinja