Run Adhoc queries in DBT with Jinja
<p>DBT Models are great in separating SQL into their own enclosures which can be summoned anytime by Directed A-cyclic Graph code scripted in Python.</p>
<p>I have mostly witnessed plain SQL sprinkled with a few built-in DBT methods such as <strong>ref()</strong> and <strong>source()</strong> within the model files. I did wonder if the logic within these models can be changed during run time like we do with DAG code (obviously via Airflow’s variables).</p>
<p>With SQL injection option out of the window; Conditional block provided by Jinja was the obvious answer.</p>
<h1>Jinja?</h1>
<p>For Novice users; Jinja is a templating language that references certain variables; values for which are then rendered during runtime! In short, it is possible to advice DBT on how to handle certain conditions that might occur during runtime.</p>
<p><img alt="" src="https://miro.medium.com/v2/resize:fit:700/0*83UIrvaFs7_3qgha" style="height:467px; width:700px" /></p>
<p>Photo by <a href="https://unsplash.com/@nathangbingle?utm_source=medium&utm_medium=referral" rel="noopener ugc nofollow" target="_blank">Nathan Bingle</a> on <a href="https://unsplash.com/?utm_source=medium&utm_medium=referral" rel="noopener ugc nofollow" target="_blank">Unsplash</a></p>
<p>This <a href="https://jinja.palletsprojects.com/en/3.0.x/templates/#tests" rel="noopener ugc nofollow" target="_blank">link </a>should provide more enlightenment over Jinja templating.</p>
<h1>Where do we use this?</h1>
<p>There are specific use cases where this trick comes in handy:</p>
<ol>
<li><strong>Adhoc code execution</strong>: when DBT is expected to run a specific logic only on a special occasion or for a bug that doesn’t occur too often. An if-else block here could house code for both the usual as well as the Adhoc logic.</li>
<li><strong>Re-instating only certain parts of a dataset</strong>: This could be useful to re-instate only a specific section or partition of the table to cut down on costs or while dealing with with large datasets.</li>
</ol>
<p><a href="https://christo-lagali.medium.com/run-adhoc-queries-in-dbt-with-jinja-26e352e6b112">Website</a> </p>