Sync Delta Tables Stored in DBFS (Managed or External) To Unity Catalog
<p>Databricks recommends for these use cases to <strong>recreate </strong>the tables in unity catalog and copy the data using a <a href="https://docs.databricks.com/delta/clone.html#clone-types-1" rel="noopener ugc nofollow" target="_blank"><strong><em>deep </em></strong></a><strong>clone</strong>. But if you are a person who’s not a fan of copying/moving data, you can follow my humble demo. I’ll show you how I get to work around.</p>
<p>The second type of delta tables cloning is called <a href="https://docs.databricks.com/delta/clone.html#clone-types-1" rel="noopener ugc nofollow" target="_blank"><strong><em>shallow</em></strong></a><strong><em> </em>clone<em>, </em></strong>which is way more cheaper to create because it does not copy the data files. However shallow cloning delta tables directly to unity catalog is not yet supported.</p>
<p><img alt="" src="https://miro.medium.com/v2/resize:fit:770/1*UGNtOvzIyO2HeJMb0nw_Tg.png" style="height:64px; width:700px" /></p>
<p>Bypassing this limitation means doing the shallow clone to a bridge external location and after synchronizing the result of the last shallow clone to the unity catalog.</p>
<p><a href="https://medium.com/@wahidatoui/sync-delta-tables-stored-in-dbfs-managed-or-external-to-unity-catalog-1fe29a3ae61b"><strong>Visit Now</strong></a></p>