Sync Delta Tables Stored in DBFS (Managed or External) To Unity Catalog

<p>Databricks recommends for these use cases to&nbsp;<strong>recreate&nbsp;</strong>the tables in unity catalog and copy the data using a&nbsp;<a href="https://docs.databricks.com/delta/clone.html#clone-types-1" rel="noopener ugc nofollow" target="_blank"><strong><em>deep&nbsp;</em></strong></a><strong>clone</strong>. But if you are a person who&rsquo;s not a fan of copying/moving data, you can follow my humble demo. I&rsquo;ll show you how I get to work around.</p> <p>The second type of delta tables cloning is called&nbsp;<a href="https://docs.databricks.com/delta/clone.html#clone-types-1" rel="noopener ugc nofollow" target="_blank"><strong><em>shallow</em></strong></a><strong><em>&nbsp;</em>clone<em>,&nbsp;</em></strong>which is way more cheaper to create because it does not copy the data files. However shallow cloning delta tables directly to unity catalog is not yet supported.</p> <p><img alt="" src="https://miro.medium.com/v2/resize:fit:770/1*UGNtOvzIyO2HeJMb0nw_Tg.png" style="height:64px; width:700px" /></p> <p>Bypassing this limitation means doing the shallow clone to a bridge external location and after synchronizing the result of the last shallow clone to the unity catalog.</p> <p><a href="https://medium.com/@wahidatoui/sync-delta-tables-stored-in-dbfs-managed-or-external-to-unity-catalog-1fe29a3ae61b"><strong>Visit Now</strong></a></p>
Tags: Unity Catalog