Perform ???SCD Type 1??? Using ???MERGE??? Operation on Delta Table Using ???SPARK SQL??? and ???PySpark??? in Databricks

What is “Slowly Changing Dimension”?

The “Slowly Changing Dimension” is “One” of the “Very Important Concept” in “Data Warehousing Solution”.

“Slowly Changing Dimension” is the “Change” of “Attribute”, or, the “Value” of the “Entities” over a “Period” of “Time”.
Example:

  • “Consider” that there is a “Delta Table”, called “Person”, which has an “Attribute”, called as “Address”. Now, a “Person” can “Stay” in an “Address” now. But, after “One Year”, the “Same Person” can “Change” the “Address”.
    As a result, the “Latest Value” of the “Address” needs to be “Updated” in the “Address” “Attribute” of the “Person” Table.
  • This is called as the “Slowly Changing Dimension”.

“Different Methods” of “Handling” the “Slowly Changing Dimension”

There are “Various Methods” of “Handling” the “Slowly Changing Dimension” in “Data Warehousing Solution”.

The “Commonly Used” “Three Approaches” to “Handle” the “Slowly Changing Dimension” are -

Click Here

Tags: SCD Type