Databricks PySpark Explode and Pivot Columns
<p>The <code>explode</code> function in PySpark is used to transform a column with an array of values into multiple rows. Each row of the resulting DataFrame will contain one element of the original array column.</p>
<p>Here is an example of how to use the <code>explode</code> function:</p>
<pre>
from pyspark.sql.functions import explode</pre>
<pre>
# create a sample DataFrame
data = [("Alice", [1, 2, 3]), ("Bob", [4, 5]), ("Charlie", [6])]
df = spark.createDataFrame(data, ["name", "numbers"])# explode the numbers column
df_exploded = df.select("name", explode("numbers").alias("number"))# show the result
df_exploded.show()</pre>
<p><a href="https://medium.com/@shuklaprashant9264/databricks-pyspark-explode-and-pivot-columns-a8d1b4e713f1"><strong>Website</strong></a></p>