You can do this using the explode function defined in org.apache.spark.sql.functions.
Here is some example code <https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1023043053387187/1075277772969587/2840265927289860/fca8d7c609.html> . On Wed, Feb 24, 2016 at 3:06 PM, Anthony Brew <atb...@gmail.com> wrote: > Hi, > I have a Dataframe containing a column with a map Map[A,B] with > multiple values. I want to explode the key,value pairs in the map into a > new column, actually planing to create 2 new cols. > > My plan had been > > - explode "input": Map[K,V] to "temp":Iterable[Map[K,V]] > - new col temp to temp.key > - new col temp to temp.value > - drop temp > > But I am failing at the first hurdle. > > For example my data looks a bit like like.... > > scala> test.show() > +----------------------------+------------------------------------------+ > | id | brand > | > +----------------------------+------------------------------------------+ > |a02d1fa5d87dce6a7...|Map(Vans -> 1, Versace ->2, ...| > > > but I want to get to > > scala> test.show() > +-----------------------------+------------------------------------------+ > | id | brand_key | brand_count | > +-----------------------------+------------------------------------------+ > | a02d1fa5d87dce6a7...| Vans | 1 | > | a02d1fa5d87dce6a7...| Versace | 2 | > > > Any suggestions would be appreciated. > > Thanks, > Anthony > > >
