Re: How to Exploding a Map[String,Int] column in a DataFrame (Scala)

-+-+---+ >> |id |myMap |brand_key|brand_count| >> +-++-+---+ >> |a02d1fa5d87dce6a7|Map(Vans -> 1, Versace -> 2)|Vans |1 | >> |a02d1fa5d8

How to Exploding a Map[String,Int] column in a DataFrame (Scala)

Hi, I have a Dataframe containing a column with a map Map[A,B] with multiple values. I want to explode the key,value pairs in the map into a new column, actually planing to create 2 new cols. My plan had been - explode "input": Map[K,V] to "temp":Iterable[Map[K,V]] - new col temp to temp.key

Re: Option[Long] parameter in case class parsed from JSON DataFrame failing when key not present in JSON

stomerEvent] > > just a pointer, I haven't tested this. > regards, > --Jakob > > On Mon, Feb 22, 2016 at 12:17 PM, Jorge Machado wrote: > > Hi Anthony, > > > > I try the code on my self. I think it is on the jsonStr: > > > > I do it with : val jsonStr = """{"

Option[Long] parameter in case class parsed from JSON DataFrame failing when key not present in JSON

Hi, I'm trying to parse JSON data into a case class using the DataFrame.as[] function, nut I am hitting an unusual error and the interweb isnt solving my pain so thought I would reach out for help. Ive truncated my code a little here to make it readable, but the error is full My case class lo

Persisting closed sessions to external store inside updateStateByKey

Hi, I'm working on storing effectively what is a session that receives its close event using spark streaming, by using updateStateByKey. private static Function2, Optional, Optional> COLLECTED_SESSION = (newItems, current) -> { SessionUpdate returnValue = current.orNull();