Hi Anbutech in that case you have variable number of columns in output df
and then in csv. it will not be the best way to read csv
On Fri, 15 Nov 2019 at 2:30 pm, anbutech wrote:
> Hello Guha,
>
> The number of keys will be different for each event id.for example if the
> event id:005 it is has
Hello Guha,
The number of keys will be different for each event id.for example if the
event id:005 it is has 10 keys then i have to flatten all those 10 keys in
the final output.here there is no fixed number of keys for each event id.
001 -> 2 keys
002 -> 4 keys
003 -> 5 keys
above event id h
Hi
How do you want your final DF to look like? Is it with all 5 value columns?
Do you have a finite set of columns?
On Fri, Nov 15, 2019 at 4:50 AM anbutech wrote:
> Hello Sir,
>
> I have a scenario to flatten the different combinations of map type(key
> value) in a column called eve_data like
Hello Sir,
I have a scenario to flatten the different combinations of map type(key
value) in a column called eve_data like below:
How do we flatten the map type into proper columns using pyspark
1) Source Dataframe having 2 columns(event id,data)
eve_id,eve_data
001, "k1":"abc",
"k2":"
