Re: Flatten log data Using Pyspark

2019-12-02 Thread Gourav Sengupta
Why do you want to use UDF? Regards, Gourav On Sat, Nov 30, 2019 at 3:06 AM anbutech wrote: > Hi, > > I have a raw source data frame having 2 columns as below > > timestamp > 2019-11-29 9:30:45 > > message_log > > <123>NOV 29 10:20:35 ips01 sfids: connection: > tcp,bytes:104,user:unknown,url:un

Flatten log data Using Pyspark

2019-11-29 Thread anbutech
Hi, I have a raw source data frame having 2 columns as below timestamp 2019-11-29 9:30:45 message_log <123>NOV 29 10:20:35 ips01 sfids: connection: tcp,bytes:104,user:unknown,url:unknown,host:127.0.0.1 how do we break above each key value as separate columns using