converting dataframe from one format to another in spark structured streaming

2020-11-26 Thread act_coder
I am creating a spark structured streaming application and I have a streaming dataframe which has the below data in it. { "name":"sensor1", "time":"2020-11-27T01:01:00", "sensorvalue":11.0, "tag1":"tagvalue" } I would like to convert that dataframe into below format. { "name":"sensor1", "value-ar

converting dataframe from one format to another in spark structured streaming

2020-11-26 Thread act_coder
I am creating a spark structured streaming application and I have a streaming dataframe which has the below data in it. { "name":"sensor1", "time":"2020-11-27T01:01:00", "sensorvalue":11.0, "tag1":"tagvalue" } I would like to convert that dataframe into below format. { "name":"sensor1", "value-ar