We are currently pulling out the JSON columns, passing them through read.json, and then joining them back onto the initial DF so something like from_json would be a nice quality of life improvement for us.
On Wed, Sep 28, 2016 at 10:52 AM, Michael Armbrust <mich...@databricks.com> wrote: > Spark SQL has great support for reading text files that contain JSON data. > However, in many cases the JSON data is just one column amongst others. > This is particularly true when reading from sources such as Kafka. This PR > <https://github.com/apache/spark/pull/15274> adds a new functions > from_json that converts a string column into a nested StructType with a > user specified schema, using the same internal logic as the json Data > Source. > > Would love to hear any comments / suggestions. > > Michael >
