Thank you for the info, is there a way to get all keys of JSON, so that i
can create a dataframe with json keys, as below,
fieldsDataframe.withColumn("data" ,
functions.get_json_object($"RecordString", "$.id")) this is for appending
a single column in dataframe with id key.
I would like to aut
You can use get
On Thu, 7 Dec 2017 at 10:39 am, satyajit vegesna
wrote:
> Does spark support automatic detection of schema from a json string in a
> dataframe.
>
> I am trying to parse a json string and do some transofrmations on to it
> (would like append new columns to the dataframe) , from th
On Thu, 7 Dec 2017 at 11:37 am, ayan guha wrote:
> You can use get_json function
>
> On Thu, 7 Dec 2017 at 10:39 am, satyajit vegesna <
> satyajit.apas...@gmail.com> wrote:
>
>> Does spark support automatic detection of schema from a json string in a
>> dataframe.
>>
>> I am trying to parse a jso
I use Play json, may be its very famous.
If you would like to try below is the sbt dependency
"com.typesafe.play" % "play-json_2.10" % "2.2.1",
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Json-parsing-library-for-Spark-Streaming-tp24016p24204.html
Sent
json4s is used by https://github.com/hammerlab/spark-json-relay
See the other thread on 'Spree'
FYI
On Mon, Jul 27, 2015 at 6:07 PM, swetha wrote:
> Hi,
>
> What is the proper Json parsing library to use in Spark Streaming?
> Currently
> I am trying to use Gson library in a Java class and call
Hi,
I usually use pattern matching for that, like
json \ "key" match { case JInt(i) => i; case _ => 0 /* default value */ }
Tobias
On Thu, Jun 12, 2014 at 7:39 AM, Michael Cutler wrote:
> Hello,
>
> You're absolutely right, the syntax you're using is returning the json4s
> value objects, not na
Hello,
You're absolutely right, the syntax you're using is returning the json4s
value objects, not native types like Int, Long etc. fix that problem and
then everything else (filters) will work as you expect. This is a short
snippet of a larger example: [1]
val lines = sc.textFile("likes.jso
