Oh, actually, we do not support MapType provided by the schema given to
jsonRDD at the moment (my bad..). Daniel, you need to wait for the patch of
4476 (I should have one soon).
Thanks,
Yin
On Wed, Nov 19, 2014 at 2:32 PM, Daniel Haviv wrote:
> Thank you Michael
> I will try it out tomorrow
>
Thank you Michael
I will try it out tomorrow
Daniel
> On 19 בנוב׳ 2014, at 21:07, Michael Armbrust wrote:
>
> You can override the schema inference by passing a schema as the second
> argument to jsonRDD, however thats not a super elegant solution. We are
> considering one option to make thi
You can override the schema inference by passing a schema as the second
argument to jsonRDD, however thats not a super elegant solution. We are
considering one option to make this easier here:
https://issues.apache.org/jira/browse/SPARK-4476
On Tue, Nov 18, 2014 at 11:06 PM, Akhil Das
wrote:
>
Something like this?
val map_rdd = json_rdd.map(json => {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
val myMap = mapper.readValue[Map[String,String]](json)
myMap
})
Thanks
Best Regards
On Wed, Nov 19, 2014 at
Hi,
I'm loading a json file into a RDD and then save that RDD as parquet.
One of the fields is a map of keys and values but it is being translated and
stored as a struct.
How can I convert the field into a map?
Thanks,
Daniel
