Marcelo and Christopher,

 Thanks for your help! The problem turned out to arise from a different part
of the code (we have multiple ObjectMappers), but because I am not very
familiar with Jackson I had thought there was a problem with the Scala
module.

Thank you again,
Kevin

From:  Christopher Currie <christop...@currie.com>
Date:  Wednesday, September 9, 2015 at 10:17 AM
To:  Kevin Chen <kc...@palantir.com>, "dev@spark.apache.org"
<dev@spark.apache.org>
Cc:  Matt Cheah <mch...@palantir.com>, Mingyu Kim <m...@palantir.com>
Subject:  Fwd: Deserializing JSON into Scala objects in Java code

Kevin,

I'm not a Spark dev, but I maintain the Scala module for Jackson. If you're
continuing to have issues with parsing JSON using the Spark Scala datatypes,
let me know or chime in on the jackson mailing list
(jackson-u...@googlegroups.com) and I'll see what I can do to help.

Christopher Currie

---------- Forwarded message ----------
From: Paul Brown <p...@mult.ifario.us>
Date: Tue, Sep 8, 2015 at 8:58 PM
Subject: Fwd: Deserializing JSON into Scala objects in Java code
To: Christopher Currie <christop...@currie.com>


Passing along. 

---------- Forwarded message ----------
From: Kevin Chen <kc...@palantir.com>
Date: Tuesday, September 8, 2015
Subject: Deserializing JSON into Scala objects in Java code
To: "dev@spark.apache.org" <dev@spark.apache.org>
Cc: Matt Cheah <mch...@palantir.com>, Mingyu Kim <m...@palantir.com>


Hello Spark Devs,

 I am trying to use the new Spark API json endpoints at /api/v1/[path]
(added in SPARK-3454).

 In order to minimize maintenance on our end, I would like to use
Retrofit/Jackson to parse the json directly into the Scala classes in
org/apache/spark/status/api/v1/api.scala (ApplicationInfo,
ApplicationAttemptInfo, etc…). However, Jackson does not seem to know how to
handle Scala Seqs, and will throw an error when trying to parse the
attempts: Seq[ApplicationAttemptInfo] field of ApplicationInfo. Our codebase
is in Java.

 My questions are:
1. Do you have any recommendations on how to easily deserialize Scala
objects from json? For example, do you have any current usage examples of
SPARK-3454 with Java?
2. Alternatively, are you committed to the json formats of /api/v1/path? I
would guess so, because of the ‘v1’, but wanted to confirm. If so, I could
deserialize the json into instances of my own Java classes instead, without
worrying about changing the class structure later due to changes in the
Spark API.
Some further information:
* The error I am getting with Jackson when trying to deserialize the json
into ApplicationInfo is Caused by:
com.fasterxml.jackson.databind.JsonMappingException: Can not construct
instance of scala.collection.Seq, problem: abstract types either need to be
mapped to concrete types, have custom deserializer, or be instantiated with
additional type information
* I tried using Jackson’s DefaultScalaModule, which seems to have support
for Scala Seqs, but got no luck.
* Deserialization works if the Scala class does not have any Seq fields, and
works if the fields are Java Lists instead of Seqs.
Thanks very much for your help!
Kevin Chen




-- 
(Sent from mobile. Pardon brevity.)



Attachment: smime.p7s
Description: S/MIME cryptographic signature

Reply via email to