I would walk through a Spark tutorial in Scala. It will be the best way to
learn this.
In brief though, a Scala case class is like a Java bean / pojo but has a
more concise syntax (no getters/setters).
case class Person(firstName: String, lastName: String, age: Int)
Thanks,
Andy.
--
Andy Gro
Honestly, moving to Scala and using case classes is the path of least
resistance in the long term.
Thanks,
Andy.
--
Andy Grove
Chief Architect
AgilData - Simple Streaming SQL that Scales
www.agildata.com
On Wed, Jan 20, 2016 at 10:19 AM, Raghu Ganti wrote:
> Thanks for your reply, Andy.
>
Ah, OK! I am a novice to Scala - will take a look at Scala case classes. It
would be awesome if you can provide some pointers.
Thanks,
Raghu
On Wed, Jan 20, 2016 at 12:25 PM, Andy Grove
wrote:
> I'm talking about implementing CustomerRecord as a scala case class,
> rather than as a Java class.
I'm talking about implementing CustomerRecord as a scala case class, rather
than as a Java class. Scala case classes implement the scala.Product trait,
which Catalyst is looking for.
Thanks,
Andy.
--
Andy Grove
Chief Architect
AgilData - Simple Streaming SQL that Scales
www.agildata.com
On W
Is it not internal to the Catalyst implementation? I should not be
modifying the Spark source to get things to work, do I? :-)
On Wed, Jan 20, 2016 at 12:21 PM, Raghu Ganti wrote:
> Case classes where?
>
> On Wed, Jan 20, 2016 at 12:21 PM, Andy Grove
> wrote:
>
>> Honestly, moving to Scala and
Case classes where?
On Wed, Jan 20, 2016 at 12:21 PM, Andy Grove
wrote:
> Honestly, moving to Scala and using case classes is the path of least
> resistance in the long term.
>
>
>
> Thanks,
>
> Andy.
>
> --
>
> Andy Grove
> Chief Architect
> AgilData - Simple Streaming SQL that Scales
> www.agi
Thanks for your reply, Andy.
Yes, that is what I concluded based on the Stack trace. The problem is
stemming from Java implementation of generics, but I thought this will go
away if you compiled against Java 1.8, which solves the issues of proper
generic implementation.
Any ideas?
Also, are you
Catalyst is expecting a class that implements scala.Row or scala.Product
and is instead finding a Java class. I've run into this issue a number of
times. Dataframe doesn't work so well with Java. Here's a blog post with
more information on this:
http://www.agildata.com/apache-spark-rdd-vs-datafram
