You could create a copy of the variable inside your "Parse" class;
that way it would be serialized with the instance you create when
calling map() below.
On Tue, Aug 12, 2014 at 10:56 AM, Sunny Khatri <sunny.k...@gmail.com> wrote:
> Are there any other workarounds that could be used to pass in the values
> from someVariable to the transformation function ?
>
>
> On Tue, Aug 12, 2014 at 10:48 AM, Sean Owen <so...@cloudera.com> wrote:
>>
>> I don't think static members are going to be serialized in the
>> closure? the instance of Parse will be looking at its local
>> SampleOuterClass, which is maybe not initialized on the remote JVM.
>>
>> On Tue, Aug 12, 2014 at 6:02 PM, Sunny Khatri <sunny.k...@gmail.com>
>> wrote:
>> > I have a class defining an inner static class (map function). The inner
>> > class tries to refer the variable instantiated in the outside class,
>> > which
>> > results in a NullPointerException. Sample Code as follows:
>> >
>> > class SampleOuterClass {
>> >
>> > private static ArrayList<String> someVariable;
>> >
>> > SampleOuterClass() {
>> > // initialize someVariable
>> > }
>> >
>> > public static class Parse implements Function<...> {
>> > public TypeReturn call (...) {
>> > // Try using someVariable: Raises NullPointerException
>> > }
>> > }
>> >
>> > public void run() {
>> > RDD<> rdd = data.map(new Parse()).rdd()
>> > }
>> > }
>> >
>> > Am I missing something with how Closures work with Spark or something
>> > else
>> > is wrong ?
>> >
>> > Thanks
>> > Sunny
>> >
>> >
>
>
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
