Hi,
Im confused on what is the right way to use broadcast variables from java.
My code looks something like this:
Map<> val = //build Map to be broadcast
Broadcast<Map<>> broadastVar = sc.broadcast(val);
sc.textFile(...).map(new SomeFunction()) {
//Do something here using broadcastVar
}
My question is, should I pass the broadcastVar to the SomeFunction as a
constructor parameter that it can keep around as an instance variable i.e.
sc.textFile(...).map(new SomeFunction(broadcastVar)) {
//Do something here using broadcastVar
}
class SomeFunction extends Function<T> {
public SomeFunction(Broadcast<Map<>> var) {
this.var = var
}
public T call() {
//Do something
}
}
Is above the right way to utilize broadcast Variables when not using
anonymous inner classes as functions?
--
Regards,
Puneet
