Hi Gerard, first, thanks for your explanations regarding the jar files!
On Thu, May 22, 2014 at 12:32 AM, Gerard Maas <gerard.m...@gmail.com> wrote: > I was discussing it with my fellow Sparkers here and I totally overlooked > the fact that you need the class files to de-serialize the closures (or > whatever) on the workers, so you always need the jar file delivered to the > workers in order for it to work. So the closure as a function is serialized, sent across the wire, deserialized there, and *still* you need the class files? (I am not sure I understand what is actually sent over the network then. Does that serialization only contain the values that I close over?) If I understand correctly what you are saying, then the documentation at https://people.apache.org/~pwendell/catalyst-docs/running-on-mesos.html (list item 8) needs to be extended quite a bit, right? Thanks Tobias
