Dear All,
I am trying to understand how exactly spark network module works. Looking at
Netty package, I would like to intercept every server response for block
fetch. As I understood the place which is responsible for sending remote
blocks is "TransportRequestHandler.processFetchRequest". Im tryin
I'm modifying hdfs module inside hadoop, and would like the see the
reflection while i'm running spark on top of it, but I still see the native
hadoop behaviour. I've checked and saw Spark is building a really fat jar
file, which contains all hadoop classes (using hadoop profile defined in
maven),
Thanks Ted for reply.
But this is not what I want. This would tell spark to read hadoop dependency
from maven repository, which is the original version of hadoop. I myslef is
modifying the hadoop code, and wanted to include them inside the spark fat
jar. "Spark-Class" would run slaves with the fat
