Hi Alexander,
The stack trace is a little misleading here: all of the time is spent in
MemoryStore, but that's because MemoryStore is unrolling an iterator (note
the iterator.next()) call so that it can be stored in-memory. Essentially
all of the computation for the tasks happens as part of that
Hi,
I am working on artificial neural networks for Spark. It is solved with
Gradient Descent, so each step the data is read, sum of gradients is calculated
for each data partition (on each worker), aggregated (on the driver) and
broadcasted back. I noticed that the gradient computation time is
