For torrent broadcast, data are read directly through the block manager:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L167
On Thu, Apr 9, 2015 at 7:27 AM, Zoltán Zvara wrote:
> Thanks! I've found the fetcher! Is there any ot
Thanks! I've found the fetcher! Is there any other places and cases where
blocks are traveled through network?
Zvara Zoltán
mail, hangout, skype: zoltan.zv...@gmail.com
mobile, viber: +36203129543
bank: 10918001-0021-50480008
address: Hungary, 2475 Kápolnásnyék, Kossuth 6/a
elte: HSKSJZ
Take a look at the following two files:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/shuffle/hash/BlockStoreShuffleFetcher.scala
and
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
On
