Github user colorant commented on the pull request:
https://github.com/apache/spark/pull/1241#issuecomment-48559404
@rxin addressed comments. Among them:
1. Rename HashShuffleBlockManager to FileShuffleBlockManager since it
mainly handle block in File way. Though say sort based shuffle manager's own
ShuffleBlockManager might also use File, but it might use another more specific
name.
2. I still keep the shuffleManager inside blockManager, the main reason is
that the netty staff need to access shuffleManager upon blockManager init,
while in this time SparkEnv not fully created yet, you can not access
shuffleManager though env.get approaching. Then you had to pass in
shuffleManager to blockManager as a parameter which further complicate
blockManager's interface. so to keep it simple, I think we might keep it this
way until one day we did get rid of netty staff. And more, since the current
BlockManager's logic depends on shuffleManager a lot, and there are no e.g.
standalone "shuffleManagerMaster" things, so all shuffle data fetch works go
through blockManager's interface firstly anyway, so it seems to me not totally
unreasonable to put shuffleManager inside BlockManager?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
