Hey everybody.
I am working on a recommender system that uses Hadoop for generating
Item Similarities. Since mahout has the RecommenderJob example, I tried
to run the recommender in my hadoop (pseudo-) cluster. I noticed that on
the MapReduce job : CoocurencesMapper - SimilarityReducer, there is a
big overhead (approximately 6 minutes). When the mapping ends, there is
a huge time gap until the reducer initiates, but one CPU was fully
loaded during that time. Why is this happening? Is there an I/O
operation hidden on this time gap?
Thank you
- RecommenderJob Hadoop execution times Nikolaos Romanos Katsipoulakis
-