Hey everybody.
I am working on a recommender system that uses Hadoop for generating Item Similarities. Since mahout has the RecommenderJob example, I tried to run the recommender in my hadoop (pseudo-) cluster. I noticed that on the MapReduce job : CoocurencesMapper - SimilarityReducer, there is a big overhead (approximately 6 minutes). When the mapping ends, there is a huge time gap until the reducer initiates, but one CPU was fully loaded during that time. Why is this happening? Is there an I/O operation hidden on this time gap?

Thank you

Reply via email to