Hi, What exactly do you mean by 'commodity' hardware?
I mean if you have the budget to build out a 10,000 node cloud, don't you think you'd have the money to beef up your job tracker? With respect to the job tracker, its more memory intensive than disk, right? So how many nodes can you run in a cluster where the job tracker has 96GB of memory? (Which I think is a 'commodity' box maxed out) -Mike -----Original Message----- From: kartik [mailto:ekar...@huawei.com] Sent: Wednesday, March 03, 2010 2:55 AM To: common-dev@hadoop.apache.org Subject: RE: Jobtrackers scalability hai geeks, The functionality of the job tracker is to divide the job among the tasktrackers and monitor the progress of the task, and even the job also... Do we address a situation where we have similar commodity hardware deployed suppose of 2GB RAM and there are some 10000 nodes and we have to submit a single batch job ... (That is think of a scenario where we do not have any high end computer which alone would be configured as a JobTracker ).. Which requires to coordinate all the 10000 TaskTrackers.. I think this will surely degrade the performance of the JobTracker.... For applications now-a-days it is impractical to think of a similar scenario, but just to confirm whether the job tracker will run under such scenarios.. Was just curious to know what would be the limit in terms of RAM size and CPU speed of a Job Tracker to support such a application. Thanks and Regards Kartik **************************************************************************** *********** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient's) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files.
