Thanks Nitin…the mysql replication is a good point Regards sanjay From: Nitin Pawar <nitinpawar...@gmail.com<mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Date: Wednesday, March 27, 2013 11:36 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: Re: MySQL instance on hadoop name node server in production ?
few writes understood but many reads will depends on how big your metastore will grow. also what size of hdfs you are looking at that will play a huge factor in this cause namenode will contain the FSImage in memory so you need a larger memory given to namenode. Potentially if its a live cluster then I guess your namenode will be under heavy loads if you are setting up a production system. i would still suggest for metastore you setup a couple of VMs with mysql replication enabled. so the reads can be distributed with load balancers etc On Thu, Mar 28, 2013 at 12:00 AM, Sanjay Subramanian <sanjay.subraman...@wizecommerce.com<mailto:sanjay.subraman...@wizecommerce.com>> wrote: Thanks Nitin. The mysql instance is for hive metastore only so few writes, many reads. Regards sanjay From: Nitin Pawar <nitinpawar...@gmail.com<mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Date: Wednesday, March 27, 2013 11:13 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: Re: MySQL instance on hadoop name node server in production ? ideally when you setup namenode and jobtracker in any hadoop cluster on production those are normally dedicated to that purpose only. Depending on how frequently you are going to run queries and how much data the hdfs is going to hold is key factor in deciding this. On Wed, Mar 27, 2013 at 11:32 PM, Sanjay Subramanian <sanjay.subraman...@wizecommerce.com<mailto:sanjay.subraman...@wizecommerce.com>> wrote: Hi all I am planning to install mysql server (as hive metastore) on the same box as my name node. My name node has 16GB RAM and hopefully I can get 2TB Any problems with mysql on the dame node as name node ? Thanks sanjay CONFIDENTIALITY NOTICE ====================== This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator. -- Nitin Pawar CONFIDENTIALITY NOTICE ====================== This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator. -- Nitin Pawar CONFIDENTIALITY NOTICE ====================== This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
