Re: Developing and deploying hadoop

2012-02-29 Thread Eric Yang
a) Standard practice is to keep data directory independent of program directory. For example, if the software is installed in /opt/hadoop/hadoop-1.0. Data may be located in /var/hadoop. When new version is available for deployment, it can be deployed to /opt/hadoop/hadoop-2.0 and use the same /v

Re: Developing and deploying hadoop

2012-02-29 Thread Roman Shaposhnik
Hi! One way to deploy Hadoop in a more formal environment is to do it via the Bigtop distribution. Bigtop provides packages and puppet deployment code for most Linux distributions. We try to make the experience of deploying Hadoop as seamless as possible, since our goal can be summarized as "tryin

Re: Developing and deploying hadoop

2012-02-29 Thread Merto Mertek
I would be glad to hear what is your development cycle and how you deploy new features to production cluster... How do you deploy them to the production cluster? With bash scripts and rsync, ant, maven or any other automation tool? I would be thankfull if you could point me to any resource describ

Developing and deploying hadoop

2012-02-13 Thread Merto Mertek
I am interested in some general tips on how to develop and deploy new versions of hadoop. I've been trying to compile a new version of hadoop and place the new jar to the cluster in the lib folder, however it was not picked despite the classpath was explicitly set to the lib folder. I am intereste