Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-09 Thread Rajiv Chittajallu
On Aug 9, 2013, at 14:43, "Roman Shaposhnik" wrote: > On Fri, Aug 9, 2013 at 2:28 PM, Doug Cutting wrote: >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur wrote: >>> pinging again, I need help from somebody with sudo access to the hadoop >>> jenkins boxes to do this or to get sudo access

Re: Hadoop build slaves software

2013-01-04 Thread Rajiv Chittajallu
t;> >>> We could then download that script onto the build machines and install >>> into something like /opt/hadoop-toolchain/ >>> AFAIK the only real dependencies we have where the Ubuntu packages are >>> too old are protoc and maven, so shouldnt be too tough. &g

Re: Hadoop build slaves software

2013-01-04 Thread Rajiv Chittajallu
asf008 has been up for a while. It was probably just added as a slave. All the dependencies should probably be installed in a build_prefix, to avoid conflict to OS specific packages and allows multiple projects to build on the same machines. This is an better alternative to provisioning vms for un

Re: Moving TB of data from NFS to HDFS

2012-01-25 Thread Rajiv Chittajallu
You will more likely be hitting NFS server limits way before you can see any noticible issues with HDFS. Writes to a file are sequential. Total throughput for your transfer is dependent on number of files and the rate at which files can be read from NFS. If the total data set is split across re