Re: Hdfs over RDMA

2011-02-22 Thread Brian Bockelman
On Feb 18, 2011, at 12:32 PM, Rajat Sharma wrote: > Hi Guys > > I am trying to work on HDFS to improve its performance by adding RDMA > functionality to its code. But i cannot find any sort of documentation or > help regarding this topic except some information about Socket Direct > Protocol or

measure the resource usage of each map/reduce task

2011-02-22 Thread bikash sharma
Hi, Is there any way in which we can measure the resource usage of each map/reduce task running? I was trying to use sar utility to track each process resource usage, however it seems these individual map/reduce tasks are not listed as processes when I do ps -ex. Thanks, Bikash

org.apache.hadoop.hdfs.protocol.Block, which class actually holds the data bytes in it?

2011-02-22 Thread Weishung Chung
I have a question about the Block class, where is the actual data bytes? Which class holds the data bytes? Thank you so much :)

class project on Hadoop

2011-02-22 Thread Nikhil Panpalia
Hello everyone, I'm a graduate student at the University of Texas at Austin. I'm looking for a research/implementation based project on Hadoop and I came across the list posted on the wiki page - http://wiki.apache.org/hadoop/ProjectSuggestions. But, this page was last updated in September, 2009.

Re: class project on Hadoop

2011-02-22 Thread Eric Yang
Hi Nikhil, For #2, there is some initial work done in a project called Chukwa. Chukwa is designed to collect hadoop metrics, job log files, hdfs/mr client trace log files, and system metrics. By using those data, it is possible to reconstruct state machines of the health of the hadoop cluste

RE: Specify queue name for a hadoop job

2011-02-22 Thread Viraj Bhat
Hi Shivani, It should be -Dmapred.job.queue.name= Viraj -Original Message- From: Shivani Rao [mailto:sg...@purdue.edu] Sent: Sunday, February 20, 2011 3:40 PM To: common-u...@hadoop.apache.org; common-dev@hadoop.apache.org Subject: Specify queue name for a hadoop job I am having troubl