Fwd: Hadoop use direct I/O in Linux?

2011-01-03 Thread Da Zheng
Hello, I don't know which mailing list is better for this question, so I like to forward my questions to this mailing list. If no one is thinking of doing direct IO in Hadoop, I will do it myself. I have located the code, but the thing is that I'm not familiar with the environment of compili

Fwd: Hadoop use direct I/O in Linux?

2011-01-03 Thread Da Zheng
Hello, I don't know which mailing list is better for this question, so I like to forward my questions to this mailing list. If no one is thinking of doing direct IO in Hadoop, I will do it myself. I have located the code, but the thing is that I'm not familiar with the environment of compili

Re: Hadoop use direct I/O in Linux?

2011-01-03 Thread Brian Bockelman
Hi Da, It's not immediately clear to me the size of the benefit versus the costs. Two cases where one normally thinks about direct I/O are: 1) The usage scenario is a cache anti-pattern. This will be true for some Hadoop use cases (MapReduce), not true for some others. - http://www.jeffshafe

[jira] Created: (HADOOP-7083) Allow SecureIO to be disabled for developer workstations

2011-01-03 Thread Todd Lipcon (JIRA)
Allow SecureIO to be disabled for developer workstations Key: HADOOP-7083 URL: https://issues.apache.org/jira/browse/HADOOP-7083 Project: Hadoop Common Issue Type: Improvement

Re: Hadoop use direct I/O in Linux?

2011-01-03 Thread Christopher Smith
On Mon, Jan 3, 2011 at 11:40 AM, Brian Bockelman wrote: > It's not immediately clear to me the size of the benefit versus the costs. > Two cases where one normally thinks about direct I/O are: > 1) The usage scenario is a cache anti-pattern. This will be true for some > Hadoop use cases (MapRedu

[jira] Created: (HADOOP-7084) Remove java5 dependencies from site's build

2011-01-03 Thread Konstantin Boudnik (JIRA)
Remove java5 dependencies from site's build --- Key: HADOOP-7084 URL: https://issues.apache.org/jira/browse/HADOOP-7084 Project: Hadoop Common Issue Type: Improvement Components: build

Re: Hadoop use direct I/O in Linux?

2011-01-03 Thread Brian Bockelman
On Jan 3, 2011, at 5:17 PM, Christopher Smith wrote: > On Mon, Jan 3, 2011 at 11:40 AM, Brian Bockelman wrote: > >> It's not immediately clear to me the size of the benefit versus the costs. >> Two cases where one normally thinks about direct I/O are: >> 1) The usage scenario is a cache anti-pat

Re: Hadoop use direct I/O in Linux?

2011-01-03 Thread Christopher Smith
On Mon, Jan 3, 2011 at 5:05 PM, Brian Bockelman wrote: > On Jan 3, 2011, at 5:17 PM, Christopher Smith wrote: > > On Mon, Jan 3, 2011 at 11:40 AM, Brian Bockelman >wrote: > > > >> It's not immediately clear to me the size of the benefit versus the > costs. > >> Two cases where one normally think

Re: Hadoop use direct I/O in Linux?

2011-01-03 Thread Brian Bockelman
On Jan 3, 2011, at 8:47 PM, Christopher Smith wrote: > On Mon, Jan 3, 2011 at 5:05 PM, Brian Bockelman wrote: > >> On Jan 3, 2011, at 5:17 PM, Christopher Smith wrote: >>> On Mon, Jan 3, 2011 at 11:40 AM, Brian Bockelman >> wrote: >>> It's not immediately clear to me the size of the benefi