Re: data node reads blocks sequentially from disk

We also issue explicit readahead via fadvise since 2011 or so, so the typical io sizes hitting the device are large enough to max out the throughput, at least for typical spinning disks. Todd On Wed, Mar 11, 2020, 9:46 AM Kihwal Lee wrote: > When Datanode was initially designed, Linux AIO was s

Re: data node reads blocks sequentially from disk

When Datanode was initially designed, Linux AIO was still early in its adoption. Kernel support was there and the libraries were almost there. No java support, of course. We would have to write a lot of native code for it and use JNI. Also, AIO means bypassing kernel page cache since you are doing

Re: data node reads blocks sequentially from disk

Hi David, We talked a bit about a similar topic on DataNode sockets a while back. Any feedback on the DataNode disk access? On Thu, Mar 5, 2020 at 4:16 PM Mania Abdi wrote: > Hello everyone > > I have a question regarding HDFS, data node code version 2.7.2. I have > posted my question as Jira is

data node reads blocks sequentially from disk

Hello everyone I have a question regarding HDFS, data node code version 2.7.2. I have posted my question as Jira issue . I have observed that datanode issues sequential synchronous 64KB reads to local disk and add then send it to use