DFS client does not write the data to local disk first. Instead, it streams data directly to the datanodes in the write pipeline. I will update the document.
On Sun, Sep 26, 2010 at 5:21 AM, Gokulakannan M <gok...@huawei.com> wrote: > Hi, > > > > Is staging still used in hdfs when writing the data? This doubt > arose when I was going through the hdfs documents. > > ref : > http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html#Staging > > > > I believe dfsclient *does not cache the datablock to local fs*(as > the document says) but it does streaming of 64KB packets to the datanode > and caches the > > packets of current block only in memory via dataqueue and > ackqueue. > > > > Is the document needs to be corrected or my understanding is > wrong? > > > > Thanks, > > Gokul > > > > > > > > > *************************************************************************************** > This e-mail and attachments contain confidential information from HUAWEI, > which is intended only for the person or entity whose address is listed > above. Any use of the information contained herein in any way (including, > but not limited to, total or partial disclosure, reproduction, or > dissemination) by persons other than the intended recipient's) is > prohibited. If you receive this e-mail in error, please notify the sender by > phone or email immediately and delete it! > > > -- Connect to me at http://www.facebook.com/dhruba
