Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-20 Thread Vinayakumar B
Yes stack.. Regards, Vinay On Sep 20, 2014 1:03 AM, "Stack" wrote: > On Thu, Sep 18, 2014 at 12:48 AM, Vinayakumar B > wrote: > > > Hi all, > > > > Currently *DFSInputStream *doen't allow reading a write-inprogress file, > > once all written bytes, by the time of opening an input stream, are rea

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-19 Thread Stack
On Thu, Sep 18, 2014 at 12:48 AM, Vinayakumar B wrote: > Hi all, > > Currently *DFSInputStream *doen't allow reading a write-inprogress file, > once all written bytes, by the time of opening an input stream, are read. > > To read further update on the same file, needs to be read by opening > anot

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-19 Thread Vinayakumar B
On Fri, Sep 19, 2014 at 10:32 PM, Colin McCabe wrote: > > On Fri, Sep 19, 2014 at 9:41 AM, Vinayakumar B > wrote: > > Thanks Colin for the detailed explanation. > > > > On Fri, Sep 19, 2014 at 9:38 PM, Colin McCabe > > wrote: > >> > >> On Thu, Sep 18, 2014 at 11:06 AM, Vinayakumar B > > wrote:

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-19 Thread Colin McCabe
On Fri, Sep 19, 2014 at 9:41 AM, Vinayakumar B wrote: > Thanks Colin for the detailed explanation. > > On Fri, Sep 19, 2014 at 9:38 PM, Colin McCabe > wrote: >> >> On Thu, Sep 18, 2014 at 11:06 AM, Vinayakumar B > wrote: >> > bq. I don't know about the merits of this, but I do know that native >

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-19 Thread Vinayakumar B
Thanks Colin for the detailed explanation. On Fri, Sep 19, 2014 at 9:38 PM, Colin McCabe wrote: > > On Thu, Sep 18, 2014 at 11:06 AM, Vinayakumar B wrote: > > bq. I don't know about the merits of this, but I do know that native > > filesystems > > implement this by not raising the EOF exception

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-19 Thread Colin McCabe
On Thu, Sep 18, 2014 at 11:06 AM, Vinayakumar B wrote: > bq. I don't know about the merits of this, but I do know that native > filesystems > implement this by not raising the EOF exception on the seek() but only on > the read ... some of the non-HDFS filesystems Hadoop support work this way. Pre

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Vinayakumar B
bq. I don't know about the merits of this, but I do know that native filesystems implement this by not raising the EOF exception on the seek() but only on the read ... some of the non-HDFS filesystems Hadoop support work this way. I agree with you steve. read only will throw EOF. But when we know

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Steve Loughran
I don't know about the merits of this, but I do know that native filesystems implement this by not raising the EOF exception on the seek() but only on the read ... some of the non-HDFS filesystems Hadoop support work this way. -I haven't ever looked to see what code assumes that it is the seek tha

[DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Vinayakumar B
Hi all, Currently *DFSInputStream *doen't allow reading a write-inprogress file, once all written bytes, by the time of opening an input stream, are read. To read further update on the same file, needs to be read by opening another stream to the same file again. Instead how about refreshing leng