On Thu, Mar 22, 2012 at 10:25AM, Eli Collins wrote: > On Thu, Mar 22, 2012 at 1:26 AM, Konstantin Shvachko > <shv.had...@gmail.com> wrote: > > Eli, > > > > I went over the entire discussion on the topic, and did not get it. Is > > there a problem with append? We know it does not work in hadoop-1, > > only flush() does. Is there anything wrong with the new append > > (HDFS-265)? If so please file a bug. > > I tested it in Hadoop-0.22 branch it works fine. > > > > I agree with people who were involved with the implementation of the > > new append that the complexity is mainly in > > 1. pipeline recovery > > 2. consistent client reading while writing, and > > 3. hflush() > > Once it is done the append itself, which is reopening of previously > > closed files for adding data, is not complex. > > > > You mentioned it and I agree you indeed should be more involved with > > your customer base. As for eBay, append was of the motivations to work > > on stabilizing 0.22 branch. And there is a lot of use cases which > > require append for our customers. > > Some of them were mentioned in this discussion. > > > > From what I've seen 0.22 isn't ready for production use. Aside from > not supporting critical features like security, it doesn't have a > size-able user-base behind it testing and fixing bugs, etc. All things > I'd imagine an org like eBay would want. I've never gotten a request > to support 0.22 from a customer.
This statement looks like FUD to me, because eBay (and a coupla other shops, as has been stated elsewhere) are using 0.22 in the production and are seemingly happy with that. And employing FUD always means that there's a reason to bring it about. Cos > Thanks, > Eli
