thanks for the info yang On Mon, Jan 26, 2015 at 2:09 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
> We did a relatively complete prototype but it isn't integrated into the > mainline code yet and there isn't a target release date. There is rather a > lot of testing and compatability work that would have to be done to fully > productionize it. I suspect someone will pick it up in 2015 but I wouldn't > block on it. > > -Jay > > On Mon, Jan 26, 2015 at 1:55 PM, Yang <teddyyyy...@gmail.com> wrote: > > > well I guess this is what I need > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Transactional+Messaging+in+Kafka > > > > anyone knows how far this effort has gone? > > > > thanks > > Yang > > > > On Mon, Jan 26, 2015 at 12:06 PM, Yang <teddyyyy...@gmail.com> wrote: > > > > > we are pulling from a DB, maybe 100k rows at a time. for each row we > send > > > it to kafka. > > > > > > now the problem is , if any errors happen during the db pulling (such > as > > > an earthquake), we stop. > > > next time we wake up. we can't write the same record again into kafka, > > > otherwise there might be over counting. > > > > > > > > > does kafka support a "BATCH mode" with commit ? so basically at the > start > > > of my session, I declare "start transaction", and after all records > have > > > been pulled and sent to kafka, I declare "commit" . > > > > > > Thanks > > > Yang > > > > > >
