Re: Task's map reading more record than CFIF's inputSplitSize

2011-09-07 Thread Mick Semb Wever
> > 3 map tasks (from 4013) is still running after read 25 million rows. > > Can this be a bug in StorageService.getSplits(..) ? > > getSplits looks pretty foolproof to me but I guess we'd need to add > more debug logging to rule out a bug there for sure. > > I guess the main alternative would b

Re: Task's map reading more record than CFIF's inputSplitSize

2011-09-07 Thread Jonathan Ellis
getSplits looks pretty foolproof to me but I guess we'd need to add more debug logging to rule out a bug there for sure. I guess the main alternative would be a bug in the recordreader paging. On Wed, Sep 7, 2011 at 6:35 AM, Mck wrote: > Cassandra-0.8.4 w/ ByteOrderedPartitioner > > CFIF's input

Task's map reading more record than CFIF's inputSplitSize

2011-09-07 Thread Mck
Cassandra-0.8.4 w/ ByteOrderedPartitioner CFIF's inputSplitSize=196608 3 map tasks (from 4013) is still running after read 25 million rows. Can this be a bug in StorageService.getSplits(..) ? With this data I've had general headache with using tokens that are longer than usual (and trying to mo