Re: Inserting files to Cassandra timeouts

2010-04-30 Thread Jonathan Ellis
compaction starts but never finishes. are you inserting all these files into the same row? don't do that. On Fri, Apr 30, 2010 at 3:04 AM, Spacejatsi wrote: > I ran again the test, inserting 64 files (15-25MB per file) with 2 threads > inserting file file at the time. > First 30 files goes rel

Re: Inserting files to Cassandra timeouts

2010-04-30 Thread Spacejatsi
I ran again the test, inserting 64 files (15-25MB per file) with 2 threads inserting file file at the time. First 30 files goes relatively fast in, but then it jams, and finally timeouts. This tpstats is taken when the first timeout came. I also tested to split the files max of 5 mb per file. T

Re: Inserting files to Cassandra timeouts

2010-04-29 Thread Jonathan Ellis
are you seeing memtable flushes and compactions in the log? what does tpstats look like when it's timing out? spending 2000ms on GC every 50s indicates that it's not GC causing your problem. (especially when all of them are ParNew, which are completely non-blocking to other threads) On Wed, Apr

Re: Inserting files to Cassandra timeouts

2010-04-28 Thread Jussi P?öri
I was thinking this too, but I think that the overall insert amount is not that big. Data is basically map data, and the files are map tiles, which I can easily make smaller. We are currently using this data from multiple nodes(GRID), but we want to get rid off the files system hassle(basically sam

Re: Inserting files to Cassandra timeouts

2010-04-28 Thread Schubert Zhang
I think your file (as cassandra column value) is too large. And I also think Cassandra is not good at store files. On Wed, Apr 28, 2010 at 10:24 PM, Jussi P?öri wrote: > new try, previous went to wrong place... > > Hi all, > > i'm trying to run a scenario of adding files from specific folder to >