Agree to Peter Schuller.
On Sun, Jul 18, 2010 at 8:40 PM, Jonathan Ellis wrote:
> On Sun, Jul 18, 2010 at 2:45 AM, Schubert Zhang wrote:
> > In a heavy inserting (many client threads), the memtable flush (generate
> new
> > sstable) is frequent (e.g. one in 30s).
>
> This is a sign you should i
On Sun, Jul 18, 2010 at 2:45 AM, Schubert Zhang wrote:
> In a heavy inserting (many client threads), the memtable flush (generate new
> sstable) is frequent (e.g. one in 30s).
This is a sign you should increase your memtable thresholds, btw. If
you wrote out larger sstables, there would be less
(adding dev@)
> (2) Can we implement multi-thread compaction?
I think this is the only way to scale. Or at least to implement
concurrent compaction (whether it is by division into threads or not)
of multiple size classes. As long as the worst-case compactions are
significantly slower than best-ba
Benjamin and Jonathan,
It is not difficult to stack thousands of small SSTables.
In a heavy inserting (many client threads), the memtable flush (generate new
sstable) is frequent (e.g. one in 30s).
The compaction only run in a single thread and is CPU bound. Consider the
compactionManager is com
Benjamin,
It is not difficult to stack thousands of SSTables.
In a heavy inserting (many client threads), the memtable flush (generate new
sstable) is fren
On Mon, Jun 14, 2010 at 2:03 AM, Benjamin Black wrote:
> On Sat, Jun 12, 2010 at 7:46 PM, Anty wrote:
> > Hi:ALL
> > I have 10 nodes clus
On Sat, Jun 12, 2010 at 7:46 PM, Anty wrote:
> Hi:ALL
> I have 10 nodes cluster ,after inserting many records into the cluster, i
> compact each node by nodetool compact.
> during the compaciton process ,something wrong with one of the 10 nodes ,
> when the size of the compacted temp file rech ne
> We've also seen similar problems
>
> https://issues.apache.org/jira/browse/CASSANDRA-1177
To be clear though; un-*flushed* data is very different from
un-*compacted* data and the above seems to be about unflushed data?
In my test case there was no problem at all flushing data. But my test
was
> If you were just inserting a lot of data fast, it may be that
> background compaction was unable to keep up with the insertion rate.
> Simply leaving the node(s) for a while after the insert storm will let
> it catch up with compaction.
>
> (At least this was the behavior for me on a recent trunk
> No, i do not disable compaction during my inserts. It is weird the minor
> compaction is triggered less ofen.
If you were just inserting a lot of data fast, it may be that
background compaction was unable to keep up with the insertion rate.
Simply leaving the node(s) for a while after the insert
THX for your reply ,Jonathan.
On Sun, Jun 13, 2010 at 11:27 AM, Jonathan Ellis wrote:
> On Sat, Jun 12, 2010 at 7:46 PM, Anty wrote:
> > Hi:ALL
> > I have 10 nodes cluster ,after inserting many records into the cluster, i
> > compact each node by nodetool compact.
>
> 5000 uncompacted sstables
On Sat, Jun 12, 2010 at 7:46 PM, Anty wrote:
> Hi:ALL
> I have 10 nodes cluster ,after inserting many records into the cluster, i
> compact each node by nodetool compact.
5000 uncompacted sstables is unusual. did you disable compaction
during your inserts? that is dangerous.
> during the compa
11 matches
Mail list logo
