We're working on this in https://issues.apache.org/jira/browse/CASSANDRA-1014
On Tue, Apr 27, 2010 at 12:28 PM, Daniel Gimenez <danie...@gmail.com> wrote: > Hi everyone, > several days ago I was doing some tests in a Cassandra installation and > everything was right (few inserts, few deletions, few reads). > > Yesterday I started with "stress testing", inserting 10M rows of a > relational database in a KeySpace. I used 200 threads working in parallel in > a Large instance (Amazon ec2) to do these inserts. > > When the java client I have implemented has inserted almost 5M rows, > Cassandra server dies returning the message "concurrent mode failure". I > have monitorized the JVM with Jconsole and the problem is that "old > generation memory" has crossed the memory that I have specified in the > JVM_OPTS as you can see in the attached image. > > That's my storage-conf.xml > > <Storage> > <ClusterName>tr_cluster</ClusterName> > <AutoBootstrap>false</AutoBootstrap> > <Keyspaces> > <Keyspace Name="Keyspace1"> > <ColumnFamily Name="Standard1" CompareWith="BytesType"/> > <ColumnFamily Name="Standard2" > CompareWith="UTF8Type" > KeysCached="100%"/> > <ColumnFamily Name="StandardByUUID1" CompareWith="TimeUUIDType" /> > <ColumnFamily Name="Super1" > ColumnType="Super" > CompareWith="BytesType" > CompareSubcolumnsWith="BytesType" /> > <ColumnFamily Name="Super2" > ColumnType="Super" > CompareWith="UTF8Type" > CompareSubcolumnsWith="UTF8Type" > RowsCached="10000" > KeysCached="50%" > Comment="A column family with supercolumns, whose column > and subcolumn names are UTF8 strings"/> > > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy> > <ReplicationFactor>1</ReplicationFactor> > > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch> > > </Keyspace> > <Keyspace Name="tr"> > <ColumnFamily ColumnType="Super" > CompareWith="UTF8Type" > CompareSubcolumnsWith="UTF8Type" > Name="Ads" > Coment="List of ads in tr"/> > <ColumnFamily CompareWith="TimeUUIDType" Name="States"/> > > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy> > <ReplicationFactor>1</ReplicationFactor> > > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch> > </Keyspace> > </Keyspaces> > > <Authenticator>org.apache.cassandra.auth.AllowAllAuthenticator</Authenticator> > <Partitioner>org.apache.cassandra.dht.RandomPartitioner</Partitioner> > <InitialToken></InitialToken> > <CommitLogDirectory>/opt/space/cassandra/commitlog</CommitLogDirectory> > <DataFileDirectories> > <DataFileDirectory>/opt/space/cassandra/data</DataFileDirectory> > </DataFileDirectories> > <Seeds> > <Seed>domU-12-31-39-0E-2A-11</Seed> > </Seeds> > <RpcTimeoutInMillis>10000</RpcTimeoutInMillis> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB> > <ListenAddress></ListenAddress> > <StoragePort>7000</StoragePort> > <ThriftAddress></ThriftAddress> > <ThriftPort>9160</ThriftPort> > <ThriftFramedTransport>false</ThriftFramedTransport> > <DiskAccessMode>auto</DiskAccessMode> > <RowWarningThresholdInMB>512</RowWarningThresholdInMB> > <SlicedBufferSizeInKB>128</SlicedBufferSizeInKB> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB> > <MemtableThroughputInMB>16</MemtableThroughputInMB> > <BinaryMemtableThroughputInMB>64</BinaryMemtableThroughputInMB> > <MemtableOperationsInMillions>0.1</MemtableOperationsInMillions> > <MemtableFlushAfterMinutes>10</MemtableFlushAfterMinutes> > <ConcurrentReads>8</ConcurrentReads> > <ConcurrentWrites>400</ConcurrentWrites> > <CommitLogSync>periodic</CommitLogSync> > <CommitLogSyncPeriodInMS>10000</CommitLogSyncPeriodInMS> > <GCGraceSeconds>864000</GCGraceSeconds> > </Storage> > > > And those are the parameters I am using for the JVM: > > JVM_OPTS=" \ > -ea \ > -Xms2G \ > -Xmx2G \ > -XX:NewSize=666m -XX:MaxNewSize=666m \ > -XX:TargetSurvivorRatio=90 \ > -XX:+AggressiveOpts \ > -XX:+UseConcMarkSweepGC > -XX:+CMSIncrementalMode \ > -XX:+CMSIncrementalPacing \ > -XX:+UseParNewGC \ > -XX:+CMSParallelRemarkEnabled \ > -XX:+HeapDumpOnOutOfMemoryError \ > -XX:HeapDumpPath=/opt/space/ \ > -XX:SurvivorRatio=128 \ > -XX:MaxTenuringThreshold=0 \ > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps > -XX:PrintCMSStatistics=2 -XX:+PrintGCApplicationStoppedTime \ > -Djava.rmi.server.hostname=xxx.compute-1.amazonaws.com \ > -Dcom.sun.management.jmxremote.ssl=false \ > -Dcom.sun.management.jmxremote.port=9876 \ > -Dcom.sun.management.jmxremote.authenticate=false" > > > Thank's for all! > Daniel Gimenez. > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
