2011/3/8 Peter Schuller <peter.schul...@infidyne.com> > > $client->batch_mutate($mutations, > > cassandra_ConsistencyLevel::QUORUM); > > Btw, what are the mutations? Are you doing something like inserting > both very small values and very large ones? > > I have big xml file (5 GB) (mysql dump in xml format) and read data from it with SAX xml parser, all records on that file looks like this:
<row> <field name="uid">5</field> <field name="aid">3619780:1</field> <field name="cleanness">0</field> <field name="counter">7</field> <field name="gcount">0</field> <field name="lastchange">1291053619</field> <field name="disaster">0</field> <field name="tdisaster">0</field> </row> mutations in that case is 10 similar records (follow fragment of code, describes situation ) $l_supercolumn = new cassandra_SuperColumn(array("name" => $l_row["aid"], "columns" => $l_columns)); $l_c_or_sc = new cassandra_ColumnOrSuperColumn(array("super_column" => $l_supercolumn)); $l_mutation = new cassandra_Mutation(array("column_or_supercolumn" => $l_c_or_sc)); if(array_key_exists($l_key, $mutations)) { array_push($mutations[$l_key]['aquarium_friend'], $l_mutation); } else { $mutations[$l_key] = array('aquarium_friend' => array($l_mutation)); }; if(!($l_i % 10)) { make_mutation($client, $mutations, $g_loger, $g_rloger); $mutations = array(); if(!($l_i % 1000)) { $g_loger->info(sprintf("inserted: %s", $l_i)); }; }; > That's why I asked about the frequency. If you're doing a long-term > stress test and seeing a 30 second pause once per week, that's a lot > more likely to be "normal" for your workload than if you're seeing it > happen once ever three minutes. The issue is that if you want to fix > your problem, one must first figure out what the problem *is*. Based > on past mailing list traffic, it seems most people's problems that are > seemingly "due to GC" end up being because of a too high live set size > or the CMS phase triggering too late. These are fixable issues if are > running into them. > > In may case this happen from time to time. For example insert all 5GB xml took about 30-40 minutes, and nodes frozen about 5-10 time on that period (avg time of frozen 15 secs) > If all you have is a single column family with a 64 mb flush threshold > and doing a bunch of insertions, and have a heap size of 5 (was it?) > gig, you should not be having these issues. But stating that helps no > one, which is why I'm asking for more information. (Widely > extrapolating and suggesting that all Cassandra nodes will always > freeze for 30 seconds every now and then is also helping no one, other > than not being true.) > > At initial state HEAP was 6GB. When i increase HEAP size to 7GB nodes frozen only one time, but took much greater time (40 secs)