the logs say it took a long time to read a saved row cache. Try removing the files from the saved_caches dir as Jonathan suggested.
The collecting log lines with the INT max count are indicative of the IdentityQueryFilter. One of the places it is used is when adding rows to the cache. Cheers ----------------- Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 16/08/2011, at 11:35 PM, Yan Chunlu wrote: > I saw alot slicequeryfilter things if changed the log level to DEBUG. just > thought even bring up a new node will be faster than start the old one..... > it is wired > > DEBUG [main] 2011-08-16 06:32:49,213 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:225@1313068845474382 > DEBUG [main] 2011-08-16 06:32:49,245 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:453@1310999270198313 > DEBUG [main] 2011-08-16 06:32:49,251 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:26@1313199902088827 > DEBUG [main] 2011-08-16 06:32:49,576 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:157@1313097239332314 > DEBUG [main] 2011-08-16 06:32:50,674 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:41729@1313190821826229 > DEBUG [main] 2011-08-16 06:32:50,811 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:6@1313174157301203 > DEBUG [main] 2011-08-16 06:32:50,867 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:98@1312011362250907 > DEBUG [main] 2011-08-16 06:32:50,881 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:42@1313201711997005 > DEBUG [main] 2011-08-16 06:32:50,910 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:96@1312939986190155 > DEBUG [main] 2011-08-16 06:32:50,954 SliceQueryFilter.java (line 123) > collecting 0 of 2147483647: 76616c7565:false:621@1313192538616112 > > > > On Tue, Aug 16, 2011 at 7:32 PM, Yan Chunlu <springri...@gmail.com> wrote: > but it seems the row cache is cluster wide, how will the change of row cache > affect the read speed? > > > On Mon, Aug 15, 2011 at 7:33 AM, Jonathan Ellis <jbel...@gmail.com> wrote: > Or leave row cache enabled but disable cache saving (and remove the > one already on disk). > > On Sun, Aug 14, 2011 at 5:05 PM, aaron morton <aa...@thelastpickle.com> wrote: > > INFO [main] 2011-08-14 09:24:52,198 ColumnFamilyStore.java (line 547) > > completed loading (1744370 ms; 200000 keys) row cache for COMMENT > > > > It's taking 29 minutes to load 200,000 rows in the row cache. Thats a > > pretty big row cache, I would suggest reducing or disabling it. > > Background > > http://www.datastax.com/dev/blog/maximizing-cache-benefit-with-cassandra > > > > and server can not afford the load then crashed. after come back, node 3 can > > not return for more than 96 hours > > > > Crashed how ? > > You may be seeing https://issues.apache.org/jira/browse/CASSANDRA-2280 > > Watch nodetool compactionstats to see when the Merkle tree build finishes > > and nodetool netstats to see which CF's are streaming. > > Cheers > > ----------------- > > Aaron Morton > > Freelance Cassandra Developer > > @aaronmorton > > http://www.thelastpickle.com > > On 15 Aug 2011, at 04:23, Yan Chunlu wrote: > > > > > > I got 3 nodes and RF=3, when I repairing ndoe3, it seems alot data > > generated. and server can not afford the load then crashed. > > after come back, node 3 can not return for more than 96 hours > > > > for 34GB data, the node 2 could restart and back online within 1 hour. > > > > I am not sure what's wrong with node3 and should I restart node 3 again? > > thanks! > > > > Address Status State Load Owns Token > > > > 113427455640312821154458202477256070484 > > node1 Up Normal 34.11 GB 33.33% 0 > > node2 Up Normal 31.44 GB 33.33% > > 56713727820156410577229101238628035242 > > node3 Down Normal 177.55 GB 33.33% > > 113427455640312821154458202477256070484 > > > > > > the log shows it is still going on, not sure why it is so slow: > > > > > > INFO [main] 2011-08-14 08:55:47,734 SSTableReader.java (line 154) Opening > > /cassandra/data/COMMENT > > INFO [main] 2011-08-14 08:55:47,828 ColumnFamilyStore.java (line 275) > > reading saved cache /cassandra/saved_caches/COMMENT-RowCache > > INFO [main] 2011-08-14 09:24:52,198 ColumnFamilyStore.java (line 547) > > completed loading (1744370 ms; 200000 keys) row cache for COMMENT > > INFO [main] 2011-08-14 09:24:52,299 ColumnFamilyStore.java (line 275) > > reading saved cache /cassandra/saved_caches/COMMENT-RowCache > > INFO [CompactionExecutor:1] 2011-08-14 10:24:55,480 CacheWriter.java (line > > 96) Saved COMMENT-RowCache (200000 items) in 2535 ms > > > > > > > > > > > > > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of DataStax, the source for professional Cassandra support > http://www.datastax.com > >
