Hi, Tyler, I'm working with Vram on this project and can respond to your questions.
We do indeed continue to get inconsistent data after many read operations. These columns and rowkeys are months old and they have had many reads done on them over that time. Using cassandra-cli we see that on 3 of the 5 servers we get 54 columns back for a specific rowkey and on the other 2 we get back 68 columns. We've also done this same query on 4 or 5 other rowkeys and different rowkeys get different (inconsistent) results with only one of the rowkeys getting back the same results from all 5 servers. Some of the rowkeys get 4 consistent and 1 different, and some get the 3:2 ratio above. From looking at the data I'm guessing that the results from the 3 nodes are correct and the results from the 2 nodes are old (the diff between the result sets is that the 54 is a subset of the 68). Running "nodetool ring" shows all 5 of them have the same view of the ring. The configuration for the cluster has not changed in quite a while, and we're running 0.6.8 right but were running 0.6.5 for quite a while. Thanks, Scott On Mon, Jan 10, 2011 at 8:27 AM, Tyler Hobbs <ty...@riptano.com> wrote: > What version of Cassandra? What consistency level are you writing/reading > at? > > Do you continue to get inconsistent results when you read the same data > over and over (i.e. read repair is not fixing something)? > > Do all of your nodes show the same thing when you run nodetool ring against > them? > > - Tyler > > > On Mon, Jan 10, 2011 at 9:18 AM, Vram Kouramajian < > vram.kouramaj...@gmail.com> wrote: > >> We are running a five node cluster in production with a replication >> factor of three. The query results from the 5 nodes are returning >> different results (2 of the 5 nodes return extra columns for the same >> row key). >> >> We are not sure the root of the problem (any config issues). Any >> suggestions? >> >> Thanks, >> Vram >> > >
