I have tried it with RF=1 and 2 nodes and the result was the same.
In this case I am not able to query anything, regardless the consistency level.
I have conducted some more tests with 4 nodes and RF=2 and before each test I
also completely cleared all datafiles (data/commit/saved_cache).
Each time I do this the nodes get different locations on the ring and the
noticed behavior is different (but similar) each time.
I have added the details for 2 concrete scenario;
----
1) clear all 3 data directories for all nodes
2) startup all nodes; the ring is;
# bin/nodetool -h 172.16.21.12 ring
Address Status Load Range
Ring
81296677393575658129031223607666487650
172.16.21.12 Up 489 bytes 61644956546475371096781179489550878842
|<--|
172.16.21.14 Up 489 bytes 63811055530446251457866768401307252614
| |
172.16.21.13 Up 489 bytes 71622328744148061175473750202165524046
| |
172.16.21.11 Up 489 bytes 81296677393575658129031223607666487650
|-->|
3) I use the following ColumnFamily <ColumnFamily Name="Customers"
ColumnType="Super" CompareWith="AsciiType"/>
And add the following rows (key and values utf8 strings)
Customer['1'].simple.id='1'
Customer['2'].simple.id='2'
Customer['3'].simple.id='3'
Customer['4'].simple.id='4'
4) Iterating over the columns now yield;
Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency
level
Iterating Node.14 ==> only row '3' and '4' with consistency
level ONE, all rows with consistency level All
----
Repating all process steps again yields the following ring and iteration results
# bin/nodetool -h 172.16.21.11 ring
Address Status Load Range
Ring
157208721428268855331814529932509412075
172.16.21.11 Up 489 bytes 18662665007498667259581716616851966032
|<--|
172.16.21.13 Up 489 bytes 125997322478065775200568923383951796531
| |
172.16.21.12 Up 489 bytes 148125235552363592916130178190645177045
| |
172.16.21.14 Up 489 bytes 157208721428268855331814529932509412075
|-->|
Iterating Nodes .11 .12 .14 ==> no data with consistency level One, all
rows with consistency level All
Iterating Node .13 ==> only row '3' and '4' with consistency
level ONE, all rows with consistency level All
I appears to me there is an inconsistency somewhere, I hope this info helps.
Eric
-----Original Message-----
From: Jonathan Ellis [mailto:[email protected]]
Sent: woensdag 10 november 2010 17:48
To: user
Subject: Re: iteration does not yield all data with consistency ONE
Interesting. Does it simplify further to RF=1 and 2 nodes?
On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
<[email protected]> wrote:
> No, all nodes were up and running while the single key was inserted.
> The insert however was with consistency One. I assume however that the
> replicas are still written in this case.
> It is btw also very reproducible.
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:[email protected]]
> Sent: woensdag 10 november 2010 15:44
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Was the node that should have the other replica of this row down when
> it was inserted?
>
> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
> <[email protected]> wrote:
>>
>> Hello,
>>
>>
>>
>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a
>> replication of 2.
>>
>> When I insert a number of rows I can always retrieve them by their explicit
>> id (get_range_slices("<key>","", 1).
>>
>> Playing with consistency levels and temporarily shutting down a Cassandra
>> node all yields the expected result.
>>
>>
>>
>> However when I use get_range_slices("","", n) to iterate over all rows, I
>> sometimes don't get anything (depending on the node).
>>
>>
>>
>> I then reduced the problem to inserting just a single row.
>>
>> Specifically, the 'iteration' only seems to succeed when I issue the request
>> to the node that contains the first copy.
>>
>> I Discovered that when I iterate using a consistency level of Quorum/All the
>> iteration always succeeds and I properly get the one row.
>>
>>
>>
>> So a solution would be to always use consistency level One/All but that has
>> a performance penalty.
>>
>>
>>
>> Can anyone explain why iterating using get_range_slices("","",n) does not
>> always function with consistency level One on all nodes?
>>
>>
>>
>> Thanks,
>>
>> Eric
>>
>>
>>
>> P.S. To rule out any discussion on whether or not to use iteration in the
>> first place, we only plan to use it for backup and periodic cleanup cycles.
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
