select * from users;
OK here’s the trace. The times are super long.
name | super
---------------+-------
user1 | False
user2 | True
user3 | True
user4 | False
user5 | True
(5 rows)
Tracing session: 55a4aa50-8da3-11e7-adbb-e7bbc3a8a72e
activity
| timestamp
| source | source_elapsed
-----------------------------------------------------------------------------------------------------------------------------------+----------------------------+---------------+----------------
Execute CQL3 query | 2017-08-30 10:50:31.413000
| xx.xx.xx.113 | 0
READ message received from
/xx.xx.xx.113 [MessagingService-Incoming-/xx.xx.xx.113] | 2017-08-30
10:50:31.398000 | xx.xx.xx.107 | 66
Executing
single-partition query on users [SharedPool-Worker-1] | 2017-08-30
10:50:31.399000 | xx.xx.xx.107 | 110
Acquiring sstable references [SharedPool-Worker-1] | 2017-08-30 10:50:31.399000
| xx.xx.xx.107 | 114
Merging memtable tombstones [SharedPool-Worker-1] | 2017-08-30 10:50:31.400000
| xx.xx.xx.107 | 121
Key cache hit for sstable 288 [SharedPool-Worker-1] | 2017-08-30
10:50:31.400000 | xx.xx.xx.107 | 129
Seeking to
partition beginning in data file [SharedPool-Worker-1] | 2017-08-30
10:50:31.400000 | xx.xx.xx.107 | 130
Skipped 0/1 non-slice-intersecting sstables,
included 0 due to tombstones [SharedPool-Worker-1] | 2017-08-30 10:50:31.401000
| xx.xx.xx.107 | 209
Merging data
from memtables and 1 sstables [SharedPool-Worker-1] | 2017-08-30
10:50:31.401000 | xx.xx.xx.107 | 211
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:50:31.402000
| xx.xx.xx.107 | 226
Enqueuing response to /xx.xx.xx.113 [SharedPool-Worker-1] | 2017-08-30
10:50:31.402000 | xx.xx.xx.107 | 321
Sending REQUEST_RESPONSE message to
/xx.xx.xx.113 [MessagingService-Outgoing-/xx.xx.xx.113] | 2017-08-30
10:50:31.402000 | xx.xx.xx.107 | 417
Parsing select * from users; [SharedPool-Worker-2] | 2017-08-30 10:50:31.414000
| xx.xx.xx.113 | 22
Preparing statement [SharedPool-Worker-2] | 2017-08-30 10:50:31.415000
| xx.xx.xx.113 | 58
reading data from /xx.xx.xx.107 [SharedPool-Worker-2] | 2017-08-30
10:50:31.415000 | xx.xx.xx.113 | 950
Sending READ message to
/xx.xx.xx.107 [MessagingService-Outgoing-/xx.xx.xx.107] | 2017-08-30
10:50:31.415000 | xx.xx.xx.113 | 1017
REQUEST_RESPONSE message received from
/xx.xx.xx.107 [MessagingService-Incoming-/xx.xx.xx.107] | 2017-08-30
10:50:31.415000 | xx.xx.xx.113 | 1744
Processing
response from /xx.xx.xx.107 [SharedPool-Worker-1] | 2017-08-30 10:50:31.416000
| xx.xx.xx.113 | 1805
Computing ranges to query [SharedPool-Worker-2] | 2017-08-30 10:50:31.416000
| xx.xx.xx.113 | 1853
Submitting range requests on 63681 ranges with a concurrency of 10056
(0.009944752 rows per range expected) [SharedPool-Worker-2] | 2017-08-30
10:50:31.424000 | xx.xx.xx.113 | 11427
PAGED_RANGE message received from
/xx.xx.xx.113 [MessagingService-Incoming-/xx.xx.xx.113] | 2017-08-30
10:51:25.002000 | xx.xx.xx.116 | 28
Executing seq scan across 1 sstables for
[min(-9223372036854775808), min(-9223372036854775808)] [SharedPool-Worker-1] |
2017-08-30 10:51:25.002000 | xx.xx.xx.116 | 82
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:51:25.003000
| xx.xx.xx.116 | 178
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:51:25.003000
| xx.xx.xx.116 | 186
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:51:25.003000
| xx.xx.xx.116 | 191
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:51:25.003000
| xx.xx.xx.116 | 194
Read
1 live and 0 tombstone cells [SharedPool-Worker-1] | 2017-08-30 10:51:25.004000
| xx.xx.xx.116 | 198
Scanned 5 rows and matched 5 [SharedPool-Worker-1] | 2017-08-30 10:51:25.004000
| xx.xx.xx.116 | 224
Enqueuing response to /xx.xx.xx.113 [SharedPool-Worker-1] | 2017-08-30
10:51:25.004000 | xx.xx.xx.116 | 240
Sending REQUEST_RESPONSE message to
/xx.xx.xx.113 [MessagingService-Outgoing-/xx.xx.xx.113] | 2017-08-30
10:51:25.004000 | xx.xx.xx.116 | 302
Enqueuing request to /xx.xx.xx.116 [SharedPool-Worker-2] | 2017-08-30
10:51:25.014000 | xx.xx.xx.113 | 601103
Submitted 1 concurrent range
requests covering 63681 ranges [SharedPool-Worker-2] | 2017-08-30
10:51:25.014000 | xx.xx.xx.113 | 601120
Sending PAGED_RANGE message to
/xx.xx.xx.116 [MessagingService-Outgoing-/xx.xx.xx.116] | 2017-08-30
10:51:25.015000 | xx.xx.xx.113 | 601190
REQUEST_RESPONSE message received from
/xx.xx.xx.116 [MessagingService-Incoming-/xx.xx.xx.116] | 2017-08-30
10:51:25.015000 | xx.xx.xx.113 | 601771
Processing
response from /xx.xx.xx.116 [SharedPool-Worker-1] | 2017-08-30 10:51:25.015000
| xx.xx.xx.113 | 601824
Request complete | 2017-08-30 10:51:25.014874
| xx.xx.xx.113 | 601874
From: Oleksandr Shulgin <oleksandr.shul...@zalando.de>
Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Wednesday, August 30, 2017 at 10:42 AM
To: User <user@cassandra.apache.org>
Subject: Re: system_auth replication factor in Cassandra 2.1
On Wed, Aug 30, 2017 at 6:40 PM, Chuck Reynolds
<creyno...@ancestry.com<mailto:creyno...@ancestry.com>> wrote:
How many users do you have (or expect to be found in system_auth.users)?
5 users.
What are the current RF for system_auth and consistency level you are using in
cqlsh?
135 in one DC and 227 in the other DC. Consistency level one
Still very surprising...
Did you try to obtain a trace of a timing-out query (with TRACING ON)?
Tracing timeout even though I increased it to 120 seconds.
Even if cqlsh doesn't print the trace because of timeout, you should be still
able to find something in system_traces.
--
Alex
