onfiguration?
>>
>> The cell count is the number of triplets: (name, value, timestamp)
>>
>> Also, I see that you have set sstable_size_in_mb at 50 MB. What is the
>> rational behind this? (Yes I'm curious :-) ). Anyway your "SSTables per
>> read&quo
stograms your data maybe just skewed
(most likely explanation is probably the correct one here)
Regard,
Ryan Svihla
_________________
From: Joseph Tech
Sent: Wednesday, August 31, 2016 11:16 PM
Subject: Re: Read timeouts on primary key queries
To:
Patrick,
The desc table is bel
each other. On the
> surface, don't see anything which indicates too much skewing (assuming
> skewing ==keys spread across many SSTables) . Please confirm. Related to
> this, what does the "cell count" metric indicate ; didn't find a clear
> explanation in the documents
correct one here)
Regard,
Ryan Svihla
_________________
From: Joseph Tech
Sent: Wednesday, August 31, 2016 11:16 PM
Subject: Re: Read timeouts on primary key queries
To:
Patrick,
The desc table is below (only col names changed) :
CREATE TABLE db.tbl ( id1 text, id2 text,
the cfhistograms run within few mins of each other. On the
>> surface, don't see anything which indicates too much skewing (assuming
>> skewing ==keys spread across many SSTables) . Please confirm. Related to
>> this, what does the "cell count" metric indicate ; didn&#
gt;
> Thanks,
> Joseph
>
>
> On Thu, Sep 1, 2016 at 6:30 PM, Ryan Svihla wrote:
>
> Have you looked at cfhistograms/tablehistograms your data maybe just
> skewed (most likely explanation is probably the correct one here)
>
> Regard,
>
> Ryan Svihla
>
&g
___
From: Joseph Tech
Sent: Wednesday, August 31, 2016 11:16 PM
Subject: Re: Read timeouts on primary key queries
To:
Patrick,
The desc table is below (only col names changed) :
CREATE TABLE db.tbl ( id1 text, id2 text, id3 text, id4 text, f1
text, f2 map, f3 map, c
ost likely explanation is probably the correct one here)
>
> Regard,
>
> Ryan Svihla
>
> _
> From: Joseph Tech
> Sent: Wednesday, August 31, 2016 11:16 PM
> Subject: Re: Read timeouts on primary key queries
> To:
>
>
>
>
anation is probably the correct one here)
Regard,
Ryan Svihla
_
From: Joseph Tech
Sent: Wednesday, August 31, 2016 11:16 PM
Subject: Re: Read timeouts on primary key queries
To:
Patrick,
The desc table is below (only col names changed) :
CREATE TABLE db.tbl (
gt;
> Regard,
>
> Ryan Svihla
>
> _
> From: Joseph Tech
> Sent: Wednesday, August 31, 2016 11:16 PM
> Subject: Re: Read timeouts on primary key queries
> To:
>
>
>
> Patrick,
>
> The desc table is below (only col names changed) :
>
> CREATE TABLE db.tbl
Have you looked at cfhistograms/tablehistograms your data maybe just skewed
(most likely explanation is probably the correct one here)
Regard,
Ryan Svihla
_
From: Joseph Tech
Sent: Wednesday, August 31, 2016 11:16 PM
Subject: Re: Read timeouts on
Patrick,
The desc table is below (only col names changed) :
CREATE TABLE db.tbl (
id1 text,
id2 text,
id3 text,
id4 text,
f1 text,
f2 map,
f3 map,
created timestamp,
updated timestamp,
PRIMARY KEY (id1, id2, id3, id4)
) WITH CLUSTERING ORDER BY (id2 ASC, id
If you are getting a timeout on one table, then a mismatch of RF and node
count doesn't seem as likely.
Time to look at your query. You said it was a 'select * from table where
key=?' type query. I would next use the trace facility in cqlsh to
investigate further. That's a good way to find hard to
On further analysis, this issue happens only on 1 table in the KS which has
the max reads.
@Atul, I will look at system health, but didnt see anything standing out
from GC logs. (using JDK 1.8_92 with G1GC).
@Patrick , could you please elaborate the "mismatch on node count + RF"
part.
On Tue, Au
There could be many reasons for this if it is intermittent. CPU usage + I/O
wait status. As read are I/O intensive, your IOPS requirement should be met
that time load. Heap issue if CPU is busy for GC only. Network health could
be the reason. So better to look system health during that time when it
Hi Patrick,
The nodetool status shows all nodes up and normal now. From OpsCenter
"Event Log" , there are some nodes reported as being down/up etc. during
the timeframe of timeout, but these are Search workload nodes from the
remote (non-local) DC. The RF is 3 and there are 9 nodes per DC.
Thanks
You aren't achieving quorum on your reads as the error is explains. That
means you either have some nodes down or your topology is not matching up.
The fact you are using LOCAL_QUORUM might point to a datacenter mis-match
on node count + RF.
What does your nodetool status look like?
Patrick
On M
17 matches
Mail list logo
