tableverify
>>>> -rwxr-xr-x. 1 root root2045 Jul 25 06:12 sstableutil
>>>> -rwxr-xr-x. 1 root root2042 Jul 25 06:12 sstableupgrade
>>>> -rwxr-xr-x. 1 root root2042 Jul 25 06:12 sstablescrub
>>>> -rwxr-xr-x. 1 root root2034 Jul 25 06:12 sstableloader
>>>>
>>>>
>>>> If this utility is no longer available how can i get sstable metadata
>>>> like repaired_at, Estimated droppable tombstones
>>>>
>>>>
>>>> Thanks
>>>> Pranay
>>>>
>>> --
Regards,
Igor Zubchenok
CTO at Multi Brains LLC
Founder of taxistartup.com saytaxi.com chauffy.com
Skype: igor.zubchenok
Hi there,
I have a Cassandra cluster running on Kubernetes. This cluster has 8
running instances with 8Gb of memory and 5 CPU cores. I can see a high load
avg in multiple instances, but no IO wait and moderate CPU usage.
Do you know how I can solve this issue?
Best,
Igor
it is noted that I should run it at ALL data
centers.
Looking for a qualified answer.
On Fri, 8 Jun 2018 at 18:08 Igor Zubchenok wrote:
> I want to repair all nodes at all data centers.
>
> Example:
> DC1
> nodeA
> nodeB
> nodeC
> DC2
> node D
> node E
&g
s option to repair only a local
> data center."
> On Jun 8, 2018, 10:42 AM -0400, user@cassandra.apache.org, wrote:
>
>
> *nodetool repair -pr*
>
> --
Regards,
Igor Zubchenok
CTO at Multi Brains LLC
Founder of taxistartup.com saytaxi.com chauffy.com
Skype: igor.zubchenok
Hi!
I want to repair all nodes in all datacenters.
Should I run *nodetool repair -pr* at all nodes of a SINGLE datacenter or
at all nodes of ALL datacenters?
--
Regards,
Igor Zubchenok
CTO at Multi Brains LLC
Founder of taxistartup.com saytaxi.com chauffy.com
Skype: igor.zubchenok
but
> hopefully shouldn't be related. Has that stuck compaction moved since last
> week?
>
>
> On 1 September 2017 at 22:54, Fay Hou [Storage Service] <
> fay...@coupang.com> wrote:
>
>> try to do a rolling restart for the cluster before doing a compation
Some generic errors:
*[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i error*
*[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i excep*
*[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i fail*
*[aladdin@ip-172-16-1-10 cassandra]$ tail debug.log | grep
Hey Kurt,
Thanks for your reply.
Soon as the whole cluster was upgraded (using existing nodes) it worked
pretty well. After a while, the high cpu usage/ pending compactions was
back affecting all cluster.
It's still an open problem.
2017-08-21 20:24 GMT-03:00 kurt greaves :
> Why are you adding
node running `Cassandra 3.9` and one node
running `Cassandra 3.11` without any further issues.
Have you guys faced this problem before?
Thanks in advance!
Igor
true`,
and everything worked fine, but these new nodes were with 90%+ of cpu
usage, so I removed them. Do you have any idea why this happened? Should I
always upgrade every machine locally, keeping its data, instead of adding
new nodes with a newer version? Best,
2017-08-04 16:01 GMT-03:00 Petrus Gomes :
Hi there,
Is it possible to add a Cassandra 3.11 node into a Cassandra 3.9 cluster?
I wish I could upgrade this cluster without uninstalling and installing
Cassandra on existing nodes, just by adding and removing nodes.
Thanks in advance!
t;
>>> Do you see data on other DC or just directory structure? Directory
>>> structure would populate because it is DDL but inserts shouldn’t populate,
>>> ideally.
>>>
>>> On May 16, 2017, at 3:19 PM, suraj pasuparthy <
>>> suraj.pasupar...@gmail
Thank you Varun and DuyHai!
2017-05-10 20:57 GMT-03:00 Varun Gupta :
> Hi Igor,
>
> You can setup cluster with configuration as below.
>
> Replication: DC1: 3 and DC2: 1.
>
> If you are using datastax java driver, then use dcaware load balancing
> policy and pass DC1,
Hey everyone,
Imagine a have Cassandra cluster with 4 nodes.
Is it possible to have a separate node which would not receive requests but
would by in sync with the rest of the cluster? Ideally this super node
would have all data of the cluster.
I want to take a snapshot of this node from time to
gt; copy system_traces.events TO 'traces_dump.csv';
>
> Also do make sure you dont set trace probability to a high number if
> working on a production database as it can adversely impact performance.
>
> Regards,
>
> On Sun, Feb 19, 2017 at 1:28 AM, Igor Leão wrote:
>
>
Hi Bhuvan,
Thanks a lot!
Any idea if something can be done for C* 2.X?
Best,
Igor
2017-02-18 16:41 GMT-03:00 Bhuvan Rawal :
> Hi Igor,
>
> If you are using java driver, you can log slow queries on client side
> using QueryLogger.
> https://docs.datastax.com/en/developer/java-dr
Hi there,
I'm wondering how to log queries from Cassandra. These queries can be
either slow queries or all queries. The only constraint is that I should do
this on server side.
I tried using `nodetool settraceprobability`, which writes all queries to
the keyspace `system_traces`. When I try to se
, May 25, 2014 8:06 PM
To: user@cassandra.apache.org
Subject: Re: cassandra boot is stuck in hint compaction.
On 05/25/2014 04:12 AM, Igor Shprukh wrote:
> hi guys, we have a 6 node cluster, consisting of 5 linux machines and a
> windows one.
Mixed linux/windows clusters are not
?hi guys, we have a 6 node cluster, consisting of 5 linux machines and a
windows one.
after a hard shutdown of the windows machine, the node is stuck on hints
compaction for more than
half an hour and cassandra won't start. must say that it is a strong machin
If you are talking about 1.2.x then I also have memory problems on the
idle cluster: java memory constantly slow grows up to limit, then spend
long time for GC. I never seen such behaviour for 1.0.x and 1.1.x, where
on idle cluster java memory stay on the same value.
On 06/07/2013 05:19 PM, Jo
Hello Christopher,
BTW, are you talking about 99th percentiles on client side, or about
percentiles from cassandra histograms for CF on cassandra side?
Thanks!
On 05/22/2013 05:41 PM, Christopher Wirt wrote:
Hi Igor,
Yea same here, 15ms for 99^th percentile is our max. Currently getting
On 05/22/2013 05:41 PM, Christopher Wirt wrote:
Hi Igor,
Yea same here, 15ms for 99^th percentile is our max. Currently getting
one or two ms for most CF. It goes up at peak times which is what we
want to avoid.
Our 99 percentile also goes up at peak times but stay at acceptable level
Hello
What level of read performance do you expect? We have limit 15 ms for 99
percentile with average read latency near 0.9ms. For some CF 99
percentile actually equals to 2ms, for other - to 10ms, this depends on
the data volume you read in each query.
Tuning read performance involved clea
me DC should have identical seeds lists.
2. At least at one DC nodes MUST have in its seed lists seeds from all
other DCs.
2013/5/17 Igor mailto:i...@4friends.od.ua>>
I see no reason to restart all nodes. You can continue to use seed
from first DC - seed used for loading ring
co
I see no reason to restart all nodes. You can continue to use seed from
first DC - seed used for loading ring configuration(locations, token
ranges, etc), not data.
On 05/17/2013 10:34 AM, Sergey Naumov wrote:
If I understand you correctly, GossipingPropertyFileSnitch is useful
for manipulatio
random IO, then that 2 MB per read seems like a lot of extra overhead.
-Bryan
On Thu, May 16, 2013 at 12:35 PM, Keith Wright
mailto:kwri...@nanigans.com>> wrote:
We actually have it set to 512. I have tried decreasing my
SSTable size to 5 MB and changi
My 5 cents: I'd check blockdev --getra for data drives - too high values
for readahead (default to 256 for debian) can hurt read performance.
On 05/16/2013 05:14 PM, Keith Wright wrote:
Hi all,
I currently have 2 clusters, one running on 1.1.10 using CQL2 and
one running on 1.2.4 using CQ
Hello
Does anybody seen memory problems on idle cluster?
I have 8-node ring with cassandra 1.2.3 which never been used and stay
idle for several weeks. Yesterday when I decided to upgrade it to 1.2.4
I found lot of messages like
INFO 11:10:56,273 GC for ParNew: 1039 ms for 1 collections, 663
You can try to disable readahead on cassandra data disk.
Jon Scarborough написал(а):
>Checked tpstats, there are very few dropped messages.
>
>Checked histograms. Mostly nothing surprising. The vast majority of
>rows
>are small, and most reads only access one or two SSTables.
>
>What I did disco
Hello!
Explain please, how this work when I request for key which is not in
database
* The closest node (as determined by proximity sorting as described
above) will be sent a command to perform an actual data read (i.e.,
return data to the co-ordinating node).
* As required by consiste
Hello!
We have 1.0.7 multi-DC cassandra setup with strict time limits for read
(15ms). We use RF=1 per DC and reads with CL=ONE. Data in datacenters
are in sync, but we have next problem:
when application looks for key which is not yet in database, coordinator
wait for digests from remote data
Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 23/02/2013, at 5:17 AM, Igor <mailto:i...@4friends.od.ua>> wrote:
Hello
Cassandra 1.0.7
Some time ago we used secondary index on one of CF. Due to
performance reasons we dropped this secondary index after w
Hello
Cassandra 1.0.7
Some time ago we used secondary index on one of CF. Due to performance
reasons we dropped this secondary index after while. But now, each time
I add and bootstrap new node I see how cassandra again build this
secondary index on this node (which takes huge time), and whe
Hello
Too much GC? Check JVM heap settings and real usage.
On 06/27/2012 01:37 AM, Dustin Wenz wrote:
We occasionally see fairly poor compaction performance on random nodes in our
7-node cluster, and I have no idea why. This is one example from the log:
[CompactionExecutor:45] 2012-06
On 05/13/2012 07:18 PM, Thanh Ha wrote:
Hi All,
Do I have to do maintenance "nodetool repair" on CFs that do not have
deletions?
Probably you should (depending how you do reads), if your nodes for some
reasons have different data (like connectivity problems, node down, etc).
I only perform
le.com
On 22/04/2012, at 10:16 PM, Igor wrote:
but after repair all nodes should be in sync regardless of whether
new files were compacted or not.
Do you suggest major compaction after repair? I'd like to avoid it.
On 04/22/2012 11:52 AM, Philippe wrote:
Repairs generate new files that
where the temporary extra volume comes from?
Le 21 avr. 2012 20:43, "Igor" <mailto:i...@4friends.od.ua>> a écrit :
Hi
I can't understand the repair behavior in my case. I have 12 nodes
ring (all 1.0.7):
10.254.237.2LA ADS-LA-1Up Nor
Hi
I can't understand the repair behavior in my case. I have 12 nodes ring
(all 1.0.7):
10.254.237.2LA ADS-LA-1Up Normal 50.92 GB
0.00% 0
10.254.238.2TX TX-24-RACK Up Normal 33.29 GB
0.00% 1
10.254.236.2VA ADS-VA-1Up
For my use case it would be nice to have per CF TTL (to protect myself
from application bug and from storage leak due to missed TTL), but seems
you can't avoid tombstones even in this case and if you change CF TTL
during runtime.
On 04/18/2012 03:06 PM, Viktor Jevdokimov wrote:
Our use case r
of sstables as second
parameter for userDefinedCompaction?
On 04/18/2012 05:53 AM, Jonathan Ellis wrote:
On Sat, Apr 14, 2012 at 4:08 AM, Igor wrote:
Assume I insert all my data with TTL=2weeks and let we have sstable A which
was created week ago at the time T, so I know that right now it co
d, so it shoould be wiped at any
compaction on table A.
Or I missed something?
On 04/14/2012 11:27 AM, Radim Kolar wrote:
Dne 4.4.2012 6:52, Igor napsal(a):
Here is small python script I run once per day. You have to adjust
size and/or age limits in the 'if' operator. Also I use mx4j
logs say you are streaming a lot of ranges?
zgrep -E "(Performing streaming repair|out of sync)"
On Tue, Apr 10, 2012 at 9:45 AM, Igor wrote:
On 04/10/2012 07:16 PM, Frank Ng wrote:
Short answer - yes.
But you are asking wrong question.
I think both processes are taking a while. When
On 04/11/2012 12:04 PM, ruslan usifov wrote:
HH - this is hinted handoff?
Yes
2012/4/11 Igor mailto:i...@4friends.od.ua>>
On 04/11/2012 11:49 AM, R. Verlangen wrote:
Not everything, just HH :)
I hope this works for me for the next reasons: I have quite large
would be pretty rare for
that to occur. At least that is how I interpret this.
2012/4/11 Igor mailto:i...@4friends.od.ua>>
BTW, I heard that we don't need to run repair if all your data
have TTL, all HH works, and you never delete your data.
On 04/11/2012 11:34 AM, ru
BTW, I heard that we don't need to run repair if all your data have TTL,
all HH works, and you never delete your data.
On 04/11/2012 11:34 AM, ruslan usifov wrote:
Sorry fo my bad english, so QUORUM allow doesn't make repair
regularity? But form your anser it does not follow
2012/4/11 R. Ve
ranges?
zgrep -E "(Performing streaming repair|out of sync)"
On Tue, Apr 10, 2012 at 9:45 AM, Igor <mailto:i...@4friends.od.ua>> wrote:
On 04/10/2012 07:16 PM, Frank Ng wrote:
Short answer - yes.
But you are asking wrong question.
I think both processes
?
On Tue, Apr 10, 2012 at 10:42 AM, Igor <mailto:i...@4friends.od.ua>> wrote:
Hi
You can check with nodetool which part of repair process is slow
- network streams or verify compactions. use nodetool netstats or
compactionstats.
On 04/10/2012 05:16 PM, Frank
Hi
You can check with nodetool which part of repair process is slow -
network streams or verify compactions. use nodetool netstats or
compactionstats.
On 04/10/2012 05:16 PM, Frank Ng wrote:
Hello,
I am on Cassandra 1.0.7. My repair processes are taking over 30 hours
to complete. Is it
Here is small python script I run once per day. You have to adjust size
and/or age limits in the 'if' operator. Also I use mx4j interface for
jmx calls.
#!/usr/bin/env python
import sys,os,glob,time,urllib2
CASSANDRA_DATA='/spool1/cassandra/data'
DONTTOUCH=('system',)
now = time.time()
def
The first is keyspace name, second is sstable name (like
transaction-hc-1024-Data.db
-Original Message-
From: Radim Kolar
To: user@cassandra.apache.org
Sent: Wed, 04 Apr 2012 3:14
Subject: Re: size tiered compaction - improvement
Dne 3.4.2012 23:04, i...@4friends.od.ua napsal(a):
>
if you know for sure that you will free lot of space compacting some old table,
then you can call UserdefinedCompaction for this table(you can do this from
cron). There is also a ticket in jira with discussion on per-sstable expierd
column and tombstones counters.
-Original Message
On 03/28/2012 02:04 PM, Radim Kolar wrote:
RAID0 would help me use more efficiently the total disk space
available at each node, but tests have shown that under write load it
behaves much worse than using separate data dirs, one per disk.
there are different strategies how RAID0 splits reads,
You can try to play with comaction thresholds - looks like your data wait too
long before sizetiered compaction start to merge old large sstables. I have the
same scenario as you (no deletes, all data with TTL) and I use script which
call userdefinedcompaction on these old sstables.
-Origin
Hello
Datasize should decrease during minor compactions. Check logs for compactions
results.
-Original Message-
From: Radim Kolar
To: user@cassandra.apache.org
Sent: Mon, 19 Mar 2012 12:16
Subject: repair broke TTL based expiration
I suspect that running cluster wide repair interf
just run "nodetool compactionstat" on other nodes.
-Original Message-
From: "R. Verlangen"
To: user@cassandra.apache.org
Sent: Wed, 07 Mar 2012 23:09
Subject: Re: Node joining / unknown
@Brandon: Thank you for the information. I'll do that next time.
@I
Maybe it wait for verification compaction on other node?
-Original Message-
From: "R. Verlangen"
To: user@cassandra.apache.org
Sent: Wed, 07 Mar 2012 22:15
Subject: Re: Node joining / unknown
At this moment the node has joined the ring (after a restart: tried that
before, but now it
Hi!
On the
http://wiki.apache.org/cassandra/Operations#Dealing_with_the_consequences_of_nodetool_repair_not_running_within_GCGraceSeconds
you can read:
"To minimize the amount of forgotten deletes, first increase
GCGraceSeconds across the cluster (rolling restart required)"
Rolling restart
hus someone trying to
paper over 6 technologies and compare them with a few bullet points is really
doing the world an injustice.
On Tue, Dec 27, 2011 at 5:01 PM, Igor Lino wrote:
Hi!
I was trying to get an understanding of the real strengths of Cassandra
against other compet
Cassandra expert can improve that generic description)
Thanks,
Igor
Hi,
10 nodes cassandra 1.0.3, several DC. weekly nodetool repair stuck for
unusual long time for node 10.254.237.2.
output log on this node:
INFO 11:19:42,045 Starting repair command #1, repairing 5 ranges.
INFO 11:19:42,053 [repair #040aae00-28a1-11e1--e378018944ff] new
session: will s
No idea, try to check logs for errors, and increase verbosity level on that
node.
-Original Message-
From: Bart Swedrowski
To: user@cassandra.apache.org
Sent: Wed, 14 Dec 2011 16:45
Subject: Re: One ColumnFamily places data on only 3 out of 4 nodes
On 14 December 2011 13:02, wrote:
Do you use randompartitiner? What nodetool getendpoints show for several random
keys?
-Original Message-
From: Bart Swedrowski
To: user@cassandra.apache.org
Sent: Wed, 14 Dec 2011 12:56
Subject: Re: One ColumnFamily places data on only 3 out of 4 nodes
Anyone?
On 12 December 2011 15:
Hi,
This is my first post, so first of all - thanks to Cassandra authors and
community for their excellent job!
Now to my question... I need a plan for transition from SimpleStrategy
to NetworkSopologyStrategy (as I have to add two servers from remote
datacenter with RTT up to 120ms to my cl
63 matches
Mail list logo
