Re: Restoring snapshot

2018-06-13 Thread Nitan Kainth
Verify dc name and replication factor in create keyspace command in new cluster. Sent from my iPhone > On Jun 13, 2018, at 2:40 AM, > wrote: > > Dear Community, > > I took a snapshot from a node which was part of a 2 node cluster. There were > 2 keyspaces in that cluster K1 and K2. I took

RE: Restoring snapshot

2018-06-13 Thread Vishal1.Sharma
For both K1 and K2, replication factor is 2 in the new cluster(although the number of nodes is 1). I can understand the portion of the warning which says that “only 1 replica could be found” but the question is, why is it giving the name of keyspace K2 when I was restoring only K1(It should have

Re: Restoring snapshot

2018-06-13 Thread Nitan Kainth
Change RF fir k2 and then see. Sent from my iPhone > On Jun 13, 2018, at 7:05 AM, > wrote: > > For both K1 and K2, replication factor is 2 in the new cluster(although the > number of nodes is 1). I can understand the portion of the warning which says > that “only 1 replica could be found” b

RE: Restoring snapshot

2018-06-13 Thread Vishal1.Sharma
On altering the Keyspace, the warning disappears. I think the warning was not totally wrong, just slightly inaccurate. From: Nitan Kainth [mailto:nitankai...@gmail.com] Sent: Wednesday, June 13, 2018 4:38 PM To: user@cassandra.apache.org Subject: Re: Restoring snapshot Change RF fir k2 and then

Migrating to Reaper: Switching From Incremental to Reaper's Full Subrange Repair

2018-06-13 Thread Fd Habash
For those who are using Reaper … Currently, I'm run repairs using crontab/nodetool using 'repair -pr' on 2.2.8 which defaults to incremental. If I migrate to Reaper, do I have to mark sstables as un-repaired first? Also, out of the box, does Reaper run full parallel repair? If yes, is it not go

Timestamp on hints file and system.hints table data

2018-06-13 Thread Nitan Kainth
Hi, I observed a strange behavior about stored hints. Time stamp of hints file shows several months old. I deleted them and saw new hints files created with same old date. Why is that? Also, I see hints files on disk but if I query system.hints table, it shows 0 rows. Why system.hints is not pop

G1GC CPU Spike

2018-06-13 Thread rajpal reddy
Hello, we are using G1GC and noticing garbage collection taking a while and during that process we are seeing cpu spiking up to 70-80%. can you please let us know. if we have to tune any parameters for that. attaching the cassandra-env file with jam-options.calculate_heap_sizes() { case "`u

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
The gc log file is best to share when asking for help with tuning. The top of file has all the computed args it ran with and it gives details on what part of the GC is taking time. I would guess the CPU spike is from full GCs which with that small heap of a heap is probably from evacuation failu

Re: G1GC CPU Spike

2018-06-13 Thread rajpal reddy
jvm_gc_collection_seconds_count{gc="G1 Young Generation”} and also young generation seconds count keep increasing > On Jun 13, 2018, at 9:52 AM, Chris Lohfink wrote: > > The gc log file is best to share when asking for help with tuning. The top of > file has all the computed args it ran with

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
That metric is the total number of seconds spent in GC, it will increase over time with every young gc which is expected. Whats interesting is the rate of growth not the fact that its increasing. If graphing tool has option to graph derivative you should use that instead. Chris > On Jun 13, 20

Re: G1GC CPU Spike

2018-06-13 Thread rajpal reddy
Chris, We have total memory on the machine as 16G so we kept 50% to heap . initial we had CMS we had same issues so thought of changing to G1GC we have the same problem still > On Jun 13, 2018, at 9:52 AM, Chris Lohfink wrote: > > The gc log file is best to share when asking for help with tuni

Re: G1GC CPU Spike

2018-06-13 Thread rajpal reddy
we have this has the Heap settings . is the HEAP_NEWSIZE is required only for CMS. can we get rid of that for G1GC so that it can be used? MAX_HEAP_SIZE="8192M" HEAP_NEWSIZE=“800M" > On Jun 13, 2018, at 11:15 AM, Chris Lohfink wrote: > > That metric is the total number of seconds spent in GC, i

Possible support for datatype timestamp in native aggregate functions ?

2018-06-13 Thread Devopam Mittra
hi , Just wondering if it would make sense to have support for timestamp datatype in CQL native aggregates like MAX/MIN. -- Devopam Mittra Life and Relations are not binary

RE: G1GC CPU Spike

2018-06-13 Thread Steinmaurer, Thomas
Explicitly setting Xmn with G1 basically results in overriding the target pause-time goal, thus should be avoided. http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html Thomas From: rajpal reddy [mailto:rajpalreddy...@gmail.com] Sent: Mittwoch, 13. Juni 2018 17:27 To: user@cassandra

Re: G1GC CPU Spike

2018-06-13 Thread rajpal reddy
does setting HEAP_NEWSIZE=“800M” mean young generation can only use 800M? > On Jun 13, 2018, at 12:51 PM, Steinmaurer, Thomas > wrote: > > Explicitly setting Xmn with G1 basically results in  <>overriding the target > pause-time goal, thus should be avoided. > http://www.oracle.com/technetwork

Re: G1GC CPU Spike

2018-06-13 Thread Subroto Barua
Chris, What is the criteria for picking up the value for G1ReservePercent? Subroto > On Jun 13, 2018, at 6:52 AM, Chris Lohfink wrote: > > G1ReservePercent - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For a

Compaction strategy for update heavy workload

2018-06-13 Thread manuj singh
Hi all, I am trying to determine compaction strategy for our use case. In our use case we will have updates on a row a few times. And we have a ttl also defined on the table level. Our typical workload is less then 1000 writes + reads per second. At the max it could go up to 2500 per second. We use

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
> What is the criteria for picking up the value for G1ReservePercent? it depends on the object allocation rate vs the size of the heap. Cassandra ideally would be sub 500-600mb/s allocations but it can spike pretty high with something like reading a wide partition or repair streaming which migh

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
There are not even a 100ms GC pause in that, are you certain theres a problem? > On Jun 13, 2018, at 3:00 PM, rajpal reddy wrote: > > Thanks Chris I did attached the gc logs already. reattaching them > now. > > it started yesterday around 11:54PM >> On Jun 13, 2018, at 3:56 PM, Chris Lohfink

Re: Migrating to Reaper: Switching From Incremental to Reaper's Full Subrange Repair

2018-06-13 Thread kurt greaves
Not strictly necessary but probably a good idea as you don't want two separate pools of SSTables unnecessarily. Also if you've set "only_purge_repaired_tombstones" you'll need to turn that off. On Wed., 13 Jun. 2018, 23:06 Fd Habash, wrote: > For those who are using Reaper … > > > > Currently, I

Re: Compaction strategy for update heavy workload

2018-06-13 Thread kurt greaves
TWCS is probably still worth trying. If you mean updating old rows in TWCS "out of order updates" will only really mean you'll hit more SSTables on read. This might add a bit of complexity in your client if your bucketing partitions (not strictly necessary), but that's about it. As long as you're n

Re: Timestamp on hints file and system.hints table data

2018-06-13 Thread kurt greaves
system.hints is not used in Cassandra 3. Can't explain the files though, are you referring to the files timestamp or the Unix timestamp in the file name? Is there a node that's been down for several months? On Wed., 13 Jun. 2018, 23:41 Nitan Kainth, wrote: > Hi, > > I observed a strange behavior

Re: Compaction strategy for update heavy workload

2018-06-13 Thread Jonathan Haddad
I wouldn't use TWCS if there's updates, you're going to risk having data that's never deleted and really small sstables sticking around forever. If you use really large buckets, what's the point of TWCS? Honestly this is such a small workload you could easily use STCS or LCS and you'd likely neve

Re: compaction_throughput: Difference between 0 (unthrottled) and large value

2018-06-13 Thread Joshua Galbraith
Thomas, This post from Ryan Svihla has a few notes in it that may or may not be useful to you: >If you read the original throttling Jira you can see that there is a hurry up and wait component to unthrottled compaction (CASSANDRA-2156- Compaction Throttling). Ultimately you will saturate your IO

Re: Timestamp on hints file and system.hints table data

2018-06-13 Thread Nitan Kainth
Kurt, No node is down for months. And yes, I am surprised to look at Unix timestamp on files. > On Jun 13, 2018, at 6:41 PM, kurt greaves wrote: > > system.hints is not used in Cassandra 3. Can't explain the files though, are > you referring to the files timestamp or the Unix timestamp in th

Re: Timestamp on hints file and system.hints table data

2018-06-13 Thread kurt greaves
Does the UUID on the filename correspond with a UUID in nodetool status? Sounds to me like it could be something weird with an old node that no longer exists, although hints for old nodes are meant to be cleaned up. On 14 June 2018 at 01:54, Nitan Kainth wrote: > Kurt, > > No node is down for m

Re: Compaction strategy for update heavy workload

2018-06-13 Thread kurt greaves
> > I wouldn't use TWCS if there's updates, you're going to risk having > data that's never deleted and really small sstables sticking around > forever. How do you risk having data sticking around forever when everything is TTL'd? If you use really large buckets, what's the point of TWCS? No one