Ideal values for coreConnectionsForRemote and maxConnectionsForRemote : CQL Java Driver

[Sourabh Agrawal]

What are the ideal values for coreConnectionsForRemote and
maxConnectionsForRemote in CQL Java Driver ( by Datastax).

-- 
Sourabh Agrawal
Bangalore
+91 9945657973

Re: Frequent Full GC that take > 30s

[Samuel CARRIERE]

>>>Hello.
>>>
>>>I'm having problems with frequent Full GCs that take a long time, and 
cause a 
>>>burst of timeouts for the client application. But first, here is my 
>>>configuration:
>>>
>>>Cassandra: 1.1.5
>>>Key Cache: size 209715168 (bytes), capacity 209715168 (bytes), 
>>>1331992571 hits, 1831790912 requests, 0.854 recent hit rate, 14400 save 
period 
>>>in seconds
>>>Row Cache: size 104588313 (bytes), capacity 104857600 (bytes), 
>>>585590776 hits, 646992444 requests, 0.880 recent hit rate, 0 save 
period in 
>>>seconds
>>>flush_largest_memtables_at: 0.8
>>>memtable_total_space_in_mb: 2048
>>>
>>>JVM: 
>>>java version "1.6.0_35"
>>>Java(TM) SE Runtime Environment (build 1.6.0_35-b10)
>>>Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01, mixed mode)
>>>
>>>JVM Options:
>>>-XX:+UseParNewGC
>>>-XX:+UseConcMarkSweepGC
>>>-XX:+CMSParallelRemarkEnabled
>>>-XX:SurvivorRatio=8
>>>-XX:MaxTenuringThreshold=1
>>>-XX:CMSInitiatingOccupancyFraction=75
>>>-XX:+UseCMSInitiatingOccupancyOnly
>>>-XX:ParallelCMSThreads=4
>>>-Xms16G
>>>-Xmx16G
>>>-Xmn800M
>>>-Xss196k
>>>
>>>6 nodes with 32GB RAM, Intel(R) Xeon(R) CPU E5-2609, 300GB data per 
node.

>> Your ParNew size is way too small. Generally 4GB ParNew (-Xmn) works 
out best 
>> for 16GB heap
>
>I was afraid that a 4GB ParNew would cause Young GCs to take too long. 
I'm 
>going to test higher ParNew values.
>
>Thanks,
>André

Hi André,
What are the results of your investigations ? Did you try to tune 
MaxTenuringThreshold and -Xmn ?
I'm interested because I'm experiencing long GC pauses during repairs 
(cassandra 1.1.12).

RE: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]

[Viktor Jevdokimov]

I mean insert/write data. When data fills memtable, memtable is flushed to disk 
as sstable, when new sstable is created, Cassandra will check if compaction is 
needed and triggers one.

From: Yatong Zhang [mailto:bluefl...@gmail.com]
Sent: Monday, May 5, 2014 9:54 AM
To: user@cassandra.apache.org
Subject: Re: Is the updating compaction strategy from 'sized tiered' to 
'leveled' automatic or need to be done manually? [heur]

What you mean 'you need write to this CF'? I've changed the schema by using 
CQL3 'alter table' statments.

On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov 
mailto:viktor.jevdoki...@adform.com>> wrote:
To trigger LCS you need to write to this CF and wait when new sstable flushes. 
I can’t find any other way to start LCS.

Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer

Email: viktor.jevdoki...@adform.com
Phone: +370 5 212 3063, Fax +370 5 261 
0453
J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
Follow us on Twitter: @adforminsider
Experience Adform DNA

[Adform News]
[Adform awarded the Best Employer 
2012]


Disclaimer: The information contained in this message and attachments is 
intended solely for the attention and use of the named addressee and may be 
confidential. If you are not the intended recipient, you are reminded that the 
information remains the property of the sender. You must not use, disclose, 
distribute, copy, print or rely on this e-mail. If you have received this 
message in error, please contact the sender immediately and irrevocably delete 
this message and any copies.

From: Yatong Zhang [mailto:bluefl...@gmail.com]
Sent: Sunday, May 4, 2014 5:22 AM
To: user@cassandra.apache.org
Subject: Is the updating compaction strategy from 'sized tiered' to 'leveled' 
automatic or need to be done manually? [heur]

Hi there,

I am updating compaction strategy from 'sized tiered' to 'leveled' and from 
http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it is 
said:
When updating an existing column family, reads and writes can continue as usual 
while leveling of existing sstables is performed in the background.

But I still see many old sstables with very large size and an old file date. So 
I am wondering is the updating of compaction done automatically? If yes, is 
there an estimate of time it will take? If not, what's the steps to do it 
manually?
I've googled a lot but can't find something helpful. Thanks for any feedbacks 
and any help is of great appreciation.

RE: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]

[Viktor Jevdokimov]

Enough to write 1 column and run nodetool flush.

From: Viktor Jevdokimov [mailto:viktor.jevdoki...@adform.com]
Sent: Tuesday, May 6, 2014 12:00 PM
To: user@cassandra.apache.org
Subject: RE: Is the updating compaction strategy from 'sized tiered' to 
'leveled' automatic or need to be done manually? [heur]

I mean insert/write data. When data fills memtable, memtable is flushed to disk 
as sstable, when new sstable is created, Cassandra will check if compaction is 
needed and triggers one.

From: Yatong Zhang [mailto:bluefl...@gmail.com]
Sent: Monday, May 5, 2014 9:54 AM
To: user@cassandra.apache.org
Subject: Re: Is the updating compaction strategy from 'sized tiered' to 
'leveled' automatic or need to be done manually? [heur]

What you mean 'you need write to this CF'? I've changed the schema by using 
CQL3 'alter table' statments.

On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov 
mailto:viktor.jevdoki...@adform.com>> wrote:
To trigger LCS you need to write to this CF and wait when new sstable flushes. 
I can’t find any other way to start LCS.

Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer

Email: viktor.jevdoki...@adform.com
Phone: +370 5 212 3063, Fax +370 5 261 
0453
J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
Follow us on Twitter: @adforminsider
Experience Adform DNA

[Adform News]
[Adform awarded the Best Employer 
2012]


Disclaimer: The information contained in this message and attachments is 
intended solely for the attention and use of the named addressee and may be 
confidential. If you are not the intended recipient, you are reminded that the 
information remains the property of the sender. You must not use, disclose, 
distribute, copy, print or rely on this e-mail. If you have received this 
message in error, please contact the sender immediately and irrevocably delete 
this message and any copies.

From: Yatong Zhang [mailto:bluefl...@gmail.com]
Sent: Sunday, May 4, 2014 5:22 AM
To: user@cassandra.apache.org
Subject: Is the updating compaction strategy from 'sized tiered' to 'leveled' 
automatic or need to be done manually? [heur]

Hi there,

I am updating compaction strategy from 'sized tiered' to 'leveled' and from 
http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra it is 
said:
When updating an existing column family, reads and writes can continue as usual 
while leveling of existing sstables is performed in the background.

But I still see many old sstables with very large size and an old file date. So 
I am wondering is the updating of compaction done automatically? If yes, is 
there an estimate of time it will take? If not, what's the steps to do it 
manually?
I've googled a lot but can't find something helpful. Thanks for any feedbacks 
and any help is of great appreciation.

Re: cassandra snapshots

[Jonathan Lacefield]

What version of Cassandra are you using?  This could be from snapshot
repairs in newer versions of Cassandra.
CASSANDRA-5950

Also, check out the snapshot settings, other than incremental, in the .yaml
file.  There are several snapshot configurations which could have been set
for your cluster.
http://www.datastax.com/documentation/cassandra/2.0/cassandra/configuration/configCassandra_yaml_r.html

Jonathan Lacefield
Solutions Architect, DataStax
(404) 822 3487






On Mon, May 5, 2014 at 3:48 PM, Batranut Bogdan  wrote:

> Hello all
>
> I have a big col family and I see that cassandra is taking snapshots for
> it. I do not have incremental enabled. What are the triggers that start the
> process of taking a snapshot? Is is automatic ?
>
> Thanks
>

Re: Is the updating compaction strategy from 'sized tiered' to 'leveled' automatic or need to be done manually? [heur]

[Yatong Zhang]

Thanks for the reply Viktor, but the whole cluster has been working all the
time and I am sure the cf got writes every second. I still don't see the
huge sstables being split into small ones


On Tue, May 6, 2014 at 5:30 PM, Viktor Jevdokimov <
viktor.jevdoki...@adform.com> wrote:

>  Enough to write 1 column and run nodetool flush.
>
>
>
> *From:* Viktor Jevdokimov [mailto:viktor.jevdoki...@adform.com]
> *Sent:* Tuesday, May 6, 2014 12:00 PM
> *To:* user@cassandra.apache.org
> *Subject:* RE: Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> I mean insert/write data. When data fills memtable, memtable is flushed to
> disk as sstable, when new sstable is created, Cassandra will check if
> compaction is needed and triggers one.
>
>
>
> *From:* Yatong Zhang [mailto:bluefl...@gmail.com ]
> *Sent:* Monday, May 5, 2014 9:54 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> What you mean 'you need write to this CF'? I've changed the schema by
> using CQL3 'alter table' statments.
>
>
>
> On Mon, May 5, 2014 at 2:28 PM, Viktor Jevdokimov <
> viktor.jevdoki...@adform.com> wrote:
>
>  To trigger LCS you need to write to this CF and wait when new sstable
> flushes. I can’t find any other way to start LCS.
>
>
>
> Best regards / Pagarbiai
>
> *Viktor Jevdokimov*
>
> Senior Developer
>
>
>
> Email: viktor.jevdoki...@adform.com
>
> Phone: +370 5 212 3063, Fax +370 5 261 0453
>
> J. Jasinskio 16C, LT-03163 Vilnius, Lithuania
>
> Follow us on Twitter: @adforminsider 
>
> Experience Adform DNA 
>
> [image: Adform News] 
>
> [image: Adform awarded the Best Employer 
> 2012]
>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>
>
> *From:* Yatong Zhang [mailto:bluefl...@gmail.com]
> *Sent:* Sunday, May 4, 2014 5:22 AM
> *To:* user@cassandra.apache.org
> *Subject:* Is the updating compaction strategy from 'sized tiered' to
> 'leveled' automatic or need to be done manually? [heur]
>
>
>
> Hi there,
>
> I am updating compaction strategy from 'sized tiered' to 'leveled' and
> from
> http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandrait is 
> said:
>
> When updating an existing column family, reads and writes can continue as
> usual while leveling of existing sstables is performed in the background.
>
>
>
> But I still see many old sstables with very large size and an old file
> date. So I am wondering is the updating of compaction done automatically?
> If yes, is there an estimate of time it will take? If not, what's the steps
> to do it manually?
>
> I've googled a lot but can't find something helpful. Thanks for any
> feedbacks and any help is of great appreciation.
>
>
>

Re: error in cassandra log file under stress

[Mohica Jasha]

Thanks Robert,

I understand that C* < 2.1 is probably not perfect for prod. But the choice
to use C* 2.0.x was made long ago and we are trying to keep up.
Anyway I filled a jira ticket:
https://issues.apache.org/jira/browse/CASSANDRA-7176




On Mon, May 5, 2014 at 5:51 PM, Robert Coli  wrote:

> On Mon, May 5, 2014 at 2:03 PM, Mohica Jasha wrote:
>
>> Our prod deployment:
>>
>> C* 2.0.5, DC1:3, DC2:3
>>
>
> Not directly related to your bug report, which I would file an Apache JIRA
> regarding, but...
>
> https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/
>
> =Rob
>
>

Re: Avoiding email duplicates when registering users

[Tyler Hobbs]

On Mon, May 5, 2014 at 10:27 AM, Ignacio Martin  wrote:

>
> When a user registers, the server generates a UUID and performs an INSERT
> ... IF NOT EXISTS into the email_to_UUID table. Immediately after, perform
> a SELECT from the same table and see if the read UUID is the same that the
> one we just generated. If it is, we are allowed to INSERT the data in the
> user table, knowing that no other will be doing it.
>

INSERT ... IF NOT EXISTS is the correct thing to do here, but you don't
need to SELECT afterwards.  If the row does exist, the query results will
show that the insert was not applied and the existing row will be returned.


-- 
Tyler Hobbs
DataStax 

Re: Automatic tombstone removal issue (STCS)

[Paulo Ricardo Motta Gomes]

Hello,

Sorry for being persistent, but I'd love to clear my understanding on this.
Has anyone seen single sstable compaction being triggered for STCS sstables
with high tombstone ratio?

Because if the above understanding is correct, the current implementation
almost never triggers this kind of compaction, since the token ranges of a
node's sstable almost always overlap. Could this be a bug or is it expected
behavior?

Thank you,



On Mon, May 5, 2014 at 8:59 AM, Paulo Ricardo Motta Gomes <
paulo.mo...@chaordicsystems.com> wrote:

> Hello,
>
> After noticing that automatic tombstone removal (CASSANDRA-3442) was not
> working in an append-only STCS CF with 40% of droppable tombstone ratio I
> investigated why the compaction was not being triggered in the largest
> SSTable with 16GB and about 70% droppable tombstone ratio.
>
> When the code goes to check if the SSTable is candidate to be compacted
> (AbstractCompactionStrategy.worthDroppingTombstones), it verifies if all
> the others SSTables overlap with the current SSTable by checking if the
> start and end tokens overlap. The problem is that all SSTables contain
> pretty much the whole node token range, so all of them overlap nearly all
> the time, so the automatic tombstone removal never happens. Is there any
> case in STCS where all sstables token ranges DO NOT overlap?
>
> I understand during the tombstone removal process it's necessary to verify
> if the compacted row exists in any other SSTable, but I don't understand
> why it's necessary to verify if the token ranges overlap to decide if a
> tombstone compaction must be executed on a single SSTable with high
> droppable tombstone ratio.
>
> Any clarification would be kindly appreciated.
>
> PS: Cassandra version: 1.2.16
>
> --
> *Paulo Motta*
>
> Chaordic | *Platform*
> *www.chaordic.com.br *
> +55 48 3232.3200
>



-- 
*Paulo Motta*

Chaordic | *Platform*
*www.chaordic.com.br *
+55 48 3232.3200

Re: Automatic tombstone removal issue (STCS)

[Robert Coli]

On Tue, May 6, 2014 at 5:14 PM, Paulo Ricardo Motta Gomes <
paulo.mo...@chaordicsystems.com> wrote:

> Sorry for being persistent, but I'd love to clear my understanding on
> this. Has anyone seen single sstable compaction being triggered for STCS
> sstables with high tombstone ratio?
>
> Because if the above understanding is correct, the current implementation
> almost never triggers this kind of compaction, since the token ranges of a
> node's sstable almost always overlap. Could this be a bug or is it expected
> behavior?
>

Unfortunately, this is probably a better question for either the
cassandra-dev mailing list or the Apache JIRA issues tracker or #cassandra.

I would not at all be surprised if STCS + opportunistic compaction + vnodes
has an edge case like this.

=Rob

Re: Automatic tombstone removal issue (STCS)

[Yuki Morishita]

Hi Paulo,

The reason we check overlap is not to resurrect deleted data by only
dropping tombstone marker from single SSTable.
And we don't want to check row by row to determine if SSTable is
droppable since it takes time, so we use token ranges to determine if
it MAY have droppable columns.

On Tue, May 6, 2014 at 7:14 PM, Paulo Ricardo Motta Gomes
 wrote:
> Hello,
>
> Sorry for being persistent, but I'd love to clear my understanding on this.
> Has anyone seen single sstable compaction being triggered for STCS sstables
> with high tombstone ratio?
>
> Because if the above understanding is correct, the current implementation
> almost never triggers this kind of compaction, since the token ranges of a
> node's sstable almost always overlap. Could this be a bug or is it expected
> behavior?
>
> Thank you,
>
>
>
> On Mon, May 5, 2014 at 8:59 AM, Paulo Ricardo Motta Gomes
>  wrote:
>>
>> Hello,
>>
>> After noticing that automatic tombstone removal (CASSANDRA-3442) was not
>> working in an append-only STCS CF with 40% of droppable tombstone ratio I
>> investigated why the compaction was not being triggered in the largest
>> SSTable with 16GB and about 70% droppable tombstone ratio.
>>
>> When the code goes to check if the SSTable is candidate to be compacted
>> (AbstractCompactionStrategy.worthDroppingTombstones), it verifies if all the
>> others SSTables overlap with the current SSTable by checking if the start
>> and end tokens overlap. The problem is that all SSTables contain pretty much
>> the whole node token range, so all of them overlap nearly all the time, so
>> the automatic tombstone removal never happens. Is there any case in STCS
>> where all sstables token ranges DO NOT overlap?
>>
>> I understand during the tombstone removal process it's necessary to verify
>> if the compacted row exists in any other SSTable, but I don't understand why
>> it's necessary to verify if the token ranges overlap to decide if a
>> tombstone compaction must be executed on a single SSTable with high
>> droppable tombstone ratio.
>>
>> Any clarification would be kindly appreciated.
>>
>> PS: Cassandra version: 1.2.16
>>
>> --
>> Paulo Motta
>>
>> Chaordic | Platform
>> www.chaordic.com.br
>> +55 48 3232.3200
>
>
>
>
> --
> Paulo Motta
>
> Chaordic | Platform
> www.chaordic.com.br
> +55 48 3232.3200



-- 
Yuki Morishita
 t:yukim (http://twitter.com/yukim)