Batch isolation within a single partition

[Martin Krasser]

Hello,

I have an application that inserts multiple rows within a single 
partition (= all rows share the same partition key) using a BATCH 
statement. Is it possible that other clients can partially read that 
batch or is the batch application isolated i.e. other clients can only 
read all rows of that batch or none of them?


I understand that a BATCH update to multiple partitions is not isolated 
but I'm not sure if this is also the case for a single partition:


- The article Atomic batches in Cassandra 1.2 
 says 
that /"... we mean atomic in the database sense that if any part of the 
batch succeeds, all of it will. No other guarantees are implied; in 
particular, there is no isolation"/.


- On the other hand, the CQL BATCH 
 docs at 
cassandra.apache.org mention that /"/ /... the [batch] operations are 
still only isolated within a single partition"/ which is a clear 
statement but doesn't it contradict the previous and the next one?


- The CQL BATCH 
 
docs at docs.datastax.com mention that /"... there is no batch 
isolation. Clients are able to read the first updated rows from the 
batch, while other rows are still being updated on the server. However, 
transactional row updates within a partition key are isolated: clients 
cannot read a partial update"/. Also, what does /"transactional row 
updates"/ mean in this context? A lightweight transaction? Something else?


Thanks for any help,
Martin

AW: Leap sec

[Stefan Podkowinski]

This seems to be a good opportunity to dig a bit deeper into certain 
operational aspects of NTP. Some things to be aware of:

How NTP Operates [1]

It can take several minutes before ntpd will update the system time for the 
first time, while all processes are already started. This can be especially 
problematic for newly provisioned systems. You can speed up the time sync 
update by using the iburst keyword with the server configuration command [4].
It took 2-3 minutes on my testing VM without iburst to correct the time after 
startup. Definitely too long, as Cassandra would already have joined the 
cluster. With iburst it was only a few seconds.

Adjustments will be done by either by stepping or slewing the clock. This can 
happen forward and backwards(!). Stepping will set the corrected value right 
away. Slewing will make adjustments in small increments of at most 0.5ms/s by 
speeding the clock up or slowing it down. It will take at least 2000 seconds to 
adjust the clock by slewing 1 second.
* Time offsets < 128ms (default) will be slewed
* Offsets > 128ms will be stepped unless -x flag is used. The threshold value 
can be changed with a tinker option.
* Offsets > 1000ms will cause ntpd to fail and expect the administrator to fix 
the issue (potential hardware error) unless the -g flag is used.

I think it’s fair to say that the –g options should be always set. I’m not 
fully sure about –x yet. Stepping the clock backwards is not a good idea of 
course. The best solution is probably to have –x set and create alerts on 
higher clock skews, that will prompt ops to resolve the situation manually.


Leap second awareness

Make sure your server is leap second aware in advance. You do not want to have 
the second corrected as part of a normal discrepancy detection process. Instead 
ntpd should be aware of the leap second in advance, so it can precisely 
schedule the adjustment.

There're two ways to make your ntpd instance aware of the upcoming leap second. 
This may happen though the upstream ntp server, which may propagate the leap 
second one day in advance. But this doesn't have to be the case. You need to 
find out if the server pool is configured correctly for this.
Another way to make your ntpd leap second aware is to use a custom file [2]. I 
had to modify the apparmor profile to make this work [3].

[1] http://doc.ntp.org/4.1.0/ntpd.htm
[2] http://support.ntp.org/bin/view/Support/ConfiguringNTP#Section_6.14.
[3] http://askubuntu.com/questions/571839/leapseconds-file-permission-denied
[4] http://doc.ntp.org/4.1.0/confopt.htm


Von: cass savy [mailto:casss...@gmail.com]
Gesendet: Freitag, 15. Mai 2015 19:25
An: user@cassandra.apache.org
Betreff: Leap sec

Just curious to know on how you are preparing Prod C* clusters for leap sec.

What are the workaorund other than upgrading kernel to 3.4+?
Are you upgrading clusters to Java 7 or higher on client and C* servers?

Fail to add a node to a cluster - Unknown keyspace system_traces

[Tzach Livyatan]

I have a dev cluster of two Cassandra 2.12 servers on EC2
When adding a new server, I get a
"Streaming error occurred java.lang.AssertionError: Unknown keyspace
system_traces"
exception on the cluster (not the new) server (full log below).

Indeed, when I cqlsh to the cluster server, I see the following:
cqlsh> DESCRIBE KEYSPACES;

system_traces  system

cqlsh> use system_traces;
code=2200 [Invalid query] message="Keyspace 'system_traces' does not exist"

While
cqlsh> DESCRIBE KEYSPACE system_traces;
Do works!

Is it a bug? feature?

Thanks
Tzach



Full log from adding a node:
INFO  [STREAM-INIT-/172.31.19.130:48054] 2015-05-18 11:36:17,734
StreamResultFuture.java:109 - [Stream #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5
ID#0] Creating new streaming plan for Bootst
rap
INFO  [STREAM-INIT-/172.31.19.130:48054] 2015-05-18 11:36:17,735
StreamResultFuture.java:116 - [Stream
#18f25dc0-fd52-11e4-a00a-c1b81a3b68f5, ID#0] Received streaming plan for
Bootstrap
INFO  [STREAM-INIT-/172.31.19.130:48055] 2015-05-18 11:36:17,736
StreamResultFuture.java:116 - [Stream
#18f25dc0-fd52-11e4-a00a-c1b81a3b68f5, ID#0] Received streaming plan for
Bootstrap
ERROR [STREAM-IN-/172.31.19.130] 2015-05-18 11:36:17,777
StreamSession.java:472 - [Stream #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5]
Streaming error occurred
java.lang.AssertionError: Unknown keyspace system_traces
at org.apache.cassandra.db.Keyspace.(Keyspace.java:273)
~[apache-cassandra-2.1.2.jar:2.1.2]
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:122)
~[apache-cassandra-2.1.2.jar:2.1.2]
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:99)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.streaming.StreamSession.getColumnFamilyStores(StreamSession.java:280)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.streaming.StreamSession.addTransferRanges(StreamSession.java:257)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:488)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:420)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:251)
~[apache-cassandra-2.1.2.jar:2.1.2]

[RELEASE] Apache Cassandra 2.0.15 released

[Jake Luciani]

The Cassandra team is pleased to announce the release of Apache Cassandra
version 2.0.15.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is a bug fix release[1] on the 2.0 series. As always, please
pay
attention to the release notes[2] and Let us know[3] if you were to
encounter
any problem.

Enjoy!

[1]: http://goo.gl/G050Kn (CHANGES.txt)
[2]: http://goo.gl/ZyvMnR (NEWS.txt)
[3]: https://issues.apache.org/jira/browse/CASSANDRA

Re: Out of memory on wide row read

[Antoine Blanchet]

Done, https://issues.apache.org/jira/browse/CASSANDRA-9413 . Feel free to
improve the description, I've only copy/paste the first message from Kévin.

Thanks.

On Fri, May 15, 2015 at 9:56 PM, Alprema  wrote:

> I William file a jira for that, thanks
> On May 12, 2015 10:15 PM, "Jack Krupansky" 
> wrote:
>
>> Sounds like it's worth a Jira - Cassandra should protect itself from
>> innocent mistakes or excessive requests from clients. Maybe there should be
>> a timeout or result size (bytes in addition to count) limit. Something.
>> Anything. But OOM seems a tad unfriendly for an innocent mistake. In this
>> particular case, maybe Cassandra could detect the total row size/slice
>> being read and error out on a configurable limiit.
>>
>> -- Jack Krupansky
>>
>> On Tue, May 12, 2015 at 1:57 PM, Robert Coli 
>> wrote:
>>
>>> On Tue, May 12, 2015 at 8:43 AM, Kévin LOVATO 
>>> wrote:
>>>
 My question is the following: Is it possible to prevent Cassandra from
 OOM'ing when a client does this kind of requests? I'd rather have an error
 thrown to the client than a multi-server crash.

>>>
>>> You can provide a default LIMIT clause, but this is based on number of
>>> results and not size.
>>>
>>> Other than that, there are not really great options.
>>>
>>> =Rob
>>>
>>>
>>
>>


-- 
Antoine Blanchet
ABC Arbitrage Asset Management
http://www.abc-arbitrage.com/

-- 

*ABC arbitrage, partenaire officiel du skipper Jean-Pierre Dick // ABC 
arbitrage, official partner of skipper Jean-Pierre Dick // www.jpdick.com 
*
Please consider your environmental responsibility before printing this email
*
Ce message peut contenir des informations confidentielles. Les idées et 
opinions presentées dans ce message sont celles de son auteur, et ne 
représentent pas nécessairement celles du groupe ABC arbitrage.
Au cas où il ne vous serait pas destiné,merci d'en aviser l'expéditeur 
immédiatement et de le supprimer.

This message may contain confidential information. Any views or opinions 
presented are solely those of its author and do not necessarily represent 
those of ABC arbitrage. 
If you are not the intended recipient, please notify the sender immediately 
and delete it.
*

Re: Batch isolation within a single partition

[DuyHai Doan]

Hello Martin

If, and only if you have RF=1, single partition mutations (including
batches) are isolated.

Otherwise, with RF>1, even a simple UPDATE is not isolated because one
client can read the updated value on one replica and another client reads
the old value on another replica



On Mon, May 18, 2015 at 12:32 PM, Martin Krasser 
wrote:

>  Hello,
>
> I have an application that inserts multiple rows within a single partition
> (= all rows share the same partition key) using a BATCH statement. Is it
> possible that other clients can partially read that batch or is the batch
> application isolated i.e. other clients can only read all rows of that
> batch or none of them?
>
> I understand that a BATCH update to multiple partitions is not isolated
> but I'm not sure if this is also the case for a single partition:
>
> - The article Atomic batches in Cassandra 1.2
>  says
> that *"... we mean atomic in the database sense that if any part of the
> batch succeeds, all of it will. No other guarantees are implied; in
> particular, there is no isolation"*.
>
> - On the other hand, the CQL BATCH
>  docs at
> cassandra.apache.org mention that *"* *... the [batch] operations are
> still only isolated within a single partition"* which is a clear
> statement but doesn't it contradict the previous and the next one?
>
> - The CQL BATCH
>  docs
> at docs.datastax.com mention that *"... there is no batch isolation.
> Clients are able to read the first updated rows from the batch, while other
> rows are still being updated on the server. However, transactional row
> updates within a partition key are isolated: clients cannot read a partial
> update"*. Also, what does *"transactional row updates"* mean in this
> context? A lightweight transaction? Something else?
>
> Thanks for any help,
> Martin
>
>

Re: Batch isolation within a single partition

[Martin Krasser]

Hi DuyHai,

thanks for your answer. What if I set RF > 1 and the consistency level 
for reads and writes to QUORUM? Would that isolate the single-partition 
batch update from reads? (I do not consider node failures here between 
the write and the read(s)).


On 19.05.15 07:50, DuyHai Doan wrote:

Hello Martin

If, and only if you have RF=1, single partition mutations (including 
batches) are isolated.


Otherwise, with RF>1, even a simple UPDATE is not isolated because one 
client can read the updated value on one replica and another client 
reads the old value on another replica




On Mon, May 18, 2015 at 12:32 PM, Martin Krasser 
mailto:krass...@googlemail.com>> wrote:


Hello,

I have an application that inserts multiple rows within a single
partition (= all rows share the same partition key) using a BATCH
statement. Is it possible that other clients can partially read
that batch or is the batch application isolated i.e. other clients
can only read all rows of that batch or none of them?

I understand that a BATCH update to multiple partitions is not
isolated but I'm not sure if this is also the case for a single
partition:

- The article Atomic batches in Cassandra 1.2

says that /"... we mean atomic in the database sense that if any
part of the batch succeeds, all of it will. No other guarantees
are implied; in particular, there is no isolation"/.

- On the other hand, the CQL BATCH
 docs at
cassandra.apache.org  mention that
/"/ /... the [batch] operations are still only isolated within a
single partition"/ which is a clear statement but doesn't it
contradict the previous and the next one?

- The CQL BATCH

docs at docs.datastax.com  mention that
/"... there is no batch isolation. Clients are able to read the
first updated rows from the batch, while other rows are still
being updated on the server. However, transactional row updates
within a partition key are isolated: clients cannot read a partial
update"/. Also, what does /"transactional row updates"/ mean in
this context? A lightweight transaction? Something else?

Thanks for any help,
Martin




--
Martin Krasser

blog:http://krasserm.github.io
code:http://github.com/krasserm
twitter: http://twitter.com/mrt1nz