Transaction Support in Cassandra

[Bhupendra Baraiya]

HI ,

  We are currently thinking of moving from Sql to Cassandra but our major worry 
here is we have a system which required concurrent 200 connection at time and 
they do the updates and insert and delete which requires Transaction support

When I searched o Google about these what I got is Cassandra support 
Lightweight Transaction And Quorum consistency is there any other way we can 
achieve the same Transaction support in Cassandra as available in Ms Sql Server

Below is the link where Team had actually build Transaction support in 
Cassandra same as there is Sql Server is it available to Public (all those who 
download latest Cassandra)

http://www.planetcassandra.org/blog/interview/acid-transactions-with-apache-cassandra-social-network-ok-ru-moves-from-microsoft-sql-to-handle-200m-users/

Thanks and regards,

Bhupendra Baraiya
Continuum Managed Services, LLC.
p: 902-933-0019
e: bhupendra.bara...@continuum.net
w: continuum.net
[http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]

Re: Transaction Support in Cassandra

[DuyHai Doan]

First Ok.ru support for acid transactions is using a complete FORK of
Apache Cassandra so people using it need to maintain the fork themselves.

Second, as far as I know, they don't intend to publish the source code so
it's not really available in the public space either. The reason is that
their architecture has been tailored for their own usage and hardware so I
'm not sure it can be adapted for general purpose public usage...
Le 22 mars 2016 08:12, "Bhupendra Baraiya" 
a écrit :

> HI ,
>
>
>
>   We are currently thinking of moving from Sql to Cassandra but our major
> worry here is we have a system which required concurrent 200 connection at
> time and they do the updates and insert and delete which requires
> Transaction support
>
>
>
> When I searched o Google about these what I got is Cassandra support
> Lightweight Transaction And Quorum consistency is there any other way we
> can achieve the same Transaction support in Cassandra as available in Ms
> Sql Server
>
>
>
> Below is the link where Team had actually build Transaction support in
> Cassandra same as there is Sql Server is it available to Public (all those
> who download latest Cassandra)
>
>
>
>
> http://www.planetcassandra.org/blog/interview/acid-transactions-with-apache-cassandra-social-network-ok-ru-moves-from-microsoft-sql-to-handle-200m-users/
>
>
>
> Thanks and regards,
>
>
>
> *Bhupendra Baraiya*
>
> Continuum Managed Services, LLC.
>
> p: 902-933-0019
>
> e: bhupendra.bara...@continuum.net
>
> w: continuum.net
>
> [image:
> http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]
> 
>
>
>

Re: disk space used vs nodetool status

[Carlos Alonso]

I'd say you have snapshots holding disk space.

Check it with nodetool listsnapshots. A snapshot is automatically taken on
destructive actions (drop, truncate...) and is basically a hard link to the
involved SSTables, so it's not considered as data load from Cassandra but
it is effectively using disk space.

Hope this helps.

Carlos Alonso | Software Engineer | @calonso 

On 22 March 2016 at 07:57, Anishek Agarwal  wrote:

> Hello,
>
> Using cassandra 2.0.17  on one of the 7 nodes i see that the "Load" column
> from nodetool status
> shows around 279.34 GB where as doing df -h on the two mounted disks the
> total is about 400GB any reason of why this difference could show up and
> how do i go about finding the cause for this ?
>
> Thanks In Advance.
> Anishek
>

RE: Transaction Support in Cassandra

[Bhupendra Baraiya]

Ok so we will not get what they had developed

So what will be the closest Alternative of Transaction in Cassandra

Thanks and regards,

Bhupendra Baraiya
Continuum Managed Services, LLC.
p: 902-933-0019
e: bhupendra.bara...@continuum.net
w: continuum.net
[http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]

From: DuyHai Doan [mailto:doanduy...@gmail.com]
Sent: March 22, 2016 1:20 PM
To: user@cassandra.apache.org
Subject: Re: Transaction Support in Cassandra


First Ok.ru support for acid transactions is using a complete FORK of Apache 
Cassandra so people using it need to maintain the fork themselves.

Second, as far as I know, they don't intend to publish the source code so it's 
not really available in the public space either. The reason is that their 
architecture has been tailored for their own usage and hardware so I 'm not 
sure it can be adapted for general purpose public usage...
Le 22 mars 2016 08:12, "Bhupendra Baraiya" 
mailto:bhupendra.bara...@continuum.net>> a 
écrit :
HI ,

  We are currently thinking of moving from Sql to Cassandra but our major worry 
here is we have a system which required concurrent 200 connection at time and 
they do the updates and insert and delete which requires Transaction support

When I searched o Google about these what I got is Cassandra support 
Lightweight Transaction And Quorum consistency is there any other way we can 
achieve the same Transaction support in Cassandra as available in Ms Sql Server

Below is the link where Team had actually build Transaction support in 
Cassandra same as there is Sql Server is it available to Public (all those who 
download latest Cassandra)

http://www.planetcassandra.org/blog/interview/acid-transactions-with-apache-cassandra-social-network-ok-ru-moves-from-microsoft-sql-to-handle-200m-users/

Thanks and regards,

Bhupendra Baraiya
Continuum Managed Services, LLC.
p: 902-933-0019
e: bhupendra.bara...@continuum.net
w: continuum.net
[http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]

Re: Transaction Support in Cassandra

[DuyHai Doan]

"So what will be the closest Alternative of Transaction in Cassandra"
--> A batch of mutations with LightweightTransaction for a single partition
or simple LightweightTransaction

On Tue, Mar 22, 2016 at 11:04 AM, Bhupendra Baraiya <
bhupendra.bara...@continuum.net> wrote:

> Ok so we will not get what they had developed
>
>
>
> So what will be the closest Alternative of Transaction in Cassandra
>
>
>
> Thanks and regards,
>
>
>
> *Bhupendra Baraiya*
>
> Continuum Managed Services, LLC.
>
> p: 902-933-0019
>
> e: bhupendra.bara...@continuum.net
>
> w: continuum.net
>
> [image:
> http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]
> 
>
>
>
> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
> *Sent:* March 22, 2016 1:20 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Transaction Support in Cassandra
>
>
>
> First Ok.ru support for acid transactions is using a complete FORK of
> Apache Cassandra so people using it need to maintain the fork themselves.
>
> Second, as far as I know, they don't intend to publish the source code so
> it's not really available in the public space either. The reason is that
> their architecture has been tailored for their own usage and hardware so I
> 'm not sure it can be adapted for general purpose public usage...
>
> Le 22 mars 2016 08:12, "Bhupendra Baraiya" <
> bhupendra.bara...@continuum.net> a écrit :
>
> HI ,
>
>
>
>   We are currently thinking of moving from Sql to Cassandra but our major
> worry here is we have a system which required concurrent 200 connection at
> time and they do the updates and insert and delete which requires
> Transaction support
>
>
>
> When I searched o Google about these what I got is Cassandra support
> Lightweight Transaction And Quorum consistency is there any other way we
> can achieve the same Transaction support in Cassandra as available in Ms
> Sql Server
>
>
>
> Below is the link where Team had actually build Transaction support in
> Cassandra same as there is Sql Server is it available to Public (all those
> who download latest Cassandra)
>
>
>
>
> http://www.planetcassandra.org/blog/interview/acid-transactions-with-apache-cassandra-social-network-ok-ru-moves-from-microsoft-sql-to-handle-200m-users/
>
>
>
> Thanks and regards,
>
>
>
> *Bhupendra Baraiya*
>
> Continuum Managed Services, LLC.
>
> p: 902-933-0019
>
> e: bhupendra.bara...@continuum.net
>
> w: continuum.net
>
> [image:
> http://cdn2.hubspot.net/hub/281750/file-393087232-png/img/logos/email-continuum-logo-151x26.png]
> 
>
>
>
>

Re: disk space used vs nodetool status

[Anishek Agarwal]

Thanks Carlos,

We didn't do any actions that would create a snapshot, and i couldn't find
the command in 2.0.17, but i found the respective snapshot directories and
they were created from more than a couple of months ago so, i it might be
that i might have forgotten, its fine now, i have cleared them.

anishek

On Tue, Mar 22, 2016 at 3:20 PM, Carlos Alonso  wrote:

> I'd say you have snapshots holding disk space.
>
> Check it with nodetool listsnapshots. A snapshot is automatically taken on
> destructive actions (drop, truncate...) and is basically a hard link to the
> involved SSTables, so it's not considered as data load from Cassandra but
> it is effectively using disk space.
>
> Hope this helps.
>
> Carlos Alonso | Software Engineer | @calonso 
>
> On 22 March 2016 at 07:57, Anishek Agarwal  wrote:
>
>> Hello,
>>
>> Using cassandra 2.0.17  on one of the 7 nodes i see that the "Load"
>> column from nodetool status
>> shows around 279.34 GB where as doing df -h on the two mounted disks the
>> total is about 400GB any reason of why this difference could show up and
>> how do i go about finding the cause for this ?
>>
>> Thanks In Advance.
>> Anishek
>>
>
>

Re: Transaction Support in Cassandra

[Emīls Šolmanis]

That said, if you *really* want transactionality other than what's provided
by batches, you should probably look into a more suitable solution. Using
Cassandra exclusively with lightweight transactions at quorum writes will
put serious load on it, and you probably won't achieve decent write
throughput. Closest you can come to that sounds reasonable is batched
mutations at single consistency with quorum reads.

You might also want to read through the docs, e.g.,

http://docs.datastax.com/en/cassandra/2.2/cassandra/architecture/archIntro.html

since it seems you don't understand some fundamental concepts of how
Cassandra operates.

On Tue, 22 Mar 2016 at 10:08 DuyHai Doan  wrote:

> "So what will be the closest Alternative of Transaction in Cassandra"
> --> A batch of mutations with LightweightTransaction for a single
> partition or simple LightweightTransaction
>
> On Tue, Mar 22, 2016 at 11:04 AM, Bhupendra Baraiya <
> bhupendra.bara...@continuum.net> wrote:
>
>> Ok so we will not get what they had developed
>>
>>
>>
>> So what will be the closest Alternative of Transaction in Cassandra
>>
>>
>>
>> Thanks and regards,
>>
>>
>>
>> *Bhupendra Baraiya*
>>
>> Continuum Managed Services, LLC.
>>
>> p: 902-933-0019
>>
>> e: bhupendra.bara...@continuum.net
>>
>> w: continuum.net
>>
>> 
>> [image: image001.png]
>>
>>
>>
>> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
>> *Sent:* March 22, 2016 1:20 PM
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: Transaction Support in Cassandra
>>
>>
>>
>> First Ok.ru support for acid transactions is using a complete FORK of
>> Apache Cassandra so people using it need to maintain the fork themselves.
>>
>> Second, as far as I know, they don't intend to publish the source code so
>> it's not really available in the public space either. The reason is that
>> their architecture has been tailored for their own usage and hardware so I
>> 'm not sure it can be adapted for general purpose public usage...
>>
>> Le 22 mars 2016 08:12, "Bhupendra Baraiya" <
>> bhupendra.bara...@continuum.net> a écrit :
>>
>> HI ,
>>
>>
>>
>>   We are currently thinking of moving from Sql to Cassandra but our major
>> worry here is we have a system which required concurrent 200 connection at
>> time and they do the updates and insert and delete which requires
>> Transaction support
>>
>>
>>
>> When I searched o Google about these what I got is Cassandra support
>> Lightweight Transaction And Quorum consistency is there any other way we
>> can achieve the same Transaction support in Cassandra as available in Ms
>> Sql Server
>>
>>
>>
>> Below is the link where Team had actually build Transaction support in
>> Cassandra same as there is Sql Server is it available to Public (all those
>> who download latest Cassandra)
>>
>>
>>
>>
>> http://www.planetcassandra.org/blog/interview/acid-transactions-with-apache-cassandra-social-network-ok-ru-moves-from-microsoft-sql-to-handle-200m-users/
>>
>>
>>
>> Thanks and regards,
>>
>>
>>
>> *Bhupendra Baraiya*
>>
>> Continuum Managed Services, LLC.
>>
>> p: 902-933-0019
>>
>> e: bhupendra.bara...@continuum.net
>>
>> w: continuum.net
>>
>> 
>> [image: image001.png]
>>
>>
>>
>>
>

Scenarios when a node can be missing writes

[Hannu Kröger]

Hi,

I'm trying to reason the possible scenarios when a node of a C* cluster is
not getting the writes and the data needs some sort of anti-entropy
(repair, read-repair, etc.). In what cases does the coordinator not realize
that a write failed and doesn't replay the write from hinted handoff table?

1) The obvious case: A node is down and doesn't recover before hinted
handoff seconds has passed. Or hinted handoff is disabled altogether. In
this case a node will miss data and repair is needed.

2) Another obvious: Disk / filesystem problems. Repair is needed.

3) Node is up and receives the write but is too overloaded to handle it and
drops the mutation. This should be visible in tpstats as dropped mutation.
Does the write still stay in the hinted handoff table of the coordinator
and if so, when is it replayed if the node is seemingly up all the time? Or
is it assumed that if dropped mutations > 0 then repair is needed?

4) Node receives the write but goes down while writing the stuff to disk.
The write should be either in the commit log OR the coordinator does not
receive an OK for it. There is the small window (10s) when OK is given but
data is not synced to disk if "commitlog_sync" is "periodic" (which it is
by default) and "commitlog_sync_period_in_ms" is 10 seconds. Can this be a
cause of node missing writes if the server has stayed on for the whole time
and only cassandra has restarted?

Any other scenarios?

Cheers,
Hannu Kröger

Re: com.datastax.driver.core.Connection "This should not happen and is likely a bug, please report."

[Alex Popescu]

Hi Thunder,

This would be better sent to the Java mailing list
https://groups.google.com/a/lists.datastax.com/forum/#!forum/java-driver-user.
A quick search revealed https://datastax-oss.atlassian.net/browse/JAVA-603
which seem to match the Cassandra/DataStax community version but not the
driver version.

On Mon, Mar 21, 2016 at 9:33 PM, Thunder Stumpges <
thunder.stump...@gmail.com> wrote:

> Hi guys,
>
> So, quick background, we are using Outworkers (previously WebSudos)
> Phantom v 1.22.0 Which appears to be using DataStax driver 3.0.0. We are
> running scala 2.10 inside Samza on Yarn (CDH 5.4.4) with Oracle JDK 8.
>
> This is all pointing at a 3 node dev cluster of DataStax Community v 2.1.13
>
> After running for a short time, the following error is thrown by the
> Datastax driver in the Samza container:
>
> Logger: com.datastax.driver.core.Connection
> Message:
> "Forcing termination of Connection[/:9042-3,
> inFlight=0, closed=true]. This should not happen and is likely a bug,
> please report."
>
> There aren't really any other logs around this time, so not sure what else
> I can provide. It said to "please report" so I guess I am. If what I have
> provided is enough information to actually file a bug report, please let me
> know and I will. Otherwise, what additional information can I collect for
> this?
>
> Thanks,
> Thunder
>
>


-- 
Bests,

Alex Popescu | @al3xandru
Sen. Product Manager @ DataStax

Re-importing Cassandra dependencies with Ant and IntelliJ 14.03

[Nishant Kelkar]

Hi all,

I very recently forked the Cassandra code base as directed here:
https://wiki.apache.org/cassandra/HowToContribute

I then changed my language level to Java 8, and opened the project in
IntelliJ. However, many of the dependencies (DataStax, Apache Commons,
etc.) appear in *red* because IntelliJ cannot find them.

I've used Maven and Gradle before, and both have an option to "Reimport"
dependencies afresh, but I don't see anything similar in Ant (it's my first
time using Ant, btw).

Does anyone know how to re-import all of the Cassandra project's
dependencies in IntelliJ? Here are some details:

OS version: Ubuntu, 14.10
IDE: IntelliJ 14.03
Ant version: 1.9.4
Java version: OpenJDK, 1.8.0_40-internal

I'm on the cassandra-3.1.1 branch in detached head state.

Also, I looked at the build.xml for the DataStax dependencies, but I see
many of them are commented out. Why is that?

Best Regards,
Nishant Kelkar

Re: Re-importing Cassandra dependencies with Ant and IntelliJ 14.03

[Nishant Kelkar]

OK, I changed my branch to a much recent version (3.5) and I've been able
to narrow the problem down to task* build-test* only. Specifically, when I
trigger that task, I get the following error:


*Compile failed; see the compiler error output for details.*

This error points to the following section in build.xml:



Specifically, the encoding="utf-8" part. Anyone know what this error is
about?

Best Regards,
Nishant Kelkar

On Tue, Mar 22, 2016 at 9:09 PM, Nishant Kelkar 
wrote:

> Hi all,
>
> I very recently forked the Cassandra code base as directed here:
> https://wiki.apache.org/cassandra/HowToContribute
>
> I then changed my language level to Java 8, and opened the project in
> IntelliJ. However, many of the dependencies (DataStax, Apache Commons,
> etc.) appear in *red* because IntelliJ cannot find them.
>
> I've used Maven and Gradle before, and both have an option to "Reimport"
> dependencies afresh, but I don't see anything similar in Ant (it's my first
> time using Ant, btw).
>
> Does anyone know how to re-import all of the Cassandra project's
> dependencies in IntelliJ? Here are some details:
>
> OS version: Ubuntu, 14.10
> IDE: IntelliJ 14.03
> Ant version: 1.9.4
> Java version: OpenJDK, 1.8.0_40-internal
>
> I'm on the cassandra-3.1.1 branch in detached head state.
>
> Also, I looked at the build.xml for the DataStax dependencies, but I see
> many of them are commented out. Why is that?
>
> Best Regards,
> Nishant Kelkar
>

Re: Re-importing Cassandra dependencies with Ant and IntelliJ 14.03

[Nishant Kelkar]

OK @all, I think I was finally able to fix the issue. I did this to fix it
(in the top-level dir of my Cassandra install):

nkelkar@nkelkar-laptop:~/cassandra$ *ant -v generate-idea-files*

Is this documented somewhere (in the README.txt, etc.)? It is super useful
for first-time Ant users like myself to know this upfront.

Best Regards,
Nishant Kelkar

On Tue, Mar 22, 2016 at 10:08 PM, Nishant Kelkar 
wrote:

> OK, I changed my branch to a much recent version (3.5) and I've been able
> to narrow the problem down to task* build-test* only. Specifically, when
> I trigger that task, I get the following error:
>
>
> *Compile failed; see the compiler error output for details.*
>
> This error points to the following section in build.xml:
>
>   debug="true"
>  debuglevel="${debuglevel}"
>  destdir="${test.classes}"
>  includeantruntime="false"
>  source="${source.version}"
>  target="${target.version}"
>  encoding="utf-8">
>
> Specifically, the encoding="utf-8" part. Anyone know what this error is
> about?
>
> Best Regards,
> Nishant Kelkar
>
> On Tue, Mar 22, 2016 at 9:09 PM, Nishant Kelkar 
> wrote:
>
>> Hi all,
>>
>> I very recently forked the Cassandra code base as directed here:
>> https://wiki.apache.org/cassandra/HowToContribute
>>
>> I then changed my language level to Java 8, and opened the project in
>> IntelliJ. However, many of the dependencies (DataStax, Apache Commons,
>> etc.) appear in *red* because IntelliJ cannot find them.
>>
>> I've used Maven and Gradle before, and both have an option to "Reimport"
>> dependencies afresh, but I don't see anything similar in Ant (it's my first
>> time using Ant, btw).
>>
>> Does anyone know how to re-import all of the Cassandra project's
>> dependencies in IntelliJ? Here are some details:
>>
>> OS version: Ubuntu, 14.10
>> IDE: IntelliJ 14.03
>> Ant version: 1.9.4
>> Java version: OpenJDK, 1.8.0_40-internal
>>
>> I'm on the cassandra-3.1.1 branch in detached head state.
>>
>> Also, I looked at the build.xml for the DataStax dependencies, but I see
>> many of them are commented out. Why is that?
>>
>> Best Regards,
>> Nishant Kelkar
>>
>
>