Re: RESTful API for GET

[Cyril Auburtin]

Ok actually in my query call instead of &page=14, I will put
&page=colName141st,

2012/6/12 Sasha Dolgy 

> https://github.com/hmsonline/virgil Brian O'Neill posted this a while ago
> ... sits on top of Cassandra to give you the RESTful API you want
>
> Another option ... http://code.google.com/p/restish/
>
> Or, you could simply build your own ...
>
>
>
> On Tue, Jun 12, 2012 at 8:46 AM, Tom  wrote:
>
>> Hi James,
>>
>> No, Cassandra doesn't supports a RESTful api.
>>
>> As Tamar points out, you have to supply this functionality yourself
>> specifically for your data model.
>>
>> When designing your RESTful server application:
>> - consider using a RESTful framework (for example: Jersey)
>> - use a cassandra client to access your Cassandra data (for example:
>> astyanax)
>>
>> Good luck,
>> Tom
>>
>>
>>
>> On 06/11/2012 11:15 PM, James Pirz wrote:
>>
>>> Hi,
>>>
>>> Thanks for the reply,
>>> But can you tell me how do you form your request URLs,
>>> I mean does Cassandra support a native RESTful api for talking to the
>>> system, and if yes, on which specific port it is listening for the
>>> coming requests ? and what does it expect for the format for the URLs ?
>>>
>>> Thanks in advance,
>>>
>>> James
>>>
>>> On Mon, Jun 11, 2012 at 11:09 PM, Tamar Fraenkel >> > wrote:
>>>
>>>Hi!
>>>I am using java and jersey.
>>>Works fine,
>>>
>>>*Tamar Fraenkel *
>>>
>>>Senior Software Engineer, TOK Media
>>>
>>>Inline image 1
>>>
>>>ta...@tok-media.com 
>>>
>>>Tel: +972 2 6409736
>>>Mob: +972 54 8356490
>>>Fax: +972 2 5612956
>>>
>>>
>>>
>>>
>>>
>>>On Tue, Jun 12, 2012 at 9:06 AM, James Pirz >>> wrote:
>>>
>>>Dear all,
>>>
>>>I am trying to query the system, specifically performing a GET
>>>for a specific key, through Jmeter (or CURL)
>>>and I am wondering what is the best pure RESTful API for the
>>>system (with the lowest overhead) that I can use.
>>>
>>>Thanks,
>>>James
>>>
>>>
>>>
>>>
>>
>
>
> --
> Sasha Dolgy
> sasha.do...@gmail.com
>

Re: Hector code not running when replication factor set to 2

[aaron morton]

What consistency level and replication factor were you using ?

UnavailableException is thrown when less the consistency level nodes are UP. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 11/06/2012, at 10:19 PM, Prakrati Agrawal wrote:

> Dear all
>  
> I had a 2 node cluster with replication factor set to 1. Then I changed the 
> replication factor to 2 and brought down one node so that only 1 node was up 
> and running. Then I ran my Hector code on the running node. But it gave me 
> Unavailable Exception. I also had a Thrift code which ran successfully. I am 
> confused as to why the Hector code did not run. Did I miss something? Please 
> help me.
>  
> Thanks and Regards
> Prakrati
>  
> 
> This email message may contain proprietary, private and confidential 
> information. The information transmitted is intended only for the person(s) 
> or entities to which it is addressed. Any review, retransmission, 
> dissemination or other use of, or taking of any action in reliance upon, this 
> information by persons or entities other than the intended recipient is 
> prohibited and may be illegal. If you received this in error, please contact 
> the sender and delete the message from your system.
> 
> Mu Sigma takes all reasonable steps to ensure that its electronic 
> communications are free from viruses. However, given Internet accessibility, 
> the Company cannot accept liability for any virus introduced by this e-mail 
> or any attachment and you are advised to use up-to-date virus checking 
> software.

RE: Hector code not running when replication factor set to 2

[Prakrati Agrawal]

I am using the consistency level one and replication factor 2

Thanks
Prakrati
From: aaron morton [mailto:aa...@thelastpickle.com]
Sent: Tuesday, June 12, 2012 2:12 PM
To: user@cassandra.apache.org
Subject: Re: Hector code not running when replication factor set to 2

What consistency level and replication factor were you using ?

UnavailableException is thrown when less the consistency level nodes are UP.

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 11/06/2012, at 10:19 PM, Prakrati Agrawal wrote:


Dear all

I had a 2 node cluster with replication factor set to 1. Then I changed the 
replication factor to 2 and brought down one node so that only 1 node was up 
and running. Then I ran my Hector code on the running node. But it gave me 
Unavailable Exception. I also had a Thrift code which ran successfully. I am 
confused as to why the Hector code did not run. Did I miss something? Please 
help me.

Thanks and Regards
Prakrati



This email message may contain proprietary, private and confidential 
information. The information transmitted is intended only for the person(s) or 
entities to which it is addressed. Any review, retransmission, dissemination or 
other use of, or taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is prohibited and may be 
illegal. If you received this in error, please contact the sender and delete 
the message from your system.

Mu Sigma takes all reasonable steps to ensure that its electronic 
communications are free from viruses. However, given Internet accessibility, 
the Company cannot accept liability for any virus introduced by this e-mail or 
any attachment and you are advised to use up-to-date virus checking software.



This email message may contain proprietary, private and confidential 
information. The information transmitted is intended only for the person(s) or 
entities to which it is addressed. Any review, retransmission, dissemination or 
other use of, or taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is prohibited and may be 
illegal. If you received this in error, please contact the sender and delete 
the message from your system.

Mu Sigma takes all reasonable steps to ensure that its electronic 
communications are free from viruses. However, given Internet accessibility, 
the Company cannot accept liability for any virus introduced by this e-mail or 
any attachment and you are advised to use up-to-date virus checking software.

Re: Dead node still being pinged

[aaron morton]

Try purging the hints for 10.10.0.24 using the HintedHandOffManager MBean.

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 12/06/2012, at 3:33 AM, Nicolas Lalevée wrote:

> finally, thanks to the groovy jmx builder, it was not that hard.
> 
> 
> Le 11 juin 2012 à 12:12, Samuel CARRIERE a écrit :
> 
>> If I were you, I would connect (through JMX, with jconsole) to one of the 
>> nodes that is sending messages to an old node, and would have a look at 
>> these MBean : 
>>   - org.apache.net.FailureDetector : does SimpleStates looks good ? (or do 
>> you see an IP of an old node)
> 
> SimpleStates:[/10.10.0.22:DOWN, /10.10.0.24:DOWN, /10.10.0.26:UP, 
> /10.10.0.25:UP, /10.10.0.27:UP]
> 
>>   - org.apache.net.MessagingService : do you see one of the old IP in one of 
>> the attributes ?
> 
> data-5:
> CommandCompletedTasks:
> [10.10.0.22:2, 10.10.0.26:6147307, 10.10.0.27:6084684, 10.10.0.24:2]
> CommandPendingTasks:
> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
> ResponseCompletedTasks:
> [10.10.0.22:1487, 10.10.0.26:6187204, 10.10.0.27:6062890, 10.10.0.24:1495]
> ResponsePendingTasks:
> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
> 
> data-6:
> CommandCompletedTasks:
> [10.10.0.22:2, 10.10.0.27:6064992, 10.10.0.24:2, 10.10.0.25:6308102]
> CommandPendingTasks:
> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:0, 10.10.0.25:0]
> ResponseCompletedTasks:
> [10.10.0.22:1463, 10.10.0.27:6067943, 10.10.0.24:1474, 10.10.0.25:6367692]
> ResponsePendingTasks:
> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:2, 10.10.0.25:0]
> 
> data-7:
> CommandCompletedTasks:
> [10.10.0.22:2, 10.10.0.26:6043653, 10.10.0.24:2, 10.10.0.25:5964168]
> CommandPendingTasks:
> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.24:0, 10.10.0.25:0]
> ResponseCompletedTasks:
> [10.10.0.22:1424, 10.10.0.26:6090251, 10.10.0.24:1431, 10.10.0.25:6094954]
> ResponsePendingTasks:
> [10.10.0.22:4, 10.10.0.26:0, 10.10.0.24:1, 10.10.0.25:0]
> 
>>   - org.apache.net.StreamingService : do you see an old IP in StreamSources 
>> or StreamDestinations ?
> 
> nothing streaming on the 3 nodes.
> nodetool netstats confirmed that.
> 
>>   - org.apache.internal.HintedHandoff : are there non-zero ActiveCount, 
>> CurrentlyBlockedTasks, PendingTasks, TotalBlockedTask ?
> 
> On the 3 nodes, all at 0.
> 
> I don't know much what I'm looking at, but it seems that some 
> ResponsePendingTasks needs to end.
> 
> Nicolas
> 
>> 
>> Samuel 
>> 
>> 
>> 
>> Nicolas Lalevée 
>> 08/06/2012 21:03
>> Veuillez répondre à
>> user@cassandra.apache.org
>> 
>> A
>> user@cassandra.apache.org
>> cc
>> Objet
>> Re: Dead node still being pinged
>> 
>> 
>> 
>> 
>> 
>> 
>> Le 8 juin 2012 à 20:02, Samuel CARRIERE a écrit :
>> 
>>> I'm in the train but just a guess : maybe it's hinted handoff. A look in 
>>> the logs of the new nodes could confirm that : look for the IP of an old 
>>> node and maybe you'll find hinted handoff related messages.
>> 
>> I grepped on every node about every old node, I got nothing since the 
>> "crash".
>> 
>> If it can be of some help, here is some grepped log of the crash:
>> 
>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>> 00:39:30,241 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>> and will not receive data for re-replication of /10.10.0.22
>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>> 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>> and will not receive data for re-replication of /10.10.0.22
>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>> 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>> and will not receive data for re-replication of /10.10.0.22
>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>> 00:39:30,243 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>> and will not receive data for re-replication of /10.10.0.22
>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>> 00:39:30,243 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>> and will not receive data for re-replication of /10.10.0.22
>> system.log.1: INFO [GossipStage:1] 2012-05-06 00:44:33,822 Gossiper.java 
>> (line 818) InetAddress /10.10.0.24 is now dead.
>> system.log.1: INFO [GossipStage:1] 2012-05-06 04:25:23,894 Gossiper.java 
>> (line 818) InetAddress /10.10.0.24 is now dead.
>> system.log.1: INFO [OptionalTasks:1] 2012-05-06 04:25:23,895 
>> HintedHandOffManager.java (line 179) Deleting any stored hints for 
>> /10.10.0.24
>> system.log.1: INFO [GossipStage:1] 2012-05-06 04:25:23,895 
>> StorageService.java (line 1157) Removing token 
>> 127605887595351923798765477786913079296 for /10.10.0.24
>> system.log.1: INFO [GossipStage:1] 2012-05-09 04:26:25,015 Gossiper.java 
>> (line 818) InetAddress /10.10.0.24 is now dead.
>> 
>> 
>> Maybe its the way I have removed nodes ? AFAIR I didn't used the 
>> decommission

Re: Much more native memory used by Cassandra then the configured JVM heap size

[aaron morton]

see http://wiki.apache.org/cassandra/FAQ#mmap

> which cause the OS low memory.
If the memory is used for mmapped access the os can get it back later. 

Is the low free memory causing a problem ?

Cheers


-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 12/06/2012, at 5:52 PM, Jason Tang wrote:

> Hi
> 
> I found some information of this issue
> And seems we can have other strategy for data access to reduce mmap usage, in 
> order to use less memory.
> 
> But I didn't find the document to describe the parameters for Cassandra 1.x, 
> is it a good way to use this parameter to reduce shared memory usage and 
> what's the impact? (btw, our data model is dynamical, which means the 
> although the through put is high, but the life cycle of the data is short, 
> one hour or less).
> 
> "
> # Choices are auto, standard, mmap, and mmap_index_only.
> disk_access_mode: auto
> "
> 
> http://comments.gmane.org/gmane.comp.db.cassandra.user/7390 
> 
> 2012/6/12 Jason Tang 
> See my post, I limit the HVM heap 6G, but actually Cassandra will use more 
> memory which is not calculated in JVM heap. 
> 
> I use top to monitor total memory used by Cassandra.
> 
> =
> -Xms6G -Xmx6G -Xmn1600M
> 
> 2012/6/12 Jeffrey Kesselman 
> Btw.  I suggest you spin up JConsole as it will give you much more detai kon 
> what your VM is actually doing.
> 
> 
>  
> On Mon, Jun 11, 2012 at 9:14 PM, Jason Tang  wrote:
> Hi
> 
>   We have some problem with Cassandra memory usage, we configure the JVM 
> HEAP 6G, but after runing Cassandra for several hours (insert, update, 
> delete). The total memory used by Cassandra go up to 15G, which cause the OS 
> low memory.
>   
>   So I wonder if it is normal to have so many memory used by cassandra?
> 
> And how to limit the native memory used by Cassandra?
> 
> 
> ===
> Cassandra 1.0.3, 64 bit jdk.
> 
> Memory ocupied by Cassandra 15G
>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
>  9567 casadm20   0 28.3g  15g 9.1g S  269 65.1 385:57.65 java
> 
> =
> -Xms6G -Xmx6G -Xmn1600M
> 
>  # ps -ef | grep  9567
> casadm9567 1 55 Jun11 ?05:59:44 /opt/jdk1.6.0_29/bin/java -ea 
> -javaagent:/opt/dve/cassandra/bin/../lib/jamm-0.2.5.jar 
> -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms6G -Xmx6G -Xmn1600M 
> -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC 
> -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 
> -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 
> -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true 
> -Dcom.sun.management.jmxremote.port=6080 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=false 
> -Daccess.properties=/opt/dve/cassandra/conf/access.properties 
> -Dpasswd.properties=/opt/dve/cassandra/conf/passwd.properties 
> -Dpasswd.mode=MD5 -Dlog4j.configuration=log4j-server.properties 
> -Dlog4j.defaultInitOverride=true -cp 
> /opt/dve/cassandra/bin/../conf:/opt/dve/cassandra/bin/../build/classes/main:/opt/dve/cassandra/bin/../build/classes/thrift:/opt/dve/cassandra/bin/../lib/Cassandra-Extensions-1.0.0.jar:/opt/dve/cassandra/bin/../lib/antlr-3.2.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-1.0.3.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-clientutil-1.0.3.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-thrift-1.0.3.jar:/opt/dve/cassandra/bin/../lib/avro-1.4.0-fixes.jar:/opt/dve/cassandra/bin/../lib/avro-1.4.0-sources-fixes.jar:/opt/dve/cassandra/bin/../lib/commons-cli-1.1.jar:/opt/dve/cassandra/bin/../lib/commons-codec-1.2.jar:/opt/dve/cassandra/bin/../lib/commons-lang-2.4.jar:/opt/dve/cassandra/bin/../lib/compress-lzf-0.8.4.jar:/opt/dve/cassandra/bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:/opt/dve/cassandra/bin/../lib/guava-r08.jar:/opt/dve/cassandra/bin/../lib/high-scale-lib-1.1.2.jar:/opt/dve/cassandra/bin/../lib/jackson-core-asl-1.4.0.jar:/opt/dve/cassandra/bin/../lib/jackson-mapper-asl-1.4.0.jar:/opt/dve/cassandra/bin/../lib/jamm-0.2.5.jar:/opt/dve/cassandra/bin/../lib/jline-0.9.94.jar:/opt/dve/cassandra/bin/../lib/json-simple-1.1.jar:/opt/dve/cassandra/bin/../lib/libthrift-0.6.jar:/opt/dve/cassandra/bin/../lib/log4j-1.2.16.jar:/opt/dve/cassandra/bin/../lib/servlet-api-2.5-20081211.jar:/opt/dve/cassandra/bin/../lib/slf4j-api-1.6.1.jar:/opt/dve/cassandra/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/dve/cassandra/bin/../lib/snakeyaml-1.6.jar:/opt/dve/cassandra/bin/../lib/snappy-java-1.0.4.1.jar
>  org.apache.cassandra.thrift.CassandraDaemon
> 
> ==
> # nodetool -h 127.0.0.1 -p 6080 info
> Token: 85070591730234615865843651857942052864
> Gossip active: true
> Load : 20.59 GB
> Generation No: 1339423322
> Uptime (seconds) : 39626
> Heap Memory (MB) : 3418.42 / 5984.00
> Data Center  : datacenter1
> Rack   

Re: Hector code not running when replication factor set to 2

[aaron morton]

What was the exact error stack you got ?

What does node tool ring look look from the node you connected to?

Did you notice any errors in the logs on the machine you connected to ?


Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 12/06/2012, at 8:41 PM, Prakrati Agrawal wrote:

> I am using the consistency level one and replication factor 2
>  
> Thanks
> Prakrati
> From: aaron morton [mailto:aa...@thelastpickle.com] 
> Sent: Tuesday, June 12, 2012 2:12 PM
> To: user@cassandra.apache.org
> Subject: Re: Hector code not running when replication factor set to 2
>  
> What consistency level and replication factor were you using ?
>  
> UnavailableException is thrown when less the consistency level nodes are UP. 
>  
> Cheers
>  
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>  
> On 11/06/2012, at 10:19 PM, Prakrati Agrawal wrote:
> 
> 
> Dear all
>  
> I had a 2 node cluster with replication factor set to 1. Then I changed the 
> replication factor to 2 and brought down one node so that only 1 node was up 
> and running. Then I ran my Hector code on the running node. But it gave me 
> Unavailable Exception. I also had a Thrift code which ran successfully. I am 
> confused as to why the Hector code did not run. Did I miss something? Please 
> help me.
>  
> Thanks and Regards
> Prakrati
>  
>  
> This email message may contain proprietary, private and confidential 
> information. The information transmitted is intended only for the person(s) 
> or entities to which it is addressed. Any review, retransmission, 
> dissemination or other use of, or taking of any action in reliance upon, this 
> information by persons or entities other than the intended recipient is 
> prohibited and may be illegal. If you received this in error, please contact 
> the sender and delete the message from your system.
> 
> Mu Sigma takes all reasonable steps to ensure that its electronic 
> communications are free from viruses. However, given Internet accessibility, 
> the Company cannot accept liability for any virus introduced by this e-mail 
> or any attachment and you are advised to use up-to-date virus checking 
> software.
>  
> 
> This email message may contain proprietary, private and confidential 
> information. The information transmitted is intended only for the person(s) 
> or entities to which it is addressed. Any review, retransmission, 
> dissemination or other use of, or taking of any action in reliance upon, this 
> information by persons or entities other than the intended recipient is 
> prohibited and may be illegal. If you received this in error, please contact 
> the sender and delete the message from your system.
> 
> Mu Sigma takes all reasonable steps to ensure that its electronic 
> communications are free from viruses. However, given Internet accessibility, 
> the Company cannot accept liability for any virus introduced by this e-mail 
> or any attachment and you are advised to use up-to-date virus checking 
> software.

Re: Dead node still being pinged

[Nicolas Lalevée]

Le 12 juin 2012 à 11:03, aaron morton a écrit :

> Try purging the hints for 10.10.0.24 using the HintedHandOffManager MBean.

As far as I could tell, there were no hinted hand off to be delivered. 
Nevertheless I have called "deleteHintsForEndpoint" on every node for the two 
expected to be out nodes.
Nothing changed, I still see packet being send to these old nodes.

I looked closer to ResponsePendingTasks of MessagingService. Actually the 
numbers change, between 0 and about 4. So tasks are ending but new ones come 
just after.

Nicolas

> 
> Cheers
> 
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 12/06/2012, at 3:33 AM, Nicolas Lalevée wrote:
> 
>> finally, thanks to the groovy jmx builder, it was not that hard.
>> 
>> 
>> Le 11 juin 2012 à 12:12, Samuel CARRIERE a écrit :
>> 
>>> If I were you, I would connect (through JMX, with jconsole) to one of the 
>>> nodes that is sending messages to an old node, and would have a look at 
>>> these MBean : 
>>>   - org.apache.net.FailureDetector : does SimpleStates looks good ? (or do 
>>> you see an IP of an old node)
>> 
>> SimpleStates:[/10.10.0.22:DOWN, /10.10.0.24:DOWN, /10.10.0.26:UP, 
>> /10.10.0.25:UP, /10.10.0.27:UP]
>> 
>>>   - org.apache.net.MessagingService : do you see one of the old IP in one 
>>> of the attributes ?
>> 
>> data-5:
>> CommandCompletedTasks:
>> [10.10.0.22:2, 10.10.0.26:6147307, 10.10.0.27:6084684, 10.10.0.24:2]
>> CommandPendingTasks:
>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
>> ResponseCompletedTasks:
>> [10.10.0.22:1487, 10.10.0.26:6187204, 10.10.0.27:6062890, 10.10.0.24:1495]
>> ResponsePendingTasks:
>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
>> 
>> data-6:
>> CommandCompletedTasks:
>> [10.10.0.22:2, 10.10.0.27:6064992, 10.10.0.24:2, 10.10.0.25:6308102]
>> CommandPendingTasks:
>> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:0, 10.10.0.25:0]
>> ResponseCompletedTasks:
>> [10.10.0.22:1463, 10.10.0.27:6067943, 10.10.0.24:1474, 10.10.0.25:6367692]
>> ResponsePendingTasks:
>> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:2, 10.10.0.25:0]
>> 
>> data-7:
>> CommandCompletedTasks:
>> [10.10.0.22:2, 10.10.0.26:6043653, 10.10.0.24:2, 10.10.0.25:5964168]
>> CommandPendingTasks:
>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.24:0, 10.10.0.25:0]
>> ResponseCompletedTasks:
>> [10.10.0.22:1424, 10.10.0.26:6090251, 10.10.0.24:1431, 10.10.0.25:6094954]
>> ResponsePendingTasks:
>> [10.10.0.22:4, 10.10.0.26:0, 10.10.0.24:1, 10.10.0.25:0]
>> 
>>>   - org.apache.net.StreamingService : do you see an old IP in StreamSources 
>>> or StreamDestinations ?
>> 
>> nothing streaming on the 3 nodes.
>> nodetool netstats confirmed that.
>> 
>>>   - org.apache.internal.HintedHandoff : are there non-zero ActiveCount, 
>>> CurrentlyBlockedTasks, PendingTasks, TotalBlockedTask ?
>> 
>> On the 3 nodes, all at 0.
>> 
>> I don't know much what I'm looking at, but it seems that some 
>> ResponsePendingTasks needs to end.
>> 
>> Nicolas
>> 
>>> 
>>> Samuel 
>>> 
>>> 
>>> 
>>> Nicolas Lalevée 
>>> 08/06/2012 21:03
>>> Veuillez répondre à
>>> user@cassandra.apache.org
>>> 
>>> A
>>> user@cassandra.apache.org
>>> cc
>>> Objet
>>> Re: Dead node still being pinged
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Le 8 juin 2012 à 20:02, Samuel CARRIERE a écrit :
>>> 
 I'm in the train but just a guess : maybe it's hinted handoff. A look in 
 the logs of the new nodes could confirm that : look for the IP of an old 
 node and maybe you'll find hinted handoff related messages.
>>> 
>>> I grepped on every node about every old node, I got nothing since the 
>>> "crash".
>>> 
>>> If it can be of some help, here is some grepped log of the crash:
>>> 
>>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>>> 00:39:30,241 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>>> and will not receive data for re-replication of /10.10.0.22
>>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>>> 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>>> and will not receive data for re-replication of /10.10.0.22
>>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>>> 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>>> and will not receive data for re-replication of /10.10.0.22
>>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>>> 00:39:30,243 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>>> and will not receive data for re-replication of /10.10.0.22
>>> system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
>>> 00:39:30,243 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
>>> and will not receive data for re-replication of /10.10.0.22
>>> system.log.1: INFO [GossipStage:1] 2012-05-06 00:44:33,822 Gossiper.java 
>>> (line 818) InetAddress /10.10.0.24 is now dead.
>>> system.log.1: INFO [GossipStage:1] 2012-05-06 04:25:23,894 Gos

Re: Dead node still being pinged

[Nicolas Lalevée]

I have one dirty solution to try: bring data-2 and data-4 back up and down 
again. Is there any way I can tell cassandra to not get any data, so when I 
would get my old node up, no streaming would start ?

cheers,
Nicolas

Le 12 juin 2012 à 12:25, Nicolas Lalevée a écrit :

> Le 12 juin 2012 à 11:03, aaron morton a écrit :
> 
>> Try purging the hints for 10.10.0.24 using the HintedHandOffManager MBean.
> 
> As far as I could tell, there were no hinted hand off to be delivered. 
> Nevertheless I have called "deleteHintsForEndpoint" on every node for the two 
> expected to be out nodes.
> Nothing changed, I still see packet being send to these old nodes.
> 
> I looked closer to ResponsePendingTasks of MessagingService. Actually the 
> numbers change, between 0 and about 4. So tasks are ending but new ones come 
> just after.
> 
> Nicolas
> 
>> 
>> Cheers
>> 
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> 
>> On 12/06/2012, at 3:33 AM, Nicolas Lalevée wrote:
>> 
>>> finally, thanks to the groovy jmx builder, it was not that hard.
>>> 
>>> 
>>> Le 11 juin 2012 à 12:12, Samuel CARRIERE a écrit :
>>> 
 If I were you, I would connect (through JMX, with jconsole) to one of the 
 nodes that is sending messages to an old node, and would have a look at 
 these MBean : 
  - org.apache.net.FailureDetector : does SimpleStates looks good ? (or do 
 you see an IP of an old node)
>>> 
>>> SimpleStates:[/10.10.0.22:DOWN, /10.10.0.24:DOWN, /10.10.0.26:UP, 
>>> /10.10.0.25:UP, /10.10.0.27:UP]
>>> 
  - org.apache.net.MessagingService : do you see one of the old IP in one 
 of the attributes ?
>>> 
>>> data-5:
>>> CommandCompletedTasks:
>>> [10.10.0.22:2, 10.10.0.26:6147307, 10.10.0.27:6084684, 10.10.0.24:2]
>>> CommandPendingTasks:
>>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
>>> ResponseCompletedTasks:
>>> [10.10.0.22:1487, 10.10.0.26:6187204, 10.10.0.27:6062890, 10.10.0.24:1495]
>>> ResponsePendingTasks:
>>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.27:0, 10.10.0.24:0]
>>> 
>>> data-6:
>>> CommandCompletedTasks:
>>> [10.10.0.22:2, 10.10.0.27:6064992, 10.10.0.24:2, 10.10.0.25:6308102]
>>> CommandPendingTasks:
>>> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:0, 10.10.0.25:0]
>>> ResponseCompletedTasks:
>>> [10.10.0.22:1463, 10.10.0.27:6067943, 10.10.0.24:1474, 10.10.0.25:6367692]
>>> ResponsePendingTasks:
>>> [10.10.0.22:0, 10.10.0.27:0, 10.10.0.24:2, 10.10.0.25:0]
>>> 
>>> data-7:
>>> CommandCompletedTasks:
>>> [10.10.0.22:2, 10.10.0.26:6043653, 10.10.0.24:2, 10.10.0.25:5964168]
>>> CommandPendingTasks:
>>> [10.10.0.22:0, 10.10.0.26:0, 10.10.0.24:0, 10.10.0.25:0]
>>> ResponseCompletedTasks:
>>> [10.10.0.22:1424, 10.10.0.26:6090251, 10.10.0.24:1431, 10.10.0.25:6094954]
>>> ResponsePendingTasks:
>>> [10.10.0.22:4, 10.10.0.26:0, 10.10.0.24:1, 10.10.0.25:0]
>>> 
  - org.apache.net.StreamingService : do you see an old IP in StreamSources 
 or StreamDestinations ?
>>> 
>>> nothing streaming on the 3 nodes.
>>> nodetool netstats confirmed that.
>>> 
  - org.apache.internal.HintedHandoff : are there non-zero ActiveCount, 
 CurrentlyBlockedTasks, PendingTasks, TotalBlockedTask ?
>>> 
>>> On the 3 nodes, all at 0.
>>> 
>>> I don't know much what I'm looking at, but it seems that some 
>>> ResponsePendingTasks needs to end.
>>> 
>>> Nicolas
>>> 
 
 Samuel 
 
 
 
 Nicolas Lalevée 
 08/06/2012 21:03
 Veuillez répondre à
 user@cassandra.apache.org
 
 A
 user@cassandra.apache.org
 cc
 Objet
 Re: Dead node still being pinged
 
 
 
 
 
 
 Le 8 juin 2012 à 20:02, Samuel CARRIERE a écrit :
 
> I'm in the train but just a guess : maybe it's hinted handoff. A look in 
> the logs of the new nodes could confirm that : look for the IP of an old 
> node and maybe you'll find hinted handoff related messages.
 
 I grepped on every node about every old node, I got nothing since the 
 "crash".
 
 If it can be of some help, here is some grepped log of the crash:
 
 system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
 00:39:30,241 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
 and will not receive data for re-replication of /10.10.0.22
 system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
 and will not receive data for re-replication of /10.10.0.22
 system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
 00:39:30,242 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
 and will not receive data for re-replication of /10.10.0.22
 system.log.1: WARN [RMI TCP Connection(1037)-10.10.0.26] 2012-05-06 
 00:39:30,243 StorageService.java (line 2417) Endpoint /10.10.0.24 is down 
 and will not receive data for re-replication of /10.1

Subscribe

[Jayesh Thakrar]

Subscribe

Re: Subscribe

[Roshan Dawrani]

On Tue, Jun 12, 2012 at 8:56 PM, Jayesh Thakrar  wrote:

> Subscribe
>

Attempt unsuccessful,

** Was expecting a voice-command in mp3 format **

Odd problem with cli and display of value types

[Holger Hoffstaette]

While trying to play around with 1.1.1 and secondary indexes I just
noticed something odd in cassandra-cli. Example straight from the README:

--> show Mr. Smith

holger>cassandra-cli
[..]
[default@Users] list users;
Using default limit of 100
Using default column limit of 100
---
RowKey: jsmith
=> (column=first, value=John, timestamp=1339507271651000)
=> (column=last, value=Smith, timestamp=1339507280745000)

1 Row Returned.
Elapsed time: 0 msec(s).

--> Hello Mr. Smith with no age.
--> You should be 64 years old:

[default@Users] set Users[jsmith][age] = long(64);
Value inserted.
Elapsed time: 16 msec(s).

[default@Users] list users;
Using default limit of 100
Using default column limit of 100
---
RowKey: jsmith
=> (column=age, value=64, timestamp=1339513585914000)
=> (column=first, value=John, timestamp=1339507271651000)
=> (column=last, value=Smith, timestamp=1339507280745000)

1 Row Returned.
Elapsed time: 0 msec(s).
[default@Users]

--> That worked, as expected. Exit & restart the cli

holger>cassandra-cli
[..]
[default@Users] list users;
Using default limit of 100
Using default column limit of 100
---
RowKey: jsmith
=> (column=age, value=   @, timestamp=1339513585914000)
=> (column=first, value=John, timestamp=1339507271651000)
=> (column=last, value=Smith, timestamp=1339507280745000)

1 Row Returned.
Elapsed time: 78 msec(s).
[default@Users]

// age=@ you say?

I understand of course that since the default validation class is set to
UTF8 I should have inserted '64' as age and not the long(64) as given in
the README - probably an oversight/bug/typo. The README uses 42 as value,
which results in a * as output. To verify the behaviour I used 64, which
is the ASCII value of @.

What I find more curious is that the cli displays the value in
human-readable form immediately after insertion, yet a new session
displays it in "native" form (as it should). Should it not always display
the value according to the validation class, i.e. show the @ immediately
after insertion?

thanks,
Holger

RE: Setting column to null

[Leonid Ilyevsky]

Thanks Roshni,

I actually looked at Hector, didn't like it that much. Too many methods, feels 
like there are ten ways to do the same thing, but very difficult to find the 
one you need. Many packages look like they are simply migrated from Cassandra 
and renamed, what is the point? In my opinion, it is not well architectured API.
Anyway, I prefer simplicity of CQL, I want to just prepare and execute 
statements. By the way, the manual for CQL is decent compare to the jungle of 
Hector (and Cassandra API too).

-Original Message-
From: Roshni Rajagopal [mailto:roshni.rajago...@wal-mart.com]
Sent: Tuesday, June 12, 2012 1:20 AM
To: user@cassandra.apache.org
Subject: Re: Setting column to null

Leonid,


Are you using some client for doing these operations..?

Hector is a java client which provides APIs for adding/deleting columns to
a column family in cassandra.
I don¹t think you really need to write your wrapper in this format- you
are restricting the number of columns it can use etc.I suggest your code
can  accept user input to get col family name, operation, keys  , and
operation, and accordingly call the appropriate hector API for
adding/deleting data.


Regards,
Roshni


On 11/06/12 7:20 PM, "Leonid Ilyevsky"  wrote:

>Thanks, I understand what you are telling me. Obviously deleting the
>column is the proper way to do this in Cassandra.
>What I was looking for, is some convenient wrapper on top of that which
>will do it for me. Here is my scenario.
>
>I have a function that takes a record to be saved in Cassandra (array of
>objects, or Map). Let say, it can have up to 8 columns. I
>prepare a statement like this:
>
>Insert into  values(?, ?, ?, ?, ?, ?, ?, ?)
>
>If I somehow could put null when I execute it, it would be enough to
>prepare that statement once and execute it multiple times. I would then
>expect that when some element is null, the corresponding column is not
>inserted (for the new key) or deleted (for the existing key).
>The way it is now, in my code I have to examine which columns are present
>and which are not, depending on that I have to generate customized
>statement, and it is going to be different for the case of existing key
>versus case of the new key.
>Isn't this too much hassle?
>
>Related question. I assumed that prepared statement in Cassandra is there
>for the same reason as in RDBMS, that is, for efficiency. In the above
>scenario, how expensive is it to execute specialized statement for every
>record compare to prepared statement executed multiple times?
>
>If I need to execute those specialized statements, should I still use
>prepared statement or should I just generate a string with everything in
>ascii format?
>
>-Original Message-
>From: Roshni Rajagopal [mailto:roshni.rajago...@wal-mart.com]
>Sent: Monday, June 11, 2012 12:58 AM
>To: user@cassandra.apache.org
>Subject: Re: Setting column to null
>
>Would you want to view data like this "there was a key, which had this
>column , but now it does not have any value as of this time."
>
>Unless you specifically want this information, I believe you should just
>delete the column, rather than have an alternate value for NULL or create
>a composite column.
>
>Because in cassandra that¹s the way deletion is dealt with, putting NULLs
>is the way we deal with it in RDBMS because we have a fixed number of
>columns which always have to have some value, even if its NULL, and we
>have to have the same set of columns for every row.
>In Cassandara, we can delete the column, and in most scenarios that¹s
>what we should do, unless we specifically want to preserve some history
>that this column was turned null at this timeŠEach row can have different
>columns.
>
>Regards,
>Roshni
>
>From: Edward Capriolo
>mailto:edlinuxg...@gmail.com>>
>Reply-To: "user@cassandra.apache.org"
>mailto:user@cassandra.apache.org>>
>To: "user@cassandra.apache.org"
>mailto:user@cassandra.apache.org>>
>Subject: Re: Setting column to null
>
>Your best bet is to define the column as a composite column where one
>part represents is null and the other part is the data.
>
>On Friday, June 8, 2012, shashwat shriparv
>mailto:dwivedishash...@gmail.com>> wrote:
>> What you can do is you can define some specific variable like
>>"NULLDATA" some thing like that to update in columns that does have value
>>
>>
>> On Fri, Jun 8, 2012 at 11:58 PM, aaron morton
>>mailto:aa...@thelastpickle.com>> wrote:
>>
>> You don't nee to set columns to null, delete the column instead.
>> Cheers
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> On 8/06/2012, at 9:34 AM, Leonid Ilyevsky wrote:
>>
>> Is it possible to explicitly set a column value to null?
>>
>> I see that if insert statement does not include a specific column, that
>>column comes up as null (assuming we are creating a record with new
>>unique key).
>> But if we want to update a record,

Number format in cqlsh

[Leonid Ilyevsky]

Is there a way to control the floating numbers format in the cqlsh output? I 
need more digits than it gives by default (in my tests, I see only one digit 
after the point).

This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately.  This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.

Re: Number format in cqlsh

[paul cannon]

No, but a way could be added pretty easily. File a ticket on Jira?

p

On Tuesday, June 12, 2012, Leonid Ilyevsky wrote:

> Is there a way to control the floating numbers format in the cqlsh output?
> I need more digits than it gives by default (in my tests, I see only one
> digit after the point).
>
> This email, along with any attachments, is confidential and may be legally
> privileged or otherwise protected from disclosure. Any unauthorized
> dissemination, copying or use of the contents of this email is strictly
> prohibited and may be in violation of law. If you are not the intended
> recipient, any disclosure, copying, forwarding or distribution of this
> email is strictly prohibited and this email and any attachments should be
> deleted immediately.  This email and any attachments do not constitute an
> offer to sell or a solicitation of an offer to purchase any interest in any
> investment vehicle sponsored by Moon Capital Management LP ("Moon
> Capital"). Moon Capital does not provide legal, accounting or tax advice.
> Any statement regarding legal, accounting or tax matters was not intended
> or written to be relied upon by any person as advice. Moon Capital does not
> waive confidentiality or privilege as a result of this email.
>

select count(*) returns 10000

[Leonid Ilyevsky]

The "select count(*) ..." query returns correct count only if it is <= 1, 
otherwise it returns exactly 1.
This happens in both Java API and cqlsh.
Can somebody verify?

This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately.  This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.

Disappearing keyspaces in Cassandra 1.1

[Oleg Dulin]

I am using cassandra 1.1.0 on a 3 node environment. I just truncated a 
few column families then restarted the nodes. now when I restarted them 
it says my keyspace doesn't exist. The data for the keyspace is still 
in the data directory. Does anyone know what could have caused this?

Re: Setting column to null

[Derek Williams]

On Tue, Jun 12, 2012 at 9:59 AM, Leonid Ilyevsky 
 wrote:

> Thanks Roshni,
>
> I actually looked at Hector, didn't like it that much. Too many methods,
> feels like there are ten ways to do the same thing, but very difficult to
> find the one you need. Many packages look like they are simply migrated
> from Cassandra and renamed, what is the point? In my opinion, it is not
> well architectured API.
> Anyway, I prefer simplicity of CQL, I want to just prepare and execute
> statements. By the way, the manual for CQL is decent compare to the jungle
> of Hector (and Cassandra API too).


There are some related tickets for null support, of course that doesn't
mean it will actually get in:

https://issues.apache.org/jira/browse/CASSANDRA-4147
https://issues.apache.org/jira/browse/CASSANDRA-3783

It would make prepared statements more flexible if there was some amount of
null support.

My only other wishlist item is the ability to set the timestamp, ttl, and
consistency level using prepared statements.

-- 
Derek Williams

Re: select count(*) returns 10000

[Derek Williams]

It's a known issue, here is a bit extra info on it:
http://stackoverflow.com/questions/8795923/wrong-count-with-cassandra-cql

On Tue, Jun 12, 2012 at 12:40 PM, Leonid Ilyevsky  wrote:

> The "select count(*) ..." query returns correct count only if it is <=
> 1, otherwise it returns exactly 1.
> This happens in both Java API and cqlsh.
> Can somebody verify?
>
> This email, along with any attachments, is confidential and may be legally
> privileged or otherwise protected from disclosure. Any unauthorized
> dissemination, copying or use of the contents of this email is strictly
> prohibited and may be in violation of law. If you are not the intended
> recipient, any disclosure, copying, forwarding or distribution of this
> email is strictly prohibited and this email and any attachments should be
> deleted immediately.  This email and any attachments do not constitute an
> offer to sell or a solicitation of an offer to purchase any interest in any
> investment vehicle sponsored by Moon Capital Management LP ("Moon
> Capital"). Moon Capital does not provide legal, accounting or tax advice.
> Any statement regarding legal, accounting or tax matters was not intended
> or written to be relied upon by any person as advice. Moon Capital does not
> waive confidentiality or privilege as a result of this email.
>



-- 
Derek Williams

RE: Number format in cqlsh

[Leonid Ilyevsky]

Filed the ticket:
Cassandra   CASSANDRA-4336

From: paul cannon [mailto:p...@datastax.com]
Sent: Tuesday, June 12, 2012 12:49 PM
To: user@cassandra.apache.org
Subject: Re: Number format in cqlsh

No, but a way could be added pretty easily. File a ticket on Jira?

p

On Tuesday, June 12, 2012, Leonid Ilyevsky wrote:
Is there a way to control the floating numbers format in the cqlsh output? I 
need more digits than it gives by default (in my tests, I see only one digit 
after the point).

This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately.  This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.


This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately. This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP (“Moon Capital”). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.

Re: enforcing ordering

[Franc Carter]

On Thu, May 10, 2012 at 9:05 PM, aaron morton wrote:

> Kewl.
>
> I'd be interested to know what you come up with.
>


Hi,

it's taken some thought, however we know have a data model that we like and
it does indeed make our major concerns non-existent ;-) So I thought I'd
explain it in case someone is doing something sufficiently close to us that
the model is useful (and in case we are doing something silly - I hope not).

The data set consist of 30 years of daily data for several million
entities, the data for each is a small number of different record types (<
10) where  is unique. Each record_type can have a
couple of hundred key/value pairs.

The query that we need to do is

  Set_of_Values = Get(, , )

Where set_of_keys is likely to be most of the keys that are valid for the
entities.

One slight complication (the one that sparked my initial question) is that
there are also corrections that completely replace the data for an
, multiple versions of the corrections can be
transmitted, but only one correction per entity/day/record_type

The data model that we have designed has a single Column Family keyed by
the entity with a composite column name consisting of
 with the value being a protobuf packing of the
key/value pairs from the record. The version is the 'receipt data of the
data' - 'date the data is for'. The properties of this that we like are:-

* Record insertion is idempotent allowing for multiple active/active order
independent loaders, this is a really big win for us(1).
* The random partitioner gives us good scalability across the entity
dimension which is the largest dimension.
* The column ordering makes it easy to find the most recent 'correct' value
for an entity on a day.
* The Column ordering give us reasonably efficient date range queries

There are a couple of implications of this data model:-

* We store more data than we have to in the ideal world.
* we push the work of decoding/extracting information of the protobuf on to
the clients along with some of the version management.

My view is that this a reasonable trade-off for systems that can have large
numbers of clients that are independent of each other as scaling client
machines is not hard.

Feedback welcome

cheers

(1) It's important as it allows us to use a large number of loading
processes to insert the historical data that is pretty large in a short
period of time.


>
> Cheers
>
>   -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 10/05/2012, at 3:03 PM, Franc Carter wrote:
>
>
>
> On Tue, May 8, 2012 at 8:21 PM, Franc Carter wrote:
>
>> On Tue, May 8, 2012 at 8:09 PM, aaron morton wrote:
>>
>>> Can you store the corrections in a separate CF?
>>>
>>
> We sat down and thought about this harder - it looks like a good solution
> for us that may makel other hard problems go away - thanks.
>
> cheers
>
>
>> Yes, I thought of that, but that turns on read in to two ;-(
>>
>>
>>>
>>> When the client reads the key, reads from the original the corrects CF
>>> at the same time. Apply the correction only on the client side.
>>>
>>> When you have confirmed the ingest has completed, run a background jobs
>>> to apply the corrections, store the updated values and delete the
>>> correction data.
>>>
>>
>> I was thinking down this path, but I ended up chasing the rabbit down a
>> deep hole of race conditions . . .
>>
>> cheers
>>
>>
>>>
>>> Cheers
>>>
>>>   -
>>> Aaron Morton
>>> Freelance Developer
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 8/05/2012, at 9:35 PM, Franc Carter wrote:
>>>
>>>
>>> Hi,
>>>
>>> I'm wondering if there is a common 'pattern' to address a scenario we
>>> will have to deal with.
>>>
>>> We will be storing a set of Column/Value pairs per Key where the
>>> Column/Values are read from a set of files that we download regularly. We
>>> need the loading to be resilient and we can receive corrections for some of
>>> the Column/Values that can only be loaded after the initial data has been
>>> inserted.
>>>
>>> The challenge we have is that we have a strong preference for
>>> active/active loading of data and can't see how to achieve this without
>>> some form of serialisation (which Cassandra doesn't support - correct ?)
>>>
>>> thanks
>>>
>>> --
>>> *Franc Carter* | Systems architect | Sirca Ltd
>>>  
>>> franc.car...@sirca.org.au | www.sirca.org.au
>>> Tel: +61 2 9236 9118
>>>  Level 9, 80 Clarence St, Sydney NSW 2000
>>> PO Box H58, Australia Square, Sydney NSW 1215
>>>
>>>
>>>
>>
>>
>> --
>> *Franc Carter* | Systems architect | Sirca Ltd
>>  
>> franc.car...@sirca.org.au | www.sirca.org.au
>> Tel: +61 2 9236 9118
>>  Level 9, 80 Clarence St, Sydney NSW 2000
>> PO Box H58, Australia Square, Sydney NSW 1215
>>
>>
>
>
> --
> *Franc Carter* | Systems architect | Sirca Ltd
>  
> franc.car...@sirca.org.au | www.sirca.org.au
> Tel: +61 2 9236 9118
>  Level 9, 80 Clarence St, Sydney NSW 2000
> PO Box H58, Australia Square, Sydney NSW

Re: Failing operations & repair

[crypto five]

It would be really great to look at your slides. Do you have any plans to
share your presentation?

On Sat, Jun 9, 2012 at 1:14 AM, Віталій Тимчишин  wrote:

> Thanks a lot. I was not sure if coordinator somehow tries to "roll-back"
> transactions that failed to reach it's consistency level.
> (Yet I could not imagine a method to do this, without 2-phase commit :) )
>
>
> 2012/6/8 aaron morton 
>
>> I am making some cassandra presentations in Kyiv and would like to check
>> that I am telling people truth :)
>>
>> Thanks for spreading the word :)
>>
>> 1) Failed (from client-side view) operation may still be applied to
>> cluster
>>
>> Yes.
>> If you fail with UnavailableException it's because from the coordinators
>> view of the cluster there is less than CL nodes available. So retry.
>> Somewhat similar story with TimedOutException.
>>
>> 2) Coordinator does not try anything to "roll-back" operation that failed
>> because it was processed by less then consitency level number of nodes.
>>
>> Correct.
>>
>> 3) Hinted handoff works only for successfull operations.
>>
>> HH will be stored if the coordinator proceeds with the request.
>> In 1.X HH is stored on the coordinator if a replica is down when the
>> request starts and if the node does not reply in rpc_timeout.
>>
>> 4) Counters are not reliable because of (1)
>>
>> If you get a TimedOutException when writing a counter you should not
>> re-send the request.
>>
>> 5) Read-repair may help to propagate operation that was failed it's
>> consistency level, but was persisted to some nodes.
>>
>> Yes. It works in the background, by default is only enabled on 10% of
>> requests.
>> Note that RR is not the same as the Consistent Level for read. If you
>> work as a CL > ONE the results from CL nodes are always compared and
>> differences resolved. RR is concerned with the replicas not involved in the
>> CL read.
>>
>> 6) Manual repair is still needed because of (2) and (3)
>>
>> Manual repair is *the* was to achieve consistency of data on disk. HH and
>> RR are optimisations designed to reduce the chance of a Digest Mismatch
>> during a read with CL > ONE.
>> It is also essential for distributing Tombstones before they are purged
>> by compaction.
>>
>> P.S. If some points apply only to some cassandra versions, I will be
>> happy to know this too.
>>
>> Assume everyone for version 1.X
>>
>> Thanks
>>
>>   -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 8/06/2012, at 1:20 AM, Віталій Тимчишин wrote:
>>
>> Hello.
>>
>> I am making some cassandra presentations in Kyiv and would like to check
>> that I am telling people truth :)
>> Could community tell me if next points are true:
>> 1) Failed (from client-side view) operation may still be applied to
>> cluster
>> 2) Coordinator does not try anything to "roll-back" operation that failed
>> because it was processed by less then consitency level number of nodes.
>> 3) Hinted handoff works only for successfull operations.
>> 4) Counters are not reliable because of (1)
>> 5) Read-repair may help to propagate operation that was failed it's
>> consistency level, but was persisted to some nodes.
>> 6) Manual repair is still needed because of (2) and (3)
>>
>> P.S. If some points apply only to some cassandra versions, I will be
>> happy to know this too.
>> --
>> Best regards,
>>  Vitalii Tymchyshyn
>>
>>
>>
>
>
> --
> Best regards,
>  Vitalii Tymchyshyn
>

Forward compatibility for CQL3 composite columns

[Tom Duffield (Mailing Lists)]

Is there documentation on a way to create a pre-CQL3 composite column CF with 
indexing that will be compatible with the built-in composite column 
capabilities in CQL3?  

-- 
Tom Duffield
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

Re: select count(*) returns 10000

[Tamar Fraenkel]

Add "limit N" and it will count more than 1.
Of course it will be slow when you increase N.

*Tamar Fraenkel *
Senior Software Engineer, TOK Media

[image: Inline image 1]

ta...@tok-media.com
Tel:   +972 2 6409736
Mob:  +972 54 8356490
Fax:   +972 2 5612956





On Tue, Jun 12, 2012 at 10:07 PM, Derek Williams  wrote:

> It's a known issue, here is a bit extra info on it:
> http://stackoverflow.com/questions/8795923/wrong-count-with-cassandra-cql
>
>
> On Tue, Jun 12, 2012 at 12:40 PM, Leonid Ilyevsky <
> lilyev...@mooncapital.com> wrote:
>
>> The "select count(*) ..." query returns correct count only if it is <=
>> 1, otherwise it returns exactly 1.
>> This happens in both Java API and cqlsh.
>> Can somebody verify?
>>
>> This email, along with any attachments, is confidential and may be
>> legally privileged or otherwise protected from disclosure. Any unauthorized
>> dissemination, copying or use of the contents of this email is strictly
>> prohibited and may be in violation of law. If you are not the intended
>> recipient, any disclosure, copying, forwarding or distribution of this
>> email is strictly prohibited and this email and any attachments should be
>> deleted immediately.  This email and any attachments do not constitute an
>> offer to sell or a solicitation of an offer to purchase any interest in any
>> investment vehicle sponsored by Moon Capital Management LP ("Moon
>> Capital"). Moon Capital does not provide legal, accounting or tax advice.
>> Any statement regarding legal, accounting or tax matters was not intended
>> or written to be relied upon by any person as advice. Moon Capital does not
>> waive confidentiality or privilege as a result of this email.
>>
>
>
>
> --
> Derek Williams
>
>
<>

portability between enterprise and community version

[Abhijit Chanda]

Hi All,

Is it possible to communicate from a  datastax enterprise edition to
datastax community edition.
Actually i want to set one of my node in linux box and other in windows.
Please suggest.


With Regards,
-- 
Abhijit Chanda
VeHere Interactive Pvt. Ltd.
+91-974395

RE: Hector code not running when replication factor set to 2

[Prakrati Agrawal]

The error stack is as follows:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: me.prettyprint.hector.api.exceptions.HUnavailableException: : May 
not be enough replicas present to handle consistency level.
at 
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:59)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
at 
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
at 
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289)
at 
me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53)
at 
me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49)
at 
me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
at 
me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
at 
me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48)
at 
me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:88)
at 
com.musigma.hectortest.HectorUtilTest.getAllColumns(HectorUtil.java:70)
at com.musigma.hectortest.HectorUtil.main(HectorUtil.java:168)
... 5 more
Caused by: UnavailableException()
at 
org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at 
org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:543)
at 
org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:527)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:273)
... 18 more

The node tool ring from the node I connected to looks like :
Address DC  RackStatus State   Load
Effective-Owership  Token

   85070591730234615865843651857942052864
162.192.100.16  datacenter1 rack1   Up Normal  239.82 MB   100.00%  
   0
162.192.100.9   datacenter1 rack1   Down   Normal  239.81 MB   100.00%  
   85070591730234615865843651857942052864

I did not find any error in the logs generated by Cassandra on the running 
machine.

Please help me

Thanks and Regards

Prakrati

From: aaron morton [mailto:aa...@thelastpickle.com]
Sent: Tuesday, June 12, 2012 2:42 PM
To: user@cassandra.apache.org
Subject: Re: Hector code not running when replication factor set to 2

What was the exact error stack you got ?

What does node tool ring look look from the node you connected to?

Did you notice any errors in the logs on the machine you connected to ?


Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 12/06/2012, at 8:41 PM, Prakrati Agrawal wrote:


I am using the consistency level one and replication factor 2

Thanks
Prakrati
From: aaron morton [mailto:aa...@thelastpickle.com]
Sent: Tuesday, June 12, 2012 2:12 PM
To: user@cassandra.apache.org
Subject: Re: Hector code not running when replication factor set to 2

What consistency level and replication factor were you using ?

UnavailableException is thrown when less the consistency level nodes are UP.

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 11/06/2012, at 10:19 PM, Prakrati Agrawal wrote:



Dear all

I had a 2 node cluster with replication factor set to 1. Then I changed the 
replication factor to 2 and brought down one node so that only 1 node was up 
and running. Then I ran my Hector code on the running node. But it gave me 
Unavailable Exception. I also had a Thrift code which ran successfully. I am 
confused as to why the Hector code did not run. Did I miss something? Please 
help me.

Thanks and Regards
Prakrati



This email me