Re: Delayed events processing / queue (anti-)pattern

2015-03-26 Thread Robin Verlangen
Interesting thought, that should work indeed, I'll evaluate both options and provide an update here once I have results. Best regards, Robin Verlangen *Chief Data Architect* W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> *What is CloudPelican? <http:

Re: Delayed events processing / queue (anti-)pattern

2015-03-25 Thread Robin Verlangen
d from a C* point of view this should work as it's write heavy. I add hundreds of thousands of columns to a key, and then read them all at once (or maybe a few times with pagination), and then remove the entire row by it's primary key. Any other thoughts on this? Best regards, Robin Verla

Delayed events processing / queue (anti-)pattern

2015-03-24 Thread Robin Verlangen
g Cassandra for this? Or should we use a different system, like Redis? Thanks. Best regards, Robin Verlangen *Chief Data Architect* W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> *What is CloudPelican? <http://goo.gl/HkB3D>* Disclaimer: The information co

Client-side compression, cassandra or both?

2014-11-03 Thread Robin Verlangen
in a row / SSTable. Thank you in advance for your help. Best regards, Robin Verlangen *Chief Data Architect* W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> *What is CloudPelican? <http://goo.gl/HkB3D>* Disclaimer: The information contained in this message and att

Re: counters + replication = awful performance?

2012-11-28 Thread Robin Verlangen
Not sure whether it's an option for you, but you might consider to do some in-memory aggregation of counter values and flushing only once every X updates / seconds. This will decrease both load, latency and throughput. However this is not possible in every single use case. Best regards,

Re: Remove crashed node

2012-11-19 Thread Robin Verlangen
Quite some time passed, but Alain, it worked. Thank you very much. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely f

Re: Remove crashed node

2012-11-09 Thread Robin Verlangen
Hi Alain, How can I access that? Web browser does not seem to work. Do I need any software to login? If so, what is proper software for Windows? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaime

Remove crashed node

2012-11-09 Thread Robin Verlangen
Hi there, We have had a crashed node that is currently removed from the rack. However when I try a schema upgrade / truncate operation it complains of the unreachable node. I tried the removetoken, but that didn't resolve. Any ideas on how to fix this? Best regards, Robin Verlangen *Sof

Simba Cassandra ODBC

2012-10-05 Thread Robin Verlangen
n top of C* is required. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be

Re: 1000's of column families

2012-09-28 Thread Robin Verlangen
I think I misunderstood your "all data in one location" note. I thought you meant to store it all in one CF. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in

Re: 1000's of column families

2012-09-27 Thread Robin Verlangen
more nodes and ever node contains a smaller piece of the cake (~ring). Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely for the att

Re: 1000's of column families

2012-09-27 Thread Robin Verlangen
Every CF adds some overhead (in memory) to each node. This is something you should really keep in mind. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this messa

Re: Integrated cassandra

2012-09-26 Thread Robin Verlangen
what to do yet ... Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may

Re: Integrated cassandra

2012-09-26 Thread Robin Verlangen
Do you have any ideas how to do this Vivek? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the

Re: Integrated cassandra

2012-09-26 Thread Robin Verlangen
Some additional information: I already read about "Embedding" http://wiki.apache.org/cassandra/Embedding however that doesn't seem a rock solid solution to me. The word "volatile" is not really comforting me ;-) Best regards, Robin Verlangen *Soft

Integrated cassandra

2012-09-25 Thread Robin Verlangen
ossible without messing around in the Cassandra core. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use o

Re: Cassandra Counters

2012-09-25 Thread Robin Verlangen
er I'm not sure whether that's the best option. I think you should stick with Cassandra counter column families. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl <http://goo.gl/Lt7BC> Disclaimer: The information contained in t

Re: are counters stable enough for production?

2012-09-18 Thread Robin Verlangen
u really bother you would prefer an overcount over an undercount. If you're counting profit you would not want to measure more than you actually made. However in all cases you'll want the rights results: spot on without loss of precision. Best regards, Robin Verlangen *Software engineer* * *

Re: are counters stable enough for production?

2012-09-18 Thread Robin Verlangen
e highest of the two inconsistent values." I was aware of that. The repair of taking the highest value of two inconsistent might cause getting higher values? Maybe even much higher values if it takes place multiple times? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinve

Re: are counters stable enough for production?

2012-09-18 Thread Robin Verlangen
about Cassandra internals can shed some light on this. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may

Re: are counters stable enough for production?

2012-09-18 Thread Robin Verlangen
@Rohit: We also use counters quite a lot (lets say 2000 increments / sec), but don't see the 50-100KB of garbage per increment. Are you sure that memory is coming from your counters? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Discl

Re: Disk configuration in new cluster node

2012-09-17 Thread Robin Verlangen
sues with JBOD, RAID or whatever. If you want to improve throughput you might consider a RAID-0 setup. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for t

Re: Differences in row iteration behavior

2012-09-15 Thread Robin Verlangen
omehow using an other consistency level? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confi

Re: Long-life TTL and extending TTL

2012-09-15 Thread Robin Verlangen
Hi Sergey, That's exactly what I mean. I really hope that this will get released soon! Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the atte

Re: Cassandra node going down

2012-09-14 Thread Robin Verlangen
sure your data location is on the right disk? Default is /var/lib/cassandra/data Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and u

Re: Cassandra node going down

2012-09-14 Thread Robin Verlangen
Robbit = Rohit of course, excuse me. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be

Re: Cassandra node going down

2012-09-14 Thread Robin Verlangen
Hi Robbit, I think it's running out of disk space, please verify that (on Linux: df -h ). Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely fo

Re: Reading column names only

2012-09-14 Thread Robin Verlangen
Hi Aaron, Is this something that's worth becoming a feature in the future? Or should I rework my data model? If so, do you have any suggestions? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in

Reading column names only

2012-09-14 Thread Robin Verlangen
Hi there, Would it be possible to read only the column names, instead of the names and values? I would like to store some in the value, but without the cost of slowing down the reads of the column names (primary task). Best regards, Robin Verlangen *Software engineer* * * W http

Re: Long-life TTL and extending TTL

2012-09-10 Thread Robin Verlangen
iterate through them and delete > the items. > > > Regards, > > Oleg > > > > On 2012-09-10 09:47:31 +, Robin Verlangen said: > > > Hi there, > > > I'm working on a project that might want to set TTL to roughly 7 years. > However it might

Long-life TTL and extending TTL

2012-09-10 Thread Robin Verlangen
? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended rec

Re: Node-tool drain on Cassandra 1.0

2012-09-09 Thread Robin Verlangen
Deleting the commitlog files is harmless. It's just a tool that tries to keep Cassandra more in-sync with the other nodes. A standard repair will fix all problems that a commitlog replay might do too. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl

Re: Helenos - web based gui tool

2012-09-07 Thread Robin Verlangen
It's also some kind of OS (link: http://www.helenos.org/ ), not a really unique name. However the software looks nice! Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachmen

Re: why do I have to use internal IP for EC2 nodes?

2012-09-04 Thread Robin Verlangen
@Yang: Sounds legit, as internal is not the same as external. Beware of the fact that internal traffic is only free when it's in the same availability zone. In the same region is charged with a small amount (~ $0.01). With kind regards, Robin Verlangen *Software engineer* * * W

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-28 Thread Robin Verlangen
. Then manually delete columns from that queue-row. With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-28 Thread Robin Verlangen
"What this means is that eventually you will have 1 row in the secondary index table with 350K columns" Is this really true? I would have expected that Cassandra used internal index sharding/bucketing? With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverl

Re: Cassandra API Library.

2012-08-23 Thread Robin Verlangen
@Brian: You're missing PhpCassa (PHP library) With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addresse

Re: Opscenter 2.1 vs 1.3

2012-08-19 Thread Robin Verlangen
Hi Nick, I'm talking about the total writes/reads in the dashboard (left graph). It exactly tripled during our update. I guess this is a change because of the fact we also replicate with RF=3. Is that true? With kind regards, Robin Verlangen *Software engineer* * * W

Opscenter 2.1 vs 1.3

2012-08-16 Thread Robin Verlangen
Hi there, I just upgraded to opscenter 2.1 (from 1.3). It appears that my writes have tripled. Is this a change in the display/measuring of opscenter? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in

Re: Cassandra OOM crash while mapping commitlog

2012-08-14 Thread Robin Verlangen
Everything still runs smooth. It's really plausible that the 1.1.3 version resolved this bug. 2012/8/13 Robin Verlangen > 3 hours ago I finished the upgraded of our cluster. Currently it runs > quite smooth. I'll give an update within a week if this really solved our >

Re: Cassandra OOM crash while mapping commitlog

2012-08-13 Thread Robin Verlangen
3 hours ago I finished the upgraded of our cluster. Currently it runs quite smooth. I'll give an update within a week if this really solved our issues. Cheers! 2012/8/13 Robin Verlangen > @Tyler: We were already running most of our machines in 64bit JVM (Sun, > not the OpenJDK).

Re: Cassandra OOM crash while mapping commitlog

2012-08-12 Thread Robin Verlangen
smaller. > Of course the question remains why an append-only commitlog needs to be > mmap'ed in the first place, especially for writing.. > > -h > > > -- With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: T

Re: Cassandra OOM crash while mapping commitlog

2012-08-12 Thread Robin Verlangen
>> MessagingService.java (line 534) Waiting for messaging service to quiesce >> * >> * INFO [ACCEPT-/85.17.231.89] 2012-08-11 18:05:46,066 >> MessagingService.java (line 690) MessagingService shutting down server >> thread.* >> *ERROR [Thrift:1532] 2012-08-11 18:05:

Cassandra OOM crash while mapping commitlog

2012-08-11 Thread Robin Verlangen
*at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3164) * *at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)* *at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) * *at org.apache.cassandra.thrift.Custo

Re: Cassandra and Tableau

2012-07-13 Thread Robin Verlangen
ate the data to Oracle. > https://github.com/hmsonline/cassandra-triggers/ > > Let us know if you get things setup with a direct connection. > We'd be *very* interested int helping out if you find a way to do it. > > -brian > > > On Fri, Jul 6, 2012 at 5:31 AM, Robi

Re: Multiple keyspace question

2012-07-06 Thread Robin Verlangen
Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin

Cassandra and Tableau

2012-07-06 Thread Robin Verlangen
Hi there, Is there anyone out there who's using Tableau in combination with a Cassandra cluster? There seems to be no standard solution to connect, at least I couldn't find one. Does anyone know how to tackle this problem? With kind regards, Robin Verlangen *Software engineer*

Re: Node crashing during read repair

2012-07-03 Thread Robin Verlangen
> Is this still an issue ? > > It looks like something shut down the messaging service. Was there > anything else in the logs ? > > Cheers > > - > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 28/

Re: Expanding Cassandra on EC2 with consistency

2012-07-03 Thread Robin Verlangen
zones in case of failure, but establish a high > level of read or write consistency within a single availability zone. > > I appreciate your responses, > Thanks, > -Mike > > > > -- With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E r

Re: Node won't start (faillout from AWS outage) - HELP

2012-07-02 Thread Robin Verlangen
r machines with their cpu stuck at 100%. so i went to restart each > cassandra node one by one. i did node with token id 0 first. i came back > but doesn't look like it doing anything. once i thought it was up i went > and restarted the next. this one got stuck on the AMI init star

Re: Adding New Nodes to a Production Cluster

2012-06-27 Thread Robin Verlangen
> > Please help me to find an answer to this. > > Thanks, > Eran Chinthaka Withana > -- With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended

Re: Ball is rolling on High Performance Cassandra Cookbook second edition

2012-06-27 Thread Robin Verlangen
everal >> people added content to the first edition and it would be great to see >> that type of participation again. >> >> Thank you, >> Edward >> > > -- With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro..

Node crashing during read repair

2012-06-27 Thread Robin Verlangen
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Is this a common bug in 1.1.1, or did I get a "race" condition? -- With kind regards, Robin Verlangen