Re: Migrating from mySQL to Cassandra

Tyler, A quick question for you - I am completely new to noSQL databases and the architecture of Cassandra. So, I'm still way down the learning curve. I downloaded and installed the OpsCenter last night, but I first need to configure it. I want to install 2 nodes on my laptop and then start bui

Re: Migrating from mySQL to Cassandra

Since your just starting out and it seems like you want to get past the admin hurdles and start coding, may I suggest you use https://github.com/pcmanus/ccm to get your local test environment setup. Setting up a production Cassandra cluster takes a fair amount of planning (before) and tuning (duri

Re: Migrating from mySQL to Cassandra

Thanks Steven - I will take a look. On 4 Mar 2013, at 11:41, Steven Siebert wrote: > Since your just starting out and it seems like you want to get past the admin > hurdles and start coding, may I suggest you use > https://github.com/pcmanus/ccm to get your local test environment setup. > > S

Re: no backwards compatibility for thrift in 1.2.2? (we get utter failure)

For us, this was an issue creating tables in 1.1.4 using thrift, then upgrading to 1.2.2. We did not use cli to create anything. I will try the complete test again today and hopefully get more detail(I didn't know I could not run the same thrift code in 1.2.2 for keyspace creation/table creati

cassandra 1.2.2 with cassandra thrift 1.1.1? or 1.1.4 with 1.2.2 thrift?

I am trying to figure out how to swing a rolling upgrade. My current version of astyanax 1.56.18 with cassandra thrift 1.1.1 against an upgraded QA(from 1.1.4 to 1.2.2) fails with the below exception...(should I be researching why?, or should I be figuring out why 1.2.2 thrift doesn't work wit

grrr. UPGRADE DOES WORK now!!!

It turns out my rpc listen address was localhost. The exception threw me off as I really should be getting connection refused stuff in there like I do when I telnet in. I will try to do an astyanax pull request on a fix for that later. Dean On 3/4/13 6:40 AM, "Hiller, Dean" wrote: >I am tryin

Re: reading the updated values

thanks for ur reply. On Mon, Mar 4, 2013 at 11:56 AM, aaron morton wrote: > my question is how do i get the updated data in cassandra for last 1 hour > or so to be indexed in elasticsearch. > > You cannot. > > The best approach is to update elastic search at the same time you update > cassandra.

Re: Consistent problem when solve Digest mismatch

The problem is, what is the sequence number you are talking about is exactly? Or let me put it another way: if you do have a sequence number that provides a total ordering of your operation, then that is exactly what you should use as your timestamp. What Cassandra calls the timestamp, is exactly

CqlPreparedResult type info

Hi, I'm using prepare_cql3_query thrift method, which returns CqlPreparedResult with type info in variable_types. The type names returned are partial for collection types. Returns just MapType without saying anything about the key-value types. A more intuitive response could be something like

Re: Column Slice Query performance after deletions

Guys, thank you very much. For my case scenario, I'm gonna need to change a little bit my data model by spliting my row nto N pieces. And implement a further control of it. That will mitigate the problem. Also, I'll try LeveledCompaction after. Thanks! On Mon, Mar 4, 2013 at 3:25 AM, aaron mort

Re: CqlPreparedResult type info

On Mon, Mar 4, 2013 at 4:13 PM, Adriano Paggi wrote: > Hi, I’m using prepare_cql3_query thrift method, which returns > CqlPreparedResult with type info in variable_types. > > The type names returned are partial for collection types. Returns just > MapType without saying anything about the

Re: no backwards compatibility for thrift in 1.2.2? (we get utter failure)

ok, we are talking about all thrift / cli / hector / no CQL tables not been read after an upgrade. If you can get some repo steps that would be handy. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 4/03/2013, at 5

Re: Consistency level for system_auth keyspace

> In this case, it means that if there is a network split between the 2 > datacenters, it is impossible to get the quorum, and all connections will be > rejected. Yes. > Is there a reason why Cassandra uses the Quorum consistency level ? I would guess to ensure there is a single, cluster wide,

Re: Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider

What version are you using ? As of 1.1 off heap caches no longer require JNA https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L327 Also the row and key caches are now set globally not per CF https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L324 Cheers - Aaron Mo

Re: backing up and restoring from only 1 replica?

That would be OK only if you never had node go down (e.g. a restart) or drop messages. It's not something I would consider trying. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 28/02/2013, at 3:21 PM, Mike Koh wr

Re: Retrieving local data

Yes. You can get the token ranges via astynax and only ask for rows that are within the token ranges. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 28/02/2013, at 2:25 PM, Everton Lima wrote: > Ok aaron. But the

Re: Connection issues

Did you get this sorted ? Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 28/02/2013, at 2:25 AM, amulya rattan wrote: > I have set up cassandra on EC2 with default localhost settings. Everything > works fine and d

Re: Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider

version 1.0.8 Just curious, what is the mechanism for off heap in 1.1? Thank you. /Jason On Mon, Mar 4, 2013 at 11:49 PM, aaron morton wrote: > What version are you using ? > > As of 1.1 off heap caches no longer require JNA > https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L327 > > A

RE: CqlPreparedResult type info

Commented on CASSANDRA-3753, maybe re-opening is better than create a new one. I don't know the correct procedure... From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Lunes, 04 de Marzo de 2013 12:23 p.m. To: user@cassandra.apache.org Subject: Re: CqlPre

Re: Connection issues

Yes, thanks. I didn't have the intelligence to use internal IPs as listen/rpc address earlier. 2013/3/4 aaron morton > Did you get this sorted ? > > Cheers > >- > Aaron Morton > Freelance Cassandra Developer > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > O

Re: Consistency level for system_auth keyspace

Hi Aaron, I have open a ticket in Jira : https://issues.apache.org/jira/browse/CASSANDRA-5310 Reading the user using the QUORUM consistency level means that in case of network outage, you are unable to open a connection, and all your data become unavailable. Regards. Jean Armel 2013/3/4 aaron m

Re: Migrating from mySQL to Cassandra

On Mon, Mar 4, 2013 at 5:08 AM, John Grogan wrote: > > A quick question for you - I am completely new to noSQL databases and the > architecture of Cassandra. So, I'm still way down the learning curve. > > I downloaded and installed the OpsCenter last night, but I first need to > configure it. I w

Re: Poor read latency

What is nodetool info say for your memory? (we hit that one with memory near the max and it slowed down our system big time…still working on resolving it too). Do any logs have the hit 0.75, running compaction OR worse hit 0.85 running compaction….you get that if the above is the case typicall

Re: Migrating from mySQL to Cassandra

Definitely read up on patterns out there. The more info you gather, the easier it goes in designs. http://buffalosw.com/wiki/Patterns-Page/ From: Tyler Hobbs mailto:ty...@datastax.com>> Reply-To: "user@cassandra.apache.org" mailto:user@cassandra.apache.org>> D

Re: Consistency level for system_auth keyspace

I thought there was already a LOCAL_QUOROM option so things continue to work when you get data center split. There was also TWO I think as well which allowed 4 nodes(2 in each data center) so you can continue to write when data center splits. Dean From: Jean-Armel Luce mailto:jaluc...@gmail.c

Re: Poor read latency

Yeah, I just checked and the heap size 0.75 warning has been appearing. nodetool info reports: Heap Memory (MB) : 563.88 / 1014.00 Heap Memory (MB) : 646.01 / 1014.00 Heap Memory (MB) : 639.71 / 1014.00 We have plenty of free memory on each instance. Do we need bigger instances or should we jus

Re: Consistency level for system_auth keyspace

Hi Dean, The new authentication modules currently uses a QUORUM consistency level when checking the user. That is the reason why it doesn't work in version 1.2.2. I thing that using LOCAL_QUORUM or ONE CL instead of QUORUM should solve this problem. But I didn't see any option in 1.2.2. Regards.

Re: backing up and restoring from only 1 replica?

Thanks for the response. Could you elaborate more on the bad things that happen during a restart or message drops that would cause a 1 replica restore to fail? I'm completely on board with not using a restore process that nobody else uses, but I need to convince somebody else who thinks that

Re: Poor read latency

Recommended settings are 8G RAM and your memory grows with the number of rows through index samples(configured in cassandra.yaml as samples per row something…look for the word index). Also, bloomfilters grow with RAM if using size tiered compaction. We are actually trying to switch to leveled

Re: Consistency level for system_auth keyspace

Ouch, a detail I hadn't learned about yet. Good to know. Dean From: Jean-Armel Luce mailto:jaluc...@gmail.com>> Reply-To: "user@cassandra.apache.org" mailto:user@cassandra.apache.org>> Date: Monday, March 4, 2013 11:34 AM To: "user@cassandra.apache.org

what size file for LCS is best for 300-500G per node?

Should we really be going with 5MB when it compresses to 3MB? That seems to be on the small side, right? We have ulimit cranked up so many files "shouldn't" be an issue but maybe we should go to 10MB or 100MB or something in between? Does anyone have any experience with changing the LCS sizes

Storage question

Hi - Can someone suggest the optimal way to store files / images ? We are planning to use cassandra for meta-data for these files. HDFS is not good for small file size .. can we look at something else ? Thanks, Kanwar

Re: Storage question

The best way would be to chunk your binary blobs into 1/2MB chunks. You could store Key (md5 of entire blob) => part1, part2, part3 etc BytesType Validation Then if you want the entire image just grab the key (md5)..obviously you'll need a index somewhere with a filename => md5 Best, Michael

Re: Storage question

Well, astyanax I know can simulate streaming into cassandra and disperses the file to multiple rows in the cluster so you could check that out. Out of curiosity, why is HDFS not good for a small file size? For reading, it should be the bomb with RF=3 since you can read from multiple nodes and s

Re: Storage question

Each storage system has its own purpose. While Cassandra would be good for metadata, depending on the size of objects Cassandra could be not the best fit. You need something more like Amazon S3 for blob storage. Try Ceph RADOS or OpenStack Object Store which both provide S3 compatible API. On

RE: Storage question

Problems with small files and HDFS A small file is one which is significantly smaller than the HDFS block size (default 64MB). If you're storing small files, then you probably have lots of them (otherwise you wouldn't turn to Hadoop), and the problem is that HDFS can't handle lots of files. Ev

Replication Question

Hi - If I configure a RF across 2 Data centres as below and assuming 3 nodes per Data centre. DC1: 2, DC2:2 I do a write with consistency level - local_quorum which ensures that there is no inter DC latency. Now say 2 nodes in DC1 crash and I am doing a read with CL = One. Will it return fail

RE: Replication Question

Keep in mind that even at consistency level ONE or LOCAL_QUORUM, the write is still sent to all replicas for the written key, even replicas in other data centers. The consistency level just determines how many replicas are required to respond that they received the write. Is this true for Reads

Re: Storage question

Thanks for the great explanation. Dean On 3/4/13 1:44 PM, "Kanwar Sangha" wrote: >Problems with small files and HDFS > >A small file is one which is significantly smaller than the HDFS block >size (default 64MB). If you're storing small files, then you probably >have lots of them (otherwise you

anyone see this user-cassandra thread get answered...

I was reading http://mail-archives.apache.org/mod_mbox/cassandra-user/201208.mbox/%3CCAGZm5drRh3VXNpHefR9UjH8H=dhad2y18s0xmam5cs4yfl5...@mail.gmail.com%3E As we are having the same issue in 1.2.2. We modify to LCS and cassandra-cli shows us at LCS on any node we run cassandra cli on, but then loo

Re: old data / tombstones are not deleted after ttl

Those older files won't be included in a compaction until there are min_compaction_threshold (4) files of that size. When you get another SS table -Data.db file that is about 12-18GB then you'll have 4 and they will be compacted together into one new file. At that time, if there are any rows with

Re: Consistent problem when solve Digest mismatch

Hi The timestamp provided by my client is unix timestamp (with ntp), and as I said, due to the ntp drift, the local unix timestamp is not accurately synchronized (compare to my case). So for short, client can not provide global sequence number to indicate the event order. But I wonder, I configu

Re: what size file for LCS is best for 300-500G per node?

We have 200G and ended going with 10M. The compaction after repair takes a day to finish. Try to run a repair and see how it goes. -Wei - Original Message - From: "Dean Hiller" To: user@cassandra.apache.org Sent: Monday, March 4, 2013 10:52:27 AM Subject: what size file for LCS is best

hinted handoff disabling trade-offs

I am looking to get a second opinion about disabling hinted-handoffs. I have an application that can tolerate a fair amount of inconsistency (advertising domain), and so I'm weighting the pros and cons of hinted handoffs. I'm running Cassandra 1.0, looking to upgrade to 1.1 soon. Pros of disabling

Re: hinted handoff disabling trade-offs

Repair is slow. On Mar 4, 2013, at 8:07 PM, "Matt Kap" wrote: > I am looking to get a second opinion about disabling hinted-handoffs. I > have an application that can tolerate a fair amount of inconsistency > (advertising domain), and so I'm weighting the pros and cons of hinted > handoffs. I'm

Re: hinted handoff disabling trade-offs

Why do you think disabling hinted handoff will improve memory usage?  Thanks. -Wei Sent from my Samsung smartphone on AT&T Original message Subject: Re: hinted handoff disabling trade-offs From: Michael Kjellman To: "user@cassandra.apache.org" CC: Repair is slow. On M

Re: hinted handoff disabling trade-offs

Also, if you have enough hints being created that its significantly impacting your heap I have a feeling things are going to get out of sync very quickly. On Mar 4, 2013, at 9:17 PM, "Wz1975" mailto:wz1...@yahoo.com>> wrote: Why do you think disabling hinted handoff will improve memory usage?

Cassandra instead of memcached

Hi Guys, I'm thinking about using Cassandra as an in-memory key/value store instead of memcached for a new project (just to get rid of a dependency if possible). I was thinking about setting the replication factor to 1, enabling off-heap row-cache and setting gc_grace_period to zero for the CF

RE: old data / tombstones are not deleted after ttl

Hi, I have done a manually compaction over the nodetool and this worked. But thx for the explanation, why it wasn´t compacted Br, Matthias Zeilinger Production Operation – Shared Services P: +43 (0) 50 858-31185 M: +43 (0) 664 85-34459 E: matthias.zeilin...@bwinparty.com bwin.party services (A

Re: old data / tombstones are not deleted after ttl

Was it a major compaction? I ask because it's definitely a solution that had to work, but it's also a solution that - in general - probably no-one here would suggest you to use. M. W dniu 05.03.2013 07:08, Matthias Zeilinger pisze: Hi, I have done a manually compaction over the nodetool and

RE: old data / tombstones are not deleted after ttl

Yes it was a major compaction. I know it´s not a great solution, but I needed something to get rid of the old data, because I went out of diskspace. Br, Matthias Zeilinger Production Operation – Shared Services P: +43 (0) 50 858-31185 M: +43 (0) 664 85-34459 E: matthias.zeilin...@bwinparty.com