Re: writes to Cassandra failing occasionally

2010-04-09 Thread Ted Zlatanov
On Thu, 8 Apr 2010 10:56:55 -0500 Jonathan Ellis wrote: JE> is N:C:E possibly ignoring thrift exceptions? I always pass them down to the user. The user is responsible for wrapping with eval(). Ted

Re: writes to Cassandra failing occasionally

2010-04-09 Thread Ted Zlatanov
On Thu, 08 Apr 2010 12:16:34 -0700 Mike Gallamore wrote: MG> Hopefully my fix helps others. I imagine it is something you'll run MG> into regardless of the language/interface you use, for example I'm MG> pretty sure that the C/C++ time function truncates values too. I'd MG> recommend anyone usi

Re: writes to Cassandra failing occasionally

2010-04-09 Thread Ted Zlatanov
On Thu, 08 Apr 2010 11:50:38 -0700 Mike Gallamore wrote: MG> Yes I agree single threaded is probably not the best. I wonder how MG> much of a performance hit it is on a single CPU machine though? I MG> guess I still would be blocking on ram writes but isn't like there is MG> multiple CPUs I nee

Re: writes to Cassandra failing occasionally

2010-04-09 Thread Mike Gallamore
That makes sense. My data is coming in from the internet and is being processed in chunks as it is using Active MQ with the stomp package. I'm getting the log lines in 20-1000 line chunks (depending on the busyness of customer sites) so there definitely is the potential for a lot of parallelism. So

Re: Basic question

2010-04-09 Thread Jonathan Ellis
On Thu, Apr 8, 2010 at 12:09 AM, Palaniappan Thiyagarajan wrote: > I am investigating how we can use Cassandra in our application.  We have > tokens and session information stored in db now and I am thinking of moving > to Cassandra.   Currently it’s write and read intensive and having > performan

Re: Very new user needs some troubleshooting pointers

2010-04-09 Thread Jonathan Ellis
A single-threaded test is meaningless. You need a multithreaded (or multiprocess) benchmark like the one in contrib/py_stress. Picture worth 1000 words: http://spyced.blogspot.com/2010/01/cassandra-05.html On Thu, Apr 8, 2010 at 3:59 PM, Heath Oderman wrote: > Hi All, > I'm brand new to Cassand

Re: Worst case #iops to read a row

2010-04-09 Thread Jonathan Ellis
worst case is 2 or 3, depending on row size: one seek to read the right row index block one seek to read the row header (bloom filter + column index) if it's a big row, one seek to read the column block (block size is configurable, default is 256KB) On Thu, Apr 8, 2010 at 5:21 PM, Scott Shealy w

Re: Very new user needs some troubleshooting pointers

2010-04-09 Thread Heath Oderman
Thanks for the reply Jonathan! I started with multi threaded tests, but when my performance was so much slower than my buddy's I switched to one to try to isolate and identify the differences. I got tunnel vision and kept on with the one thread tests. I'll modify the tests and try again. Thanks

RE: Very new user needs some troubleshooting pointers

2010-04-09 Thread Mark Jones
Sounds like we are some experiencing the same problems. (I'm using 0.6RC1) I have a 3 node cluster with 8GB/machine (dual core CPU). I'm peaking on inserts at about 6000-7000/second running 40 threads. Separate spindles for commitlog and data. My read speed is atrocious, 800/sec sustained

Re: Very new user needs some troubleshooting pointers

2010-04-09 Thread Jonathan Ellis
If you're only seeing 1-2 RPS then you should turn on debug logging to see where the latency is. On Fri, Apr 9, 2010 at 11:14 AM, Mark Jones wrote: > Sounds like we are some experiencing the same problems. (I’m using 0.6RC1) I > have a 3 node cluster with 8GB/machine (dual core CPU).  I’m peaking

Re: RE: Very new user needs some troubleshooting pointers

2010-04-09 Thread Heath Oderman
What's interesting for my case is that I put a timer around the thrift method to insert_batch Every iteration of that call against debian (any hardware, same network or in amazon cloud with windows machine in ec2 as well) takes 400,000 ticks. Super consistent. One thread. My friends setup with c

Re: Worst case #iops to read a row

2010-04-09 Thread Ryan King
On Fri, Apr 9, 2010 at 8:39 AM, Jonathan Ellis wrote: > worst case is 2 or 3, depending on row size: > > one seek to read the right row index block > one seek to read the row header (bloom filter + column index) > if it's a big row, one seek to read the column block (block size is > configurable,

RE: RE: Very new user needs some troubleshooting pointers

2010-04-09 Thread Mark Jones
I'm seeing an average write time of 20-30ms/insert with between the 60-67 million row point. (I think at this point I was actually running 80 threads simultaneously, 2 40 thread clients). From: Heath Oderman [mailto:he...@526valley.com] Sent: Friday, April 09, 2010 11:23 AM To: user@cassandra.ap

Re: RE: Very new user needs some troubleshooting pointers

2010-04-09 Thread Jonathan Ellis
The jit on debian may take longer to warm up by default. Do 100k ops first before benchmarking. Benchmark with multiple threads. And use a known benchmark first like py_stress. On Fri, Apr 9, 2010 at 11:23 AM, Heath Oderman wrote: > What's interesting for my case is that I put a timer around t

Re: Worst case #iops to read a row

2010-04-09 Thread Jonathan Ellis
Right. On Fri, Apr 9, 2010 at 11:23 AM, Ryan King wrote: > On Fri, Apr 9, 2010 at 8:39 AM, Jonathan Ellis wrote: >> worst case is 2 or 3, depending on row size: >> >> one seek to read the right row index block >> one seek to read the row header (bloom filter + column index) >> if it's a big row,

Re: RE: Very new user needs some troubleshooting pointers

2010-04-09 Thread Heath Oderman
Will do, thanks for the advice. :) On Apr 9, 2010 12:28 PM, "Jonathan Ellis" wrote: The jit on debian may take longer to warm up by default. Do 100k ops first before benchmarking. Benchmark with multiple threads. And use a known benchmark first like py_stress. On Fri, Apr 9, 2010 at 11:23 A

Grails Cassandra plugin 0.6 compatible

2010-04-09 Thread Ned Wolpert
Folks- Complete an upgrade to the grails Cassandra plugin I've been working on to make it compatible with Cassandra 0.6. Let me know if anyone is having trouble with it. Thanks -- Virtually, Ned Wolpert "Settle thy studies, Faustus, and begin..." --Marlowe

How to perform queries on Cassandra?

2010-04-09 Thread Onur AKTAS
Hi, I want to use Cassandra for a new project, as you can guess I have a RDBMS background however do not have any experience with NoSQL databases except key/value pair in memory data grids/caches. (Oracle Coherence /Memcached.). I'm trying to find out how do you perform queries with calculations

Questions regarding network topography and automated adminstration

2010-04-09 Thread Todd Nine
Hi all, First off, thanks for putting out such a great product and documentation. I had a node up and running on CentOs in 10 minutes, and had our C# app communicating with it in 10 more! Now that I have basic prototyping working, I have a few networking and data center configurations. We will

Re: How to perform queries on Cassandra?

2010-04-09 Thread Paul Prescod
2010/4/9 Onur AKTAS : > ... > I'm trying to find out how do you perform queries with calculations on the > fly without inserting the data as calculated from the beginning. > Lets say we have latitude and longitude coordinates of all users and we have >  Distance(from_lat, from_long, to_lat, to_long

Re: How to perform queries on Cassandra?

2010-04-09 Thread malsmith
It's sort of an interesting problem - in RDBMS one relatively simple approach would be calculate a rectangle that is X km by Y km with User 1's location at the center. So the rectangle is UserX - 10KmX , UserY-10KmY to UserX+10KmX , UserY+10KmY Then you could query the database for all other us

Re: How to perform queries on Cassandra?

2010-04-09 Thread Mike Gallamore
I apologize in advance if this goes into esoteric algorithms a bit too much but I think this will get to an interesting idea to solve your problem. My background is physics particularly computer simulations of complex systems. Anyways in cosmology an interesting algorithm is called an n-body tr

Re: How to perform queries on Cassandra?

2010-04-09 Thread Malcolm Smith
Mike are you stuck on a train too? :-) On Apr 9, 2010, at 8:51 PM, Mike Gallamore > wrote: I apologize in advance if this goes into esoteric algorithms a bit too much but I think this will get to an interesting idea to solve your problem. My background is physics particularly computer s

Re: How to perform queries on Cassandra?

2010-04-09 Thread dir dir
Does Cassandra has a default query language such as SQL in RDBMS and Object Query in OODBMS? Thank you. Dir. On Sat, Apr 10, 2010 at 7:01 AM, malsmith wrote: > > > It's sort of an interesting problem - in RDBMS one relatively simple > approach would be calculate a rectangle that is X km by Y km

Re: How to perform queries on Cassandra?

2010-04-09 Thread Paul Prescod
No. Cassandra has an API. http://wiki.apache.org/cassandra/API On Fri, Apr 9, 2010 at 8:00 PM, dir dir wrote: > Does Cassandra has a default query language such as SQL in RDBMS > and Object Query in OODBMS?  Thank you. > > Dir. > > On Sat, Apr 10, 2010 at 7:01 AM, malsmith > wrote: >> >> >> It'

How many KeySpace will you use in a single application?

2010-04-09 Thread Dop Sun
Hi, a question troubles me now: how many KeySpaces one application is better to use? The question is coming out since 0.6, Cassandra introduced a new API named as "login", which is done against a specific keySpace. Thanks to the org.apache.cassandra.auth.AllowAllAuthenticator, the old version c