Re: Memcached protocol?

2010-04-05 Thread David Strauss
On 2010-04-05 03:42, Paul Prescod wrote: > On Sun, Apr 4, 2010 at 5:06 PM, Benjamin Black wrote: >> ... >> >> Are you suggesting this would give you counter semantics? > > Yes: My understanding of cassandra-580 is that it gives you increment > and decrement which are the basis of counters. There

Flush Commit Log

2010-04-05 Thread JKnight JKnight
Dear all, How can I flush all Commit Log for Cassandra version 042? I use nodeprobe flush but It seem does not run. Thank a lot for support. -- Best regards, JKnight

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 12:01 AM, David Strauss wrote: > On 2010-04-05 03:42, Paul Prescod wrote: >... > > There is a difference between Cassandra allowing inc/dec on values and > actually *knowing* the resultant value at the time of the write. It's > likely that inc/dec support will still feature

Re: Memcached protocol?

2010-04-05 Thread David Strauss
On 2010-04-05 07:47, Paul Prescod wrote: > On Mon, Apr 5, 2010 at 12:01 AM, David Strauss wrote: >> On 2010-04-05 03:42, Paul Prescod wrote: >> ... >> >> There is a difference between Cassandra allowing inc/dec on values and >> actually *knowing* the resultant value at the time of the write. It's

Re: Cassandra Design or another solution

2010-04-05 Thread JKnight JKnight
Thanks for for reply, David. I will tell more the detail about the system. My system is used to store the score (point) user earn when they play game. "Mark" is the score. User's score changes when user win game, buy or sell anything. Sorry I make a mistake. My data model is: Mark{ //Column Fam

Re: Cassandra Design or another solution

2010-04-05 Thread David Strauss
I need the question about monotonicity answered, too. You should also know: Cassandra is not ideal for directly tracking values you increment or decrement. On 2010-04-05 08:04, JKnight JKnight wrote: > Thanks for for reply, David. > > I will tell more the detail about the system. My system is us

Re: Cassandra Design or another solution

2010-04-05 Thread JKnight JKnight
Thanks David, But what's does "monotonicity" mean? User's score belongs to their action. When they win the game or sale something, user's score will increase. When user lose the game or buy something, user's score will decrease. On Mon, Apr 5, 2010 at 4:09 AM, David Strauss wrote: > I need the

Re: Cassandra Design or another solution

2010-04-05 Thread Andriy Bohdan
Hello guys I have a pretty similar task. There's a need to store tags of products with score. Score may go up and down and tags have to be ordered by their score for each product. Score is updated "very" often. I was thinking of using the following model (simplified here for clarity): Product =

Re: Cassandra Design or another solution

2010-04-05 Thread David Timothy Strauss
If user scores move in more than one direction, as they apparently do in your case, they are not monotonic. Monotonicity can make system design a bit easier for various reasons. - "JKnight JKnight" wrote: Thanks David, But what's does "monotonicity" mean? User's score belongs to thei

Re: Cassandra Design or another solution

2010-04-05 Thread David Timothy Strauss
In any case, the common approach to this in Cassandra is to not directly manipulate the user's total score but to insert columns representing changes to the score, later totaling them (and possibly inserting them elsewhere so you get the automatic sort). There are many fancy ways to approach th

Re: Cassandra Design or another solution

2010-04-05 Thread David Timothy Strauss
Cache the => map as you write values (a "write-through" cache) so that reading the current score hits something like memcached instead of Cassandra. With a cache hit, you get an ideal, write-only path in Cassandra. Three blind writes in Cassandra is cheap -- no matter what your scale. The only

Re: Cassandra Design or another solution

2010-04-05 Thread Andriy Bohdan
It makes sense. Thanks, David! On Mon, Apr 5, 2010 at 2:34 PM, David Timothy Strauss wrote: > Cache the => map as you write values (a > "write-through" cache) so that reading the current score hits something like > memcached instead of Cassandra. With a cache hit, you get an ideal, > write-

Re: Memcached protocol?

2010-04-05 Thread Ryan Daum
It seems pretty clear to me that the full memcached protocol is not appropriate for Cassandra. The question is whether some subset of it is of any use to anybody. The only advantage I can see is that there are a large number of clients out there that can speak it already; but any app that is making

Re: multinode cluster wiki page

2010-04-05 Thread Ted Zlatanov
On Sat, 3 Apr 2010 13:52:22 -0700 Benjamin Black wrote: BB> What happens if the IP I get back is for a seed that happens to be BB> down right then? And then that IP is cached locally by my resolver? You have to set the TTL to be the right number of seconds for your environment. With tinydns o

Re: multinode cluster wiki page

2010-04-05 Thread Ted Zlatanov
On Sat, 3 Apr 2010 14:10:37 -0500 Jonathan Ellis wrote: JE> IMO the "right" way to do it is to configure your machines so that JE> autodetecting listenaddress Just Works, so you can deploy exactly the JE> same config to all nodes. It would be nice if Cassandra looked at all the available interf

Re: Cassandra Design or another solution

2010-04-05 Thread JKnight JKnight
Thanks for your help, David. On Mon, Apr 5, 2010 at 7:25 PM, Andriy Bohdan wrote: > It makes sense. > > Thanks, David! > > On Mon, Apr 5, 2010 at 2:34 PM, David Timothy Strauss > wrote: > > Cache the => map as you write values (a > "write-through" cache) so that reading the current score hi

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 1:02 AM, David Strauss wrote: > ... > > But your "write then read" model lacks the atomicity of the memcached > API. It's possible for two clients to read the same value. Do you have an example application where this particular side effect of eventual consistency is problem

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 5:29 AM, Ryan Daum wrote: > It seems pretty clear to me that the full memcached protocol is not > appropriate for Cassandra. The question is whether some subset of it is of > any use to anybody. The only advantage I can see is that there are a large > number of clients out t

Re: Memcached protocol?

2010-04-05 Thread Ryan Daum
Are these applications using memcached for caching or for something else? I don't see the point in putting Cassandra in as a level 1 or 2 cache replacement? Especially given as it does not support any reasonable expiration policy that would be of use in those circumstances. Ryan On Mon, Apr 5, 2

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 10:19 AM, Ryan Daum wrote: > Are these applications using memcached for caching or for something else? > I don't see the point in putting Cassandra in as a level 1 or 2 cache > replacement? Especially given as it does not support any reasonable > expiration policy that would

Re: Memcached protocol?

2010-04-05 Thread Mike Malone
> > Here are a couple of example projects for info. > > Django: > > http://docs.djangoproject.com/en/dev/topics/cache/ > > It says of "increment/decrement": "incr()/decr() methods are not > guaranteed to be atomic. On those backends that support atomic > increment/decrement (most notably, the memca

Re: multinode cluster wiki page

2010-04-05 Thread Brandon Williams
2010/4/5 Ted Zlatanov > On Sat, 3 Apr 2010 14:10:37 -0500 Jonathan Ellis > wrote: > > JE> IMO the "right" way to do it is to configure your machines so that > JE> autodetecting listenaddress Just Works, so you can deploy exactly the > JE> same config to all nodes. > > It would be nice if Cassand

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 10:45 AM, Mike Malone wrote: > ... > > FWIW, I added the atomic increment/decrement operations to the Django cache > interface (and wrote that documentation) because the functionality was > useful for large scale apps. I didn't implement atomic increment/decrement > or atomi

Re: multinode cluster wiki page

2010-04-05 Thread Ted Zlatanov
On Mon, 5 Apr 2010 13:10:38 -0500 Brandon Williams wrote: BW> 2010/4/5 Ted Zlatanov >> It would be nice if Cassandra looked at all the available interfaces and >> selected the one whose reverse DNS lookup returned ".*cassandra.*" (or >> some keyword the user provided). >> >> In other words, wh

Re: Memcached protocol?

2010-04-05 Thread Mike Malone
> > That's useful information Mike. I am a bit curious about what the most > common use cases are for atomic increment/decrement. I'm familiar with > atomic add as a sort of locking mechanism. > They're useful for caching denormalized counts of things. Especially things that change rapidly. Instea

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 1:35 PM, Mike Malone wrote: >> That's useful information Mike. I am a bit curious about what the most >> common use cases are for atomic increment/decrement. I'm familiar with >> atomic add as a sort of locking mechanism. > > They're useful for caching denormalized counts of

Re: Question about node failure...

2010-04-05 Thread Jonathan Ellis
On Mon, Mar 29, 2010 at 6:42 PM, Tatu Saloranta wrote: > Perhaps it would be good to have convenience workflow for replacing > broken host ("squashing lemons")? I would assume that most common use > case is to effectively replace host that can't be repaired (or perhaps > it might sometimes be best

Re: Slow Responses from 2 of 3 nodes in RC1

2010-04-05 Thread Jonathan Ellis
When you're saying you can check 50 or 100 per second, how many rows and columns does a check involve? What query api are you using? Your cassandra nodes look mostly idle. Is each client thread getting the same amount of work or are some finishing sooner than others? Is your client cpu or disk

Re: cms content and numerous sort operations

2010-04-05 Thread Brandon Williams
On Fri, Apr 2, 2010 at 10:06 AM, S Ahmed wrote: > Greetings! > > Content management systems usually have complex sort operations, how would > this be best handled with Cassandra? > > Is the only way to handle this type of situation to build indexes for each > and every sort? > > example model: >

Re: Heap sudden jump during import

2010-04-05 Thread Jonathan Ellis
Usually sudden heap jumps involve compacting large rows. 0.6 (since beta3) includes a warning log when it finishes compacts a row over 500MB by default, in the hopes that this will give you enough time to fix things before whatever is making large rows makes one too large to fit in memory. On Fri

Re: 0.5.1 exception: java.io.IOException: Reached an EOL or something bizzare occured

2010-04-05 Thread Jonathan Ellis
Short answer: upgrade to 0.6. On Sat, Apr 3, 2010 at 7:56 AM, Anty wrote: > Does anyone have solve the problem?I encounter the same error too. > > On Mon, Mar 29, 2010 at 12:12 AM, Benoit Perroud wrote: >> >> I got the same error when the nodes are using lot of I/O, i.e during >> compaction. >>

Re: cascal - high level scala cassandra client (yes - another one)

2010-04-05 Thread Jonathan Ellis
Cool, you should add it to http://wiki.apache.org/cassandra/ClientOptions. (Click Login to get a sign up page.) On Sat, Apr 3, 2010 at 11:38 AM, Chris Shorrock wrote: > For the past week or so I've been developing (another) Scala based high > level Cassandra client - Cascal. While I know there's

Re: Memcached protocol?

2010-04-05 Thread Mike Malone
On Mon, Apr 5, 2010 at 1:46 PM, Paul Prescod wrote: > On Mon, Apr 5, 2010 at 1:35 PM, Mike Malone wrote: > >> That's useful information Mike. I am a bit curious about what the most > >> common use cases are for atomic increment/decrement. I'm familiar with > >> atomic add as a sort of locking me

Re: Flush Commit Log

2010-04-05 Thread Jonathan Ellis
You'll have to give a more detailed error. "nodeprobe flush" is exactly what you should be trying. On Mon, Apr 5, 2010 at 2:37 AM, JKnight JKnight wrote: > Dear all, > > How can I flush all Commit Log for Cassandra version 042? > I use nodeprobe flush but It seem does not run. > > Thank a lot fo

Re: cascal - high level scala cassandra client (yes - another one)

2010-04-05 Thread Mike Malone
On Sat, Apr 3, 2010 at 12:12 PM, Matthew Chambers wrote: > Your git page looks great, I like your cassandra explanation and graphic. +1 on the docs - they're very nice. Off-topic, but what'd you use to create that graphic? Mike

Re: cascal - high level scala cassandra client (yes - another one)

2010-04-05 Thread Chris Shorrock
Thanks guys - Will definitely toss mention of it in the Wiki..(the graphic was created using http://yuml.me/ - Great tool for quickly throwing something together using pretty simple syntax) On Mon, Apr 5, 2010 at 2:43 PM, Mike Malone wrote: > On Sat, Apr 3, 2010 at 12:12 PM, Matthew Chambers >

Re: Question about node failure...

2010-04-05 Thread Rob Coli
On 4/5/10 2:11 PM, Jonathan Ellis wrote: On Mon, Mar 29, 2010 at 6:42 PM, Tatu Saloranta wrote: Perhaps it would be good to have convenience workflow for replacing broken host ("squashing lemons")? I would assume that most common use [ snip ] Does anyone have numbers on how badly "nodetool re

Re: Question about node failure...

2010-04-05 Thread Jonathan Ellis
On Mon, Apr 5, 2010 at 5:20 PM, Rob Coli wrote: > On 4/5/10 2:11 PM, Jonathan Ellis wrote: >> >> On Mon, Mar 29, 2010 at 6:42 PM, Tatu Saloranta >>  wrote: >>> >>> Perhaps it would be good to have convenience workflow for replacing >>> broken host ("squashing lemons")? I would assume that most com

Re: Memcached protocol?

2010-04-05 Thread Tatu Saloranta
On Mon, Apr 5, 2010 at 1:46 PM, Paul Prescod wrote: > On Mon, Apr 5, 2010 at 1:35 PM, Mike Malone wrote: >>> That's useful information Mike. I am a bit curious about what the most >>> common use cases are for atomic increment/decrement. I'm familiar with >>> atomic add as a sort of locking mechan

cassandra data viewer?

2010-04-05 Thread AJ Chen
Is there a generic GUI tool for viewing cassandra datastore? being able to view and edit data from a GUI tool like oracle sqldeveloper is very useful. -aj

Re: Memcached protocol?

2010-04-05 Thread Paul Prescod
On Mon, Apr 5, 2010 at 4:48 PM, Tatu Saloranta wrote: > ... > > I would think that there is also possibility of losing some > increments, or perhaps getting duplicate increments? I believe that with vector clocks in Cassandra 0.7 you won't lose anything. The conflict resolver will do the summatio

Re: cassandra data viewer?

2010-04-05 Thread selam
look at chiton on github. On Tue, Apr 6, 2010 at 3:06 AM, AJ Chen wrote: > Is there a generic GUI tool for viewing cassandra datastore? being able to > view and edit data from a GUI tool like oracle sqldeveloper is very useful. > -aj > -- Saygılar && İyi Çalışmalar Timu EREN ( a.k.a selam )

Re: Flush Commit Log

2010-04-05 Thread JKnight JKnight
Thanks Jonathan, When I run "nodeprobe flush" with parameter -host is Cassandra server setup on my computer, my computer is hang up by Cassandra. (When I kill all Java process, the computer will work well) Yesterday, when run "nodeprobe flush" on my live server, I didn't flush all keyspace so tha

Re: Flush Commit Log

2010-04-05 Thread Jonathan Ellis
On Mon, Apr 5, 2010 at 9:11 PM, JKnight JKnight wrote: > Thanks Jonathan, > > When I run "nodeprobe flush" with parameter -host is Cassandra server setup > on my computer, my computer is hang up by Cassandra. (When I kill all Java > process, the computer will work well) Sounds like flush generate

Overwhelming a cluster with writes?

2010-04-05 Thread Ilya Maykov
Hi all, I've just started experimenting with Cassandra to get a feel for the system. I've set up a test cluster and to get a ballpark idea of its performance I wrote a simple tool to load some toy data into the system. Surprisingly, I am able to "overwhelm" my 4-node cluster with writes from a sin

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Ilya Maykov
I just tried the same test with ConsistencyLevel.ALL, and the problem went away - the writes are somewhat slower but the cluster never gets into a bad state. So, I wonder if this is a bug in Cassandra's handling of async / "non-ConsistencyLevel.ALL" writes ... -- Ilya On Mon, Apr 5, 2010 at 9:31

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Boris Shulman
You are running out of memory on your nodes. Before the final crash your nodes are probably slow due to GC. What is your memtable size? What cache options did you configure? On Tue, Apr 6, 2010 at 7:31 AM, Ilya Maykov wrote: > Hi all, > > I've just started experimenting with Cassandra to get a f

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Ilya Maykov
I'm running the nodes with a JVM heap size of 6GB, and here are the related options from my storage-conf.xml. As mentioned in the first email, I left everything at the default value. I briefly googled around for "Cassandra performance tuning" etc but haven't found a definitive guide ... any help wi

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Ran Tavory
Do you see one of the disks used by cassandra filled up when a node crashes? On Tue, Apr 6, 2010 at 9:39 AM, Ilya Maykov wrote: > I'm running the nodes with a JVM heap size of 6GB, and here are the > related options from my storage-conf.xml. As mentioned in the first > email, I left everything a

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Ilya Maykov
No, the disks on all nodes have about 750GB free space. Also as mentioned in my follow-up email, writing with ConsistencyLevel.ALL makes the slowdowns / crashes go away. -- Ilya On Mon, Apr 5, 2010 at 11:46 PM, Ran Tavory wrote: > Do you see one of the disks used by cassandra filled up when a no

Re: Overwhelming a cluster with writes?

2010-04-05 Thread Benjamin Black
You are blowing away the mostly saner JVM_OPTS running it that way. Edit cassandra.in.sh (or wherever config is on your system) to increase mx to 4G (not 6G, for now) and leave everything else untouched and do not specify JVM_OPTS on the command line. See if you get the same behavior. b On Mon,

Re: Memcached protocol?

2010-04-05 Thread Tatu Saloranta
On Mon, Apr 5, 2010 at 5:10 PM, Paul Prescod wrote: > On Mon, Apr 5, 2010 at 4:48 PM, Tatu Saloranta wrote: >> ... >> >> I would think that there is also possibility of losing some >> increments, or perhaps getting duplicate increments? > > I believe that with vector clocks in Cassandra 0.7 you w