Re: Multi-tenancy, and authentication and authorization

I think tuning of Cassandra is overly complex, and even with a single tenant you can run into problems with too many CFs. Right now there is a one-to-one mapping between memtables and SSTables. Instead of that, would it be possible to have one giant memtable for each Cassandra instance, with parti

Keys must be written in ascending order

I'm upgrading an 0.6 cluster to 0.7 in a testing environment. In cleaning up one of the nodes I get the exception below. Googling around seems to reveal people having trouble with it caused by too-small heap sizes but that doesn't look to be what's going on here. Am I missing something obvious?

Re: Java cient

Thanks everyone. I guess, I should go with hector On 18 Jan 2011 17:41, "Alois Bělaška" wrote: > Definitelly Pelops https://github.com/s7/scale7-pelops > > 2011/1/18 Noble Paul നോബിള്‍ नोब्ळ् > >> What is the most commonly used java client library? Which is the the most >> mature/feature complete

RE: about the hector client

I will try it again, thank you . -Original Message- From: Ashish [mailto:paliwalash...@gmail.com] Sent: Wednesday, January 19, 2011 2:16 PM To: user@cassandra.apache.org Subject: Re: about the hector client Working fine for me. Can you pls try again. thanks ashish On Wed, Jan 19, 2011

Re: about the hector client

Working fine for me. Can you pls try again. thanks ashish On Wed, Jan 19, 2011 at 11:42 AM, raoyixuan (Shandy) wrote: > The url is unavailable > > > > From: Aaron Morton [mailto:aa...@thelastpickle.com] > Sent: Wednesday, January 19, 2011 12:17 PM > To: user@cassandra.apache.org > Subject: Re: a

RE: about the hector client

The url is unavailable From: Aaron Morton [mailto:aa...@thelastpickle.com] Sent: Wednesday, January 19, 2011 12:17 PM To: user@cassandra.apache.org Subject: Re: about the hector client Try the hector user group for help on how to use the client http://groups.google.com/group/hector-users You ca

Re: about the hector client

OK if I add a link to https://github.com/zznate/hector-examples to the wiki page for clients http://wiki.apache.org/cassandra/ClientOptions  ?AOn 19 Jan, 2011,at 05:22 PM, Jonathan Ellis wrote:Most often, you will define schema with the cli. Programmatic schema definition is "advanced" in Cassand

Re: about the hector client

Most often, you will define schema with the cli. Programmatic schema definition is "advanced" in Cassandra, just as in relational databases. On Tue, Jan 18, 2011 at 10:19 PM, Jeremy Hanna wrote: > Definitely get involved with that google group, but some examples are found > here: > https://gith

Re: about the hector client

Definitely get involved with that google group, but some examples are found here: https://github.com/zznate/hector-examples/blob/master/src/main/java/com/riptano/cassandra/hector/example/SchemaManipulation.java On Jan 18, 2011, at 10:17 PM, Aaron Morton wrote: > Try the hector user group for hel

Re: about the hector client

Try the hector user group for help on how to use the client http://groups.google.com/group/hector-usersYou can also create a keyspace in a cassandra cluster via the cassandra-cli command line interface Take a look at the tools online help if you're interested. AaronOn 19 Jan, 2011,at 05:00 PM, "rao

about the hector client

Can you tell me the exactly steps to create a keyspace by hector client? 华为技术有限公司 Huawei Technologies Co., Ltd.[Company_logo] Phone: 28358610 Mobile: 13425182943 Email: raoyix...@huawei.com 地址：深圳市龙岗区坂田华为基地 邮编：518129 Huawei Technologies Co., Ltd. Bantian, Longgang

Re: Tombstone lifespan after multiple deletions

On Wed, Jan 19, 2011 at 11:35 AM, Germán Kondolf wrote: > Yes, that's what I meant, but correct me if I'm wrong, when a deletion > comes after another deletion for the same row or column will the gc-before > count against the last one, isn't it? > > IIRC, after compaction. even if the row key is n

Re: Tombstone lifespan after multiple deletions

Yes, that's what I meant, but correct me if I'm wrong, when a deletion comes after another deletion for the same row or column will the gc-before count against the last one, isn't it? Maybe knowing that all the subsequent versions of a deletion are deletions too, it could take the first timesta

Re: Tombstone lifespan after multiple deletions

I'm not clear here. Are you worried about the later inserted tombstone prevents the whole row from being reclaimed and the storage space can not be freed? To my knowledge, after major compaction, only the row key and tombstone are kept. Is it a big deal? best regards, hanzhu On Tue, Jan 18,

Re: Tombstone lifespan after multiple deletions

If the tombstone is older than the row or column inserted later, is the tombstone skipped entirely after compaction? best regards, hanzhu On Wed, Jan 19, 2011 at 11:16 AM, Jonathan Ellis wrote: > If you mean that multiple tombstones for the same row or column should > be merged into a single o

Re: Tombstone lifespan after multiple deletions

If you mean that multiple tombstones for the same row or column should be merged into a single one at compaction time, then yes, that is what happens. On Tue, Jan 18, 2011 at 7:53 PM, Germán Kondolf wrote: > Maybe it could be taken into account when the compaction is executed, > if I only have a

Re: Tombstone lifespan after multiple deletions

Maybe it could be taken into account when the compaction is executed, if I only have a consecutive list of uninterrupted tombstones it could only care about the first. It sounds like the-way-it-should-be, maybe as a part of the "row-reduce" process. Is it feasible? Looking into the CASSANDRA-1074

Re: Java cient

Pelops is a nice lib. I found it very easy to use and the developers are very responsive to requests for information and/or bugs, etc. I have not tried hector On Tue, Jan 18, 2011 at 11:11 PM, Alois Bělaška wrote: > Definitelly Pelops https://github.com/s7/scale7-pelops > > 2011/1/18 Noble Paul

Re: json2sstable NPE

AFAIK the CF must exist. Create it and try again.AOn 19 Jan, 2011,at 12:03 PM, ruslan usifov wrote:Thats odd, the line before line 68 has an assertion that should have kicked in. Are you on the release version of 0.7.0 ?  Yes i use release downloaded from official site  Does the "test" CF exist in

Re: json2sstable NPE

Thats odd, the line before line 68 has an assertion that should have kicked > in. Are you on the release version of 0.7.0 ? > > Yes i use release downloaded from official site > Does the "test" CF exist in the keyspace "test" in your cluster ? > > > no it doesn't exists

Re: json2sstable NPE

Thats odd, the line before line 68 has an assertion that should have kicked in. Are you on the release version of 0.7.0 ? Does the "test" CF exist in the keyspace "test" in your cluster ? AaronOn 19 Jan, 2011,at 11:37 AM, ruslan usifov wrote:HelloI have problem when use json2sstable (in cassandra

json2sstable NPE

Hello I have problem when use json2sstable (in cassandra 0.7). When i invoke: json2sstable -K test -c test D:\apache-cassandra-0.7.0\bin\test-e-1-Data.json F:\cassandra\test\test\test-e-1-Data.db I got NPE: WARN 01:31:38,750 Schema definitions were defined both locally and in cassandra.yaml. D

Re: please help with multiget

On Tue, Jan 18, 2011 at 4:29 PM, Shu Zhang wrote: > Well, I don't think what I'm describing is complicated semantics. I think > I've described general batch operation design and something that is > symmetrical the batch_mutate method already on the Cassandra API. You are > right, I can solve th

Re: Multi-tenancy, and authentication and authorization

I've used an S3 style data model with a REST interface (varnish > nginx > tornado > cassandra), users do not see anything remotely cassandra like. AaronOn 19 Jan, 2011,at 10:27 AM, Stephen Connolly wrote:I would imagine it to be somewhat easy to implement this via a thrift wrapper so that each ten

RE: please help with multiget

Well, I don't think what I'm describing is complicated semantics. I think I've described general batch operation design and something that is symmetrical the batch_mutate method already on the Cassandra API. You are right, I can solve the problem with further denormalization, and the approach of

Re: Multi-tenancy, and authentication and authorization

I would imagine it to be somewhat easy to implement this via a thrift wrapper so that each tenant is connecting to the proxy thrift server that masks the fact that there are multiple tenants... or is that how people are thinking about this - Stephen --- Sent from my Android phone, so random spell

Re: Multi-tenancy, and authentication and authorization

As everyone says, it's not issues with the Keyspace directly as they are just a container. It's the CF's in the keyspace, but let's just say keyspace cause it's easier. As things stand, if you allow point and click creation for keyspaces you will hand over control of the memory requirements to

Re: please help with multiget

I think the general approach is to denormalise data to remove the need for complicated semantics when reading. Aaron On 19/01/2011, at 7:57 AM, Shu Zhang wrote: > Well, maybe making a batch-get is not anymore efficient on the server side > but without it, you can get bottlenecked on client-

Re: Tombstone lifespan after multiple deletions

Sylvain, Just to check my knowledge. Is this only the case if the delete is sent without a super column or predicate? What about a delete for a specific column that did not exist? Thanks Aaron On 19/01/2011, at 2:58 AM, David Boxenhorn wrote: > Thanks. > > On Tue, Jan 18, 2011 at 3:55 PM,

Re: Multi-tenancy, and authentication and authorization

Hi Jeremy, thanks, I was really coming at it from the question of whether keyspaces were a functional basis for multitenancy in Cassandra. I think the MT issues discussed on the wiki page are the , but I'd like to get a better understanding of the core issue of keyspaces and then try to get that o

Re: Multi-tenancy, and authentication and authorization

Feel free to use that wiki page or another wiki page to collaborate on more pressing multi tenant issues. The wiki is editable by all. The MultiTenant page was meant as a launching point for tracking progress on things we could think of wrt MT. Obviously the memtable problem is the largest co

Re: changing the replication level on the fly

On 01/18/2011 11:36 AM, Edward Capriolo wrote: On Tue, Jan 18, 2011 at 2:14 PM, Jeremy Stribling wrote: Hi, I've noticed in the new Cassandra 0.7.0 release that if I have a keyspace with a replication level of 2, but only one Cassandra node, I cannot insert anything into the system. Lik

Re: changing the replication level on the fly

On Tue, Jan 18, 2011 at 2:14 PM, Jeremy Stribling wrote: > Hi, > > I've noticed in the new Cassandra 0.7.0 release that if I have a keyspace > with a replication level of 2, but only one Cassandra node, I cannot insert > anything into the system.  Likely this was a bug in the old release I was > u

changing the replication level on the fly

Hi, I've noticed in the new Cassandra 0.7.0 release that if I have a keyspace with a replication level of 2, but only one Cassandra node, I cannot insert anything into the system. Likely this was a bug in the old release I was using (0.6.8 -- is there a JIRA describing this problem?). Howev

Re: Multi-tenancy, and authentication and authorization

Hi Indika, I've done a lot of work using the keyspace per tenant model, and I'm seeing big problems with the memory consumption, even though it's certainly the most clean way to implement it. Luckily, before I used the keyspace per tenant approach, I'd implemented my system using a single keyspace

RE: please help with multiget

Well, maybe making a batch-get is not anymore efficient on the server side but without it, you can get bottlenecked on client-server connections and client resources. If the number of requests you want to batch is on the order of connections in your pool, then yes, making gets in parallel is as

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

When doing mixed types on slicing operations, you should use ByteArraySerializer and handle the conversions by hand. We have an issue open for making this more graceful. On Tue, Jan 18, 2011 at 10:07 AM, Timo Nentwig wrote: > > On Jan 18, 2011, at 12:05, Timo Nentwig wrote: > >> >> On Jan 18, 20

Re: Multi-tenancy, and authentication and authorization

Hi Aaron, I read some articles about the Cassandra, and now understand a little bit about trade-offs. I feel the goal should be to optimize memory as well as performance. I have to consider the number of column families, the columns per a family, the number of rows, the memtable’s threshold, and

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

On Jan 18, 2011, at 12:05, Timo Nentwig wrote: > > On Jan 18, 2011, at 12:02, Aaron Morton wrote: > >> Does wrapping foo in single quotes help? > > No. > >> Also, does this help >> http://www.datastax.com/blog/whats-new-cassandra-07-secondary-indexes > > Actually this doesn't even compile b

Re: Question re: the use of multiple ColumnFamilies

> Sorry for the delayed reply, but thanks very much - this pointed me at the > exact problem. I found that the queue size here was equal to the number of > configured DataFileDirectories, so a good test was to lie to Cassandra and > claim that there were more DataFileDirectories than I needed. Inte

Re: Question re: the use of multiple ColumnFamilies

Sorry for the delayed reply, but thanks very much - this pointed me at the exact problem. I found that the queue size here was equal to the number of configured DataFileDirectories, so a good test was to lie to Cassandra and claim that there were more DataFileDirectories than I needed. Interest

Re: What is be the best possible client option available to a PHP developer for implementing an application ready for production environments ?

I think we might need to go with full Java implementation only, in that case, to live up with Hector as we do not find any other better option. @Dave: Thanks for the links but we wouldn't much prefer to go with thrift implementation because of frequently changing api and other complexities there.

Re: Tombstone lifespan after multiple deletions

Thanks. On Tue, Jan 18, 2011 at 3:55 PM, Sylvain Lebresne wrote: > On Tue, Jan 18, 2011 at 2:41 PM, David Boxenhorn > wrote: > > Thanks, Aaron, but I'm not 100% clear. > > > > My situation is this: My use case spins off rows (not columns) that I no > > longer need and want to delete. It is possi

Re: Tombstone lifespan after multiple deletions

On Tue, Jan 18, 2011 at 2:41 PM, David Boxenhorn wrote: > Thanks, Aaron, but I'm not 100% clear. > > My situation is this: My use case spins off rows (not columns) that I no > longer need and want to delete. It is possible that these rows were never > created in the first place, or were already de

Re: Tombstone lifespan after multiple deletions

Thanks, Aaron, but I'm not 100% clear. My situation is this: My use case spins off rows (not columns) that I no longer need and want to delete. It is possible that these rows were never created in the first place, or were already deleted. This is a very large cleanup task that normally deletes a l

Re: Java cient

Definitelly Pelops https://github.com/s7/scale7-pelops 2011/1/18 Noble Paul നോബിള്‍ नोब्ळ् > What is the most commonly used java client library? Which is the the most > mature/feature complete? > Noble >

Re: Java cient

We moved over to Hector when we went to Cassandra 0.7, it was a painless and worthwhile experience. > What is the most commonly used java client library? Which is the the most > mature/feature complete? > --Jools

Re: Multi-tenancy, and authentication and authorization

Hi Aaron, I appreciate your help. I am a newbie to Cassandra - just began to study the code-base. Do you suggest the following approach? *1) No changes are in either keyspace names or column family names but the row-key would be ‘the actual row key’ + 'tenant ID'. It is needed to keep separate m

Re: Multi-tenancy, and authentication and authorization

Moving to user list On Tue, Jan 18, 2011 at 4:05 PM, Aaron Morton wrote: > Have a read about JVM heap sizing here > http://wiki.apache.org/cassandra/MemtableThresholds > > If you let people create keyspaces with a mouse click you will soon run out > of memory. > > I use Cassandra to provide a sel

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

On Jan 18, 2011, at 12:02, Aaron Morton wrote: > Does wrapping foo in single quotes help? No. > Also, does this help > http://www.datastax.com/blog/whats-new-cassandra-07-secondary-indexes Actually this doesn't even compile because addGtExpression expects a String type (?!). StringSerialize

RE: Super CF or two CFs?

Thanks for the clarification. Hugo & Steven. Subject: Re: Super CF or two CFs? From: aa...@thelastpickle.com Date: Tue, 18 Jan 2011 23:51:25 +1300 To: user@cassandra.apache.org Sorry was not suggesting super CF is better in the first para, I think it applies to any CF. The role of compaction i

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

Does wrapping foo in single quotes help? Also, does this help http://www.datastax.com/blog/whats-new-cassandra-07-secondary-indexes Aaron On 18/01/2011, at 11:54 PM, Timo Nentwig wrote: > I put a secondary index on rc (IntegerType) and user_agent (AsciiType). > > Don't understand this bevahio

Re: Is there a concept of a session

Thanks Aaron. I will look into codebase. Thanks, Indika On Tue, Jan 18, 2011 at 4:55 PM, Aaron Morton wrote: > There are no cookies in thrift. > > All connection state is managed by the server. It's a tcp connection. > Multiple request are sent over it,it stays around as long as the client > wa

Re: Java cient

http://wiki.apache.org/cassandra/ClientOptions Hector On 18/01/2011, at 11:48 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote: > What is the most commonly used java client library? Which is the the most > mature/feature complete? > Noble

Re: Is there a concept of a session

There are no cookies in thrift. All connection state is managed by the server. It's a tcp connection. Multiple request are sent over it,it stays around as long as the client wants it to. Try the Hector mailing list for details on it's implementation. Aaron On 18/01/2011, at 11:15 PM, indika kum

Re: Java cient

Hector is excellent. https://github.com/rantav/hector http://www.datastax.com/sites/default/files/hector-v2-client-doc.pdf 2011/1/18 Noble Paul നോബിള്‍ नोब्ळ् : > What is the most commonly used java client library? Which is the the most > mature/feature complete? > Noble

cassandra-cli: where a and b (works) vs. where b and a (doesn't)

I put a secondary index on rc (IntegerType) and user_agent (AsciiType). Don't understand this bevahiour at all, can somebody explain? [default@tracking] get crawler where user_agent=foo and rc=200; 0 Row Returned. [default@tracking] get crawler where rc=200 and user_agent=foo; ---

Re: Super CF or two CFs?

Sorry was not suggesting super CF is better in the first para, I think it applies to any CF. The role of compaction is to (among other things) reduce the number of SSTables for each CF. The logical endpoint of this process would be a single file for each CF, giving the lowest possible IO. The v

Re: Java cient

I think its Hector... 2011/1/18 Noble Paul നോബിള്‍ नोब्ळ् > What is the most commonly used java client library? Which is the the most > mature/feature complete? > Noble >

Java cient

What is the most commonly used java client library? Which is the the most mature/feature complete? Noble

Re: Is there a concept of a session

Thanks Aaron... Hector cannot uses strategies such as cookies for maintaining session, so it has to make the authentication call each time? In the Cassandra server, I see 'ThreadLocal'. It keeps the session information? How long is a session alive? Does the connection means a TCP connection? i

RE: Super CF or two CFs?

Thanks for the answer. It provides me the insight I'm looking for. However, I'm also a bit confused as your first paragraph seems to indicate that using a SCF is better, whereas the last sentence states just the opposite. Do I interpret correctly that this is because of the compactions that put

Re: Is there a concept of a session

I'm just going to assume Hector is doing the right thing, and you probably can as well :) Have you checked out the documentation here ? http://www.riptano.com/sites/default/files/hector-v2-client-doc.pdf (also yes the session is server side, each connection has a thread on the server it connect

Re: Is there a concept of a session

Hi Aaron, Thank you very much. I am going to use the hector client library. There is a method for creating a connection for a cluster in that library. But, inside the source code, I noticed that each time it calls 'login' method. Is there a server-side session? Thanks, Indika On Tue, Jan 18, 2

Re: Tombstone lifespan after multiple deletions

AFAIK that's not necessary, there is no need to worry about previous deletes. You can delete stuff that does not even exist, neither batch_mutate or remove are going to throw an error. All the columns that were (roughly speaking) present at your first deletion will be available for GC at the en

Re: Cassandra/Hadoop only write few columns

It was a typo in my example code in this email. I logged the list to make sure that everything was correct before trigger the write. Will try to enable debug on both cassandra and hadoop next. Thanks, Trung. On Tue, Jan 18, 2011 at 1:21 AM, Aaron Morton wrote: > May just be your example code, b

Re: Cassandra/Hadoop only write few columns

May just be your example code, but you are repeating colName2 . Can you log the mutation list before you write it and confirm you have unique column names? Can you turn up the logging to DEBUG for the hadoop job and the Cassandra cluster to see what's happening? Aaron On 18/01/2011, at 9:40 PM

Re: What is be the best possible client option available to a PHP developer for implementing an application ready for production environments ?

I can't comment of phpcassa directly, but we use Cassandra plus PHP in production without any difficulties. We are happy with the performance. Most of the information we needed to get started we found here: https://wiki.fourkitchens.com/display/PF/Using+Cassandra+with+PHP This includes details o

Re: balancing load

On 17/01/2011 19:27, Edward Capriolo wrote: cfstats is reporting you have an 8GB Row! I think you could be writing all your data to a few keys. Your right, my n00b fault, I was writing everything to one key, the problem was i had Offer['id'][$UID] = value it made it easy before to do a "c

Re: Is there a concept of a session

Yes, the client should maintain it's connection to the cluster. The connection holds the login credentials and the keyspace to use. This is normally managed by the client, which one are you using? Aaron On 18/01/2011, at 9:58 PM, indika kumara wrote: > Hi All, > > Is there a concept of a sess

Is there a concept of a session

Hi All, Is there a concept of a session? I would like to log-in(authenticate) one time into the Cassandra, and then subsequently access the Cassandra without authenticating again. Thanks, Indika

Re: Super CF or two CFs?

With regard to overwrites, and assuming you always want to get all the data for a stock ticker. Any read on the volatile data will potentially touch many sstables, this IO is unavoidable to read this data so we may as well read as many cols as possible at this time. Whereas if you split the data

Cassandra/Hadoop only write few columns

Hi, I'm working on ColumnFamilyOutputFormat and for some reasons my reduce class does not write all columns to cassandra. I tried to modify mapreduce.output.columnfamilyoutputformat.batch.threshold with some different values (1, 8, .. etc) but no thing changes. What i'm having in my reduce class

Re: Tombstone lifespan after multiple deletions

Thanks. In other words, before I delete something, I should check to see whether it exists as a live row in the first place. On Tue, Jan 18, 2011 at 9:24 AM, Ryan King wrote: > On Sun, Jan 16, 2011 at 6:53 AM, David Boxenhorn > wrote: > > If I delete a row, and later on delete it again, before

RE: Super CF or two CFs?

Some of the fields are indeed written in one shot, but others (such as label and categories) are added later, so I think the question still stands. Hugo. From: dri...@gmail.com Date: Mon, 17 Jan 2011 18:47:28 -0600 Subject: Re: Super CF or two CFs? To: user@cassandra.apache.org On Mon, Jan 17,

Re: quorum calculation seems to depend on previous selected nodes

On 18 January 2011 07:15, Samuel Benz wrote: > On 01/17/2011 09:28 PM, Jonathan Ellis wrote: >> On Mon, Jan 17, 2011 at 2:10 PM, Samuel Benz wrote: > Case1: > If 'TEST' was previous stored on Node1, Node2, Node3 -> The update will > succeed. > > Case2: > If 'TEST' was prev