Re: SSL & Streaming

2011-06-13 Thread Sasha Dolgy
AJ was responding to an email I sent in Marchalthough i do appreciate the quick reaponse from the community ;) i moved on to our implementation of vpn... On Jun 14, 2011 1:35 AM, "aaron morton" wrote: > Sasha does https://github.com/apache/cassandra/blob/cassandra-0.8.0/conf/cassandra.yaml#L36

Re: odd logs after repair

2011-06-13 Thread aaron morton
Count of the columns in a row, not an exact count as that would requiring stopping clients from writing to the row and we do not do that. Have a poke around http://www.datastax.com/docs/0.8/data_model/index and http://wiki.apache.org/cassandra/DataModel Or are you asking about counter columns

Re: SSL & Streaming

2011-06-13 Thread aaron morton
Sasha does https://github.com/apache/cassandra/blob/cassandra-0.8.0/conf/cassandra.yaml#L362 help ? A - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 13 Jun 2011, at 23:26, AJ wrote: > Performance-wise, I think it would be better to j

Re: get_indexed_slices ~ simple map-reduce

2011-06-13 Thread aaron morton
From a quick read of the code in o.a.c.db.ColumnFamilyStore.scan()... Candidate rows are first read by applying the most selected equality predicate. From those candidate rows... 1) If the SlicePredicate has a SliceRange the query execution will read all columns for the candidate row if the b

Re: insufficient space to compact even the two smallest files, aborting

2011-06-13 Thread Jonathan Ellis
You may also have been running into https://issues.apache.org/jira/browse/CASSANDRA-2765. We'll have a fix for this in 0.8.1. 2011/6/13 Héctor Izquierdo Seliva : > I was already way over the minimum. There were 12 sstables. Also, is > there any reason why scrub got stuck? I did not see anything in

Is this the proper use of OPP?

2011-06-13 Thread AJ
I'm just becoming aware of the restrictions of using an OPP as compared to Random. Please let me know if I understand this correctly. First off, if using the OPP only for an increased performance of range queries, then it will probably be very hard to predict if you will end up with hotspots

Re: Re: minor vs major compaction and purging data

2011-06-13 Thread Sebastien Coutu
Thanks! This clarifies a few things :) On Mon, Jun 13, 2011 at 4:09 PM, wrote: > Cleanup removes any data that node is no longer responsible for, according > to the node's token range. A node can have data it is no longer responsible > for if you do certain maintenance operations like move or lo

Re: Re: minor vs major compaction and purging data

2011-06-13 Thread jonathan . colby
Cleanup removes any data that node is no longer responsible for, according to the node's token range. A node can have data it is no longer responsible for if you do certain maintenance operations like move or loadbalance. On , Sebastien Coutu wrote: How about cleanups? What would be the diff

Re: Counter Column in Cassandra

2011-06-13 Thread Patricio Echagüe
It's a column whose content represents a distributed counter. http://wiki.apache.org/cassandra/Counters On Mon, Jun 13, 2011 at 8:29 AM, Sijie YANG wrote: > Hi, All > > I am newbie to cassandra. I have a simple question but don't find any clear > answer by searching google: > What's the meaning

Re: one way to make counter delete work better

2011-06-13 Thread Yang
ok, I think it's better to understand it this way, then it is really simple and intuitive: my proposed way of counter update can be simply seen as a combination of regular columns + current counter columns: regular column : [ value: "wipes out every bucket to nil" , clock: epoch number] then w

Re: minor vs major compaction and purging data

2011-06-13 Thread Sebastien Coutu
How about cleanups? What would be the difference between cleanup and compactions? On Sat, Jun 11, 2011 at 8:14 AM, Jonathan Ellis wrote: > Yes. > > On Sat, Jun 11, 2011 at 6:08 AM, Jonathan Colby > wrote: > > I've been reading inconsistent descriptions of what major and minor > compactions do.

Re: one way to make counter delete work better

2011-06-13 Thread Yang
I think this approach also works for your scenario: I thought that the issue is only concerned with merging within the same leader; but you pointed out that a similar merging happens between leaders too, now I see that the same rules on epoch number also applies to inter-leader data merging, speci

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread AJ
On 6/13/2011 10:14 AM, Stephen Connolly wrote: store the query inverted. that way empty -> deleted I don't know what that means... get the other columns? Can you elaborate? Is there docs for this or is this a hack/workaround? the tombstones are stored for each column that had data IIRC.

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread Stephen Connolly
On 13 June 2011 17:09, AJ wrote: > On 6/13/2011 9:25 AM, Stephen Connolly wrote: >> >> On 13 June 2011 16:14, AJ  wrote: >>> >>> On 6/13/2011 7:03 AM, Stephen Connolly wrote: It returns the set of columns for the set of rows... how do you determine the difference between a completel

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread AJ
On 6/13/2011 9:25 AM, Stephen Connolly wrote: On 13 June 2011 16:14, AJ wrote: On 6/13/2011 7:03 AM, Stephen Connolly wrote: It returns the set of columns for the set of rows... how do you determine the difference between a completely empty row and a row that just does not have any of the matc

Re: one way to make counter delete work better

2011-06-13 Thread Jonathan Ellis
I don't think that's bulletproof either. For instance, what if the two adds go to replica 1 but the delete to replica 2? Bottom line (and this was discussed on the original delete-for-counters ticket, https://issues.apache.org/jira/browse/CASSANDRA-2101), counter deletes are not fully commutative

one way to make counter delete work better

2011-06-13 Thread Yang
as https://issues.apache.org/jira/browse/CASSANDRA-2101 indicates, the problem with counter delete is in scenarios like the following: add 1, clock 100 delete , clock 200 add 2 , clock 300 if the 1st and 3rd operations are merged in SStable compaction, then we have delete clock 200 add 3, clo

Re: problem in using get_range() function

2011-06-13 Thread Richard Low
If you set key_start and key_finish to empty strings then you will get all rows. You can do this with either RP or OPP, although with RP they will not be returned in order. -- Richard Low Acunu | http://www.acunu.com | @acunu On Mon, Jun 13, 2011 at 2:31 PM, Amrita Jayakumar wrote: > can u tel

Re: count column in Cassandra

2011-06-13 Thread Sasha Dolgy
probably helpful if you change the subject when posting about a different topic. Is your question about "counters" or the "count" function? Counters are cool. Count allows you to determine how many columns exist in a row. -sd On Mon, Jun 13, 2011 at 5:27 PM, Sijie YANG wrote: > Hi, All > I am

Re: insufficient space to compact even the two smallest files, aborting

2011-06-13 Thread Jonathan Ellis
As Terje already said in this thread, the threshold is per bucket (group of similarly sized sstables) not per CF. 2011/6/13 Héctor Izquierdo Seliva : > I was already way over the minimum. There were 12 sstables. Also, is > there any reason why scrub got stuck? I did not see anything in the > logs.

Counter Column in Cassandra

2011-06-13 Thread Sijie YANG
Hi, All I am newbie to cassandra. I have a simple question but don't find any clear answer by searching google: What's the meaning of counter column in Cassandra? Best

Re: odd logs after repair

2011-06-13 Thread Sijie YANG
Hi, All I am newbie to cassandra. I have a simple question but don't find any clear answer by searching google: What's the meaning of count column in Cassandra? Thanks.

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread Stephen Connolly
On 13 June 2011 16:14, AJ wrote: > On 6/13/2011 7:03 AM, Stephen Connolly wrote: >> >> It returns the set of columns for the set of rows... how do you >> determine the difference between a completely empty row and a row that >> just does not have any of the matching columns? > > I would expect it

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread AJ
On 6/13/2011 7:03 AM, Stephen Connolly wrote: It returns the set of columns for the set of rows... how do you determine the difference between a completely empty row and a row that just does not have any of the matching columns? I would expect it to not return anything (no row at all) for both

Re: insufficient space to compact even the two smallest files, aborting

2011-06-13 Thread Héctor Izquierdo Seliva
I was already way over the minimum. There were 12 sstables. Also, is there any reason why scrub got stuck? I did not see anything in the logs. Via jmx I saw that the scrubbed bytes were equal to one of the sstables size, and it stuck there for a couple hours . El lun, 13-06-2011 a las 22:55 +0900,

Re: odd logs after repair

2011-06-13 Thread Sasha Dolgy
hm. that's not it. we've been using a non-standard jmx port for some time i've dropped the keyspace and recreated ... wonder if that'll help On Mon, Jun 13, 2011 at 3:57 PM, Tyler Hobbs wrote: > On Mon, Jun 13, 2011 at 8:41 AM, Sasha Dolgy wrote: >> >> I recall there being a discussion a

Re: odd logs after repair

2011-06-13 Thread Tyler Hobbs
On Mon, Jun 13, 2011 at 8:41 AM, Sasha Dolgy wrote: > I recall there being a discussion about a default port changing from > 0.7.x to 0.8.x ...this was JMX, correct? Or were there others. > Yes, the default JMX port changed from 8080 to 7199. I don't think there were any others. -- Tyler Hob

Re: insufficient space to compact even the two smallest files, aborting

2011-06-13 Thread Terje Marthinussen
That most likely happened just because after scrub you had new files and got over the "4" file minimum limit. https://issues.apache.org/jira/browse/CASSANDRA-2697 Is the bug report. 2011/6/13 Héctor Izquierdo Seliva > Hi All. I found a way to be able to compact. I have to call scrub on > the

Re: odd logs after repair

2011-06-13 Thread Sasha Dolgy
I recall there being a discussion about a default port changing from 0.7.x to 0.8.x ...this was JMX, correct? Or were there others. On Mon, Jun 13, 2011 at 3:34 PM, Sasha Dolgy wrote: > Hi Aaron, > > The error is being reported on all 4 nodes. I have confirmed (for my > own sanity) that each nod

Re: odd logs after repair

2011-06-13 Thread Sasha Dolgy
Hi Aaron, The error is being reported on all 4 nodes. I have confirmed (for my own sanity) that each node is running: ReleaseVersion: 0.8.0 I can reproduce the error on any node by trailing cassandra/logs/system.log and running "nodetool repair" INFO [manual-repair-1c6b33bc-ef14-4ec8-94f6-f146

Re: problem in using get_range() function

2011-06-13 Thread Amrita Jayakumar
can u tell me how to retrieve all the row keys in a column family??? On Mon, Jun 13, 2011 at 6:25 PM, Dan Kuebrich wrote: > Are you using the order preserving partitioner or the random partitioner > for this CF? In order to get the results you expect, you'll need to use the > OPP. > > More info:

Re: Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread Stephen Connolly
It returns the set of columns for the set of rows... how do you determine the difference between a completely empty row and a row that just does not have any of the matching columns? Well the answer is that Cassandra does not go and check whether there are any columns outside of the range you are

Re: odd logs after repair

2011-06-13 Thread aaron morton
You can double check with node tool e.g. $ ./bin/nodetool -h localhost version ReleaseVersion: 0.8.0-SNAPSHOT This error is about the internode wire protocol one node thinks another is using. Not sure how it could get confused, does it go away if you restart the node that logged the error ?

Docs: "Why do deleted keys show up during range scans?"

2011-06-13 Thread AJ
http://wiki.apache.org/cassandra/FAQ#range_ghosts "So to special case leaving out result entries for deletions, we would have to check the entire rest of the row to make sure there is no undeleted data anywhere else either (in which case leaving the key out would be an error)." The above doe

Re: insufficient space to compact even the two smallest files, aborting

2011-06-13 Thread Héctor Izquierdo Seliva
Hi All. I found a way to be able to compact. I have to call scrub on the column family. Then scrub gets stuck forever. I restart the node, and voila! I can compact again without any message about not having enough space. This looks like a bug to me. What info would be needed to fill a report? This

Re: problem in using get_range() function

2011-06-13 Thread Dan Kuebrich
Are you using the order preserving partitioner or the random partitioner for this CF? In order to get the results you expect, you'll need to use the OPP. More info: http://ria101.wordpress.com/2010/02/22/cassandra-randompartitioner-vs-orderpreservingpartitioner/ On Mon, Jun 13, 2011 at 8:47 AM,

problem in using get_range() function

2011-06-13 Thread Amrita Jayakumar
Hi, I am trying to retrieve the row_keys in a column_family witht he following code. $rows = $column_family->get_range($key_start='R17889000', $key_finish='R17893999', $row_count=1000); $count = 0; foreach($rows as $rows) { echo $count.''; $count += 1;

Re: Troubleshooting IO performance ?

2011-06-13 Thread aaron morton
To reduce the number of SSTables increase the memtable_threshold for the CF. The IO numbers may be because of compaction kicking in. The CompactionManager provides information via JMX on it's progress, or you can check the logs. You could increase the min_compaction_threshold for the CF or disa

Re: SSL & Streaming

2011-06-13 Thread AJ
Performance-wise, I think it would be better to just let the client encrypt sensitive data before storing it, versus encrypting all traffic all the time. If individual values are encrypted, then they don't have to be encrypted/decrypted during transit between nodes during the initial updates a

Re: [jira] [Commented] (CASSANDRA-2474) CQL support for compound columns

2011-06-13 Thread Mick Semb Wever
On Sun, 2011-06-12 at 18:53 +, Mick Semb Wever wrote: > > This issue could stand to be summarized (I still wish we used a > > mailing list for monsters like this). > > > This i actually really appreciate about the cassandra community. To formulate this: As a newbie here it has allowed me to