Re: unable to read saved rowcache from disk

2012-11-13 Thread Manu Zhang
actually, I'm thinking about a bug or something On Wed, Nov 14, 2012 at 3:13 PM, Edward Capriolo wrote: > http://wiki.apache.org/cassandra/LargeDataSetConsiderations > > A negative side-effect of a large row-cache is start-up time. The > periodic saving of the row cache information only saves th

Re: unable to read saved rowcache from disk

2012-11-13 Thread Edward Capriolo
http://wiki.apache.org/cassandra/LargeDataSetConsiderations A negative side-effect of a large row-cache is start-up time. The periodic saving of the row cache information only saves the keys that are cached; the data has to be pre-fetched on start-up. On a large data set, this is probably going to

RE: CF metadata syntax for an array

2012-11-13 Thread Kevin Burton
Does the array have to be a KEY? Sorry I don't understand this question. In the samples you give you specify array as CREATE COLUMNFAMILY Description ( PRIMARY_KEY (vector_name, index),

Re: read request distribution

2012-11-13 Thread aaron morton
> As soon as one CF is created, the column changes to Effective Ownership There has to be a user defined keyspace to calculate effective ownership. Effective Ownership is calculated for all keyspaces if the share the same RF, and for a named KS if specified. It's the % of the total cluster data

Re: CF metadata syntax for an array

2012-11-13 Thread aaron morton
> Would this syntax be the same for CREATE COLUMNFAMILY (as an aside what is a > ‘TABLE’ in Cassandra)? Yes, CQL 2 uses COLUMN FAMILY or Table and CQL 3 uses TABLE http://www.datastax.com/dev/blog/cql3-evolutions > In other words is this valid: Does it work ? Is so it's valid. > Does the arra

Re: Strange delay in query

2012-11-13 Thread aaron morton
Minor compactions will still be triggered whenever a size tier gets 4+ sstables (for the default compaction strategy). So it does not affect new data. It just takes longer for the biggest size tier to get to 4 files. So it takes longer to compact the big output from the major compaction. Assu

Re: Memory Manager

2012-11-13 Thread aaron morton
Have you tried using the defaults in cassandra-env.sh ? Your setting are very different. https://github.com/apache/cassandra/blob/cassandra-1.1/conf/cassandra-env.sh Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 13/11/2012, at 11:35 P

Re: Read during digest mismatch

2012-11-13 Thread sankalp kohli
Thats correct On Tue, Nov 13, 2012 at 7:06 PM, Wz1975 wrote: > From my understanding, if CL = 2, one read, one digest are sent. Only > if it isread repair, digest is sent to all replicates. > > > Thanks. > -Wei > > Sent from my Samsung smartphone on AT&T > > > Original message

Re: Read during digest mismatch

2012-11-13 Thread Wz1975
From my understanding,  if CL = 2, one read,  one digest are sent.  Only if it is    read repair,  digest is sent to all replicates. Thanks. -Wei Sent from my Samsung smartphone on AT&T Original message Subject: Re: Read during digest mismatch From: Edward Capriolo To: "us

Re: unable to read saved rowcache from disk

2012-11-13 Thread Manu Zhang
"incorrect"... what do you mean? I think it's only 15MB, which is not big. On Wed, Nov 14, 2012 at 10:38 AM, Edward Capriolo wrote: > Yes the row cache "could be" incorrect so on startup cassandra verify they > saved row cache by re reading. It takes a long time so do not save a big > row cache.

Re: Read during digest mismatch

2012-11-13 Thread Edward Capriolo
I think the code base does not benefit from having too many different read code paths. Logically what your suggesting is reasonable, but you have to consider the case of one being slow to respond. Then what? On Tuesday, November 13, 2012, Manu Zhang wrote: > If consistency is two, don't we just

Re: unable to read saved rowcache from disk

2012-11-13 Thread Edward Capriolo
Yes the row cache "could be" incorrect so on startup cassandra verify they saved row cache by re reading. It takes a long time so do not save a big row cache. On Tuesday, November 13, 2012, Manu Zhang wrote: > I have a rowcache provieded by SerializingCacheProvider. > The data that has been read

Fwd: Multiple Services

2012-11-13 Thread jer...@simpleartmarketing.com
I have multiple servers that use individual cassandra cluster. I would like to add a keyspace to store state information unique to each web service. Ideally I want to back up the keyspace that contains state information to a local core cluster without having any information shared between the web

Re: read request distribution

2012-11-13 Thread Wei Zhu
I am new to Cassandra, and 1.1.6 is the only version I have tested. Not sure about the old behavior, for 1.1.6, my observation is that for brand new cluster (with no CF created), it shows Ownership from nodetool ring, the value is 100/. As soon as one CF is created, the column changes to Effecti

Re: composite column validation_class question

2012-11-13 Thread Manu Zhang
thrift http://www.datastax.com/dev/blog/introduction-to-composite-columns-part-1 thrift-to-cql3 http://www.datastax.com/dev/blog/thrift-to-cql3 On Wed, Nov 14, 2012 at 1:57 AM, Tyler Hobbs wrote: > I'm not completely sure what the latest Thrift behavior around this is > (it's never been great),

Re: get_range_slice gets no rowcache support?

2012-11-13 Thread Tyler Hobbs
As far as I know, the row cache has never been populated by get_range_slices(), only normal gets/multigets. The behavior is this way because get_range_slices() is almost exclusively used to page over an entire column family, which generally would not fit into the cache and would simply a) ruin you

Re: composite column validation_class question

2012-11-13 Thread Tyler Hobbs
I'm not completely sure what the latest Thrift behavior around this is (it's never been great), but this is one of the things that cql3 is designed to solve well. I would consider using cql3 if that's an option for you. On Thu, Nov 8, 2012 at 11:38 PM, Wei Zhu wrote: > Any thoughts? > > Thanks

Re: remove DC

2012-11-13 Thread William Oberman
My situation is that DC2 was not written to, thanks! On Mon, Nov 12, 2012 at 7:48 PM, Jeremiah Jordan wrote: > If you have any data that you wrote to DC2, since the last time you ran > repair, you should probably run repair to make sure that data made it over > to DC1, if you never wrote data di

Re: Read during digest mismatch

2012-11-13 Thread Manu Zhang
If consistency is two, don't we just send data request to one and digest request to another? On Mon, Nov 12, 2012 at 2:49 AM, Jonathan Ellis wrote: > Correct. Which is one reason there is a separate setting for > cross-datacenter read repair, by the way. > > On Thu, Nov 8, 2012 at 4:43 PM, san

RE: CF metadata syntax for an array

2012-11-13 Thread Kevin Burton
Sorry to be so slow but I am just learning CQL. Would this syntax be the same for CREATE COLUMNFAMILY (as an aside what is a 'TABLE' in Cassandra)? In other words is this valid: CREATE COLUMNFAMILY Description ( PRI

Re: Strange delay in query

2012-11-13 Thread J. D. Jordan
Correct On Nov 13, 2012, at 5:21 AM, André Cruz wrote: > On Nov 13, 2012, at 8:54 AM, aaron morton wrote: > >>> I don't think that statement is accurate. >> Which part ? > > Probably this part: > "After running a major compaction, automatic minor compactions are no longer > triggered, freque

Re: Strange delay in query

2012-11-13 Thread André Cruz
On Nov 13, 2012, at 8:54 AM, aaron morton wrote: >> I don't think that statement is accurate. > Which part ? Probably this part: "After running a major compaction, automatic minor compactions are no longer triggered, frequently requiring you to manually run major compactions on a routine basis

Re: Memory Manager

2012-11-13 Thread Everton Lima
-Xmx2G -Xms512M -Xmn128M -Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=10.10.0.211 -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -XX:+UseConcMarkSweepGC -XX:ParallelCMSThreads=4 -XX:+CMSIncrementalMod

Re: CF metadata syntax for an array

2012-11-13 Thread aaron morton
> While this solves the problem for an array of 'primitive' types. What if I > want an array or collection of an arbitrary type like list, where foo is > a user defined type? Do you mean a custom Cassandra data type that sub classes AbstractType? I dont think CQL can support those, I may be wron

Re: removing SSTABLEs

2012-11-13 Thread aaron morton
You can also kick off a user defined compaction via JMX. https://github.com/apache/cassandra/blob/cassandra-1.1/src/java/org/apache/cassandra/db/compaction/CompactionManagerMBean.java#L58 This will allow you to compact big files, medium files, and teeny tinny little files. Cheers

Re: Strange delay in query

2012-11-13 Thread aaron morton
> I don't think that statement is accurate. Which part ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 13/11/2012, at 6:31 AM, Binh Nguyen wrote: > I don't think that statement is accurate. The minor compaction is still > triggered for