Re: RE Ordering counters in Cassandra

[samal]

In some cases Cassandra is really good and in some cases it is not.

The way I see your approach is your are recording all of your events in
single "key" is it? Not recommended. It can go really big also if your have
cluster of servers, "It will hit only one server all the time make it
overwhelm, and rest will sit ideal, take a nap.

I will do like, I will figure out what are similar events that is
occurring, and then bucket by those event.

eg: if event is occurred from IOS or Andriod. I will bucket by IOS and
android KEY, so here counter will give me all events occurred from  IOS or
andriod.

KEY, concat can also be use to filter out more deep: IOS#safari,
andriod#chrome.

Less number of columns will help to reverse index more efficiently.

/Samal

On Mon, May 21, 2012 at 11:53 PM, Tamar Fraenkel wrote:

> Indeed I took the not delete approach. If time bucket rows are not that
> big, this is a good temporary solution.
> I just finished implementation and testing now on a small staging
> environment. So far so good.
> Tamar
>
> Sent from my iPod
>
> On May 21, 2012, at 9:11 PM, Filippo Diotalevi  wrote:
>
>  Hi Tamar,
> the solution you propose is indeed a "temporary solution", but it might be
> the best one.
>
> Which approach did you follow?
> I'm a bit concerned about the deletion approach, since in case of
> concurrent writes on the same counter you might "lose" the pointer to the
> column to delete.
>
> --
> Filippo Diotalevi
>
>
> On Monday, 21 May 2012 at 18:51, Tamar Fraenkel wrote:
>
> I also had a similar problem. I have a temporary solution, which is not
> best, but may be of help.
> I have the coutner cf to count events, but apart from that I hold leaders
> CF:
>
> leaders = {
>   // key is time bucket
>   // values are composites(rank, event) ordered by
>   // descending order of the rank
>   // set relevant TTL on columns
>   time_bucket1 : {
> composite(1000,event1) : ""
> composite(999, event2) : ""
>   },
>   ...
> }
>
> Whenever I increment counter for a specific event, I add a column in the
> time bucket row of the leaders CF, with the new value of the counter and
> the event name.
> There are two ways to go here, either delete the old column(s) for that
> event (with lower counters) from leaders CF. Or let them be.
> If you choose to delete, there is the complication of not having getAndSetfor 
> counters, so you may end up not deleting all the old columns.
> If you choose not to  delete old column, and live with duplicate columns
> for events (each with different count), it will make your query to
> retrieve leaders run longer.
> Anyway, when you need to retrieve the leaders, you can do slice query 
> onleaders CF and ignore
> duplicates events using client (I use Java). This will happen less if you
> do delete old columns.
>
> Another option is not to use Cassandra for that purpose, http://redis.io/ is
> a nice tool
>
> Will be happy to hear you comments.
> Thanks,
>
> *Tamar Fraenkel *
> Senior Software Engineer, TOK Media
>
> 
>
>
> ta...@tok-media.com
> Tel:   +972 2 6409736
> Mob:  +972 54 8356490
> Fax:   +972 2 5612956
>
>
>
>
>
> On Mon, May 21, 2012 at 8:05 PM, Filippo Diotalevi wrote:
>
> Hi Romain,
> thanks for your suggestion.
>
> When you say " build every day a ranking in a dedicated CF by iterating
> over events:" do you mean
> - load all the columns for the specified row key
> - iterate over each column, and write a new column in the inversed index
> ?
>
> That's my current approach, but since I have many of these wide rows (1
> per day), the process is extremely slow as it involves moving an entire row
> from Cassandra to client, inverting every column, and sending the data back
> to create the inversed index.
>
> --
> Filippo Diotalevi
>
>
> On Monday, 21 May 2012 at 17:19, Romain HARDOUIN wrote:
>
>
> If I understand you've got a data model which looks like this:
>
> CF Events:
> "row1": { "event1": 1050, "event2": 1200, "event3": 830, ... }
>
> You can't query on column values but you can build every day a ranking in
> a dedicated CF by iterating over events:
>
> create column family Ranking
> with comparator = 'LongType(reversed=true)'
> ...
>
> CF Ranking:
> "rank": { 1200: "event2", 1050: "event1", 830: "event3", ... }
>
> Then you can make a "top ten" or whatever you want because counter values
> will be sorted.
>
>
> Filippo Diotalevi  a écrit sur 21/05/2012 16:59:43 :
>
> > Hi,
> > I'm trying to understand what's the best design for a simple
> > "ranking" use cases.
> > I have, in a row, a good number (10k - a few 100K) of counters; each
> > one is counting the occurrence of an event. At the end of day, I
> > want to create a ranking of the most occurred event.
> >
> > What's the best approach to perform this task?
> > The brute force approach of retrieving the row and ordering it
> > doesn't work well (the call usually goes timeout, especially is
> > Cassandra is also under load); I also don't know in advance the full
> > set of event

Re: RE Ordering counters in Cassandra

[Romain HARDOUIN]

I mean iterate over each column -- more precisly: *bunches of columns* 
using slices -- and write new columns in the inversed index.
Tamar's data model is made for real time analysis. It's maybe overdesigned 
for a daily ranking.
I agree with Samal, you should split your data across the space of tokens. 
Only CF Ranking feeding would be affected, not the "top N" queries.

Filippo Diotalevi  a écrit sur 21/05/2012 19:05:28 :

> Hi Romain, 
> thanks for your suggestion.
> 
> When you say " build every day a ranking in a dedicated CF by 
> iterating over events:" do you mean
> - load all the columns for the specified row key
> - iterate over each column, and write a new column in the inversed index
> ?
> 
> That's my current approach, but since I have many of these wide rows
> (1 per day), the process is extremely slow as it involves moving an 
> entire row from Cassandra to client, inverting every column, and 
> sending the data back to create the inversed index.

Re: nodetool repair taking forever

[aaron morton]

> I also dont understand if all these nodes are replicas of each other why is 
> that the first node has almost double the data.
Have you performed any token moves ? Old data is not deleted unless you run 
nodetool cleanup. 
Another possibility is things like a lot of hints. Admittedly it would have to 
be a *lot* of hints.
The third is that compaction has fallen behind. 

> This week its even worse, the nodetool repair has been running for the last 
> 15 hours just on the first node and when I run nodetool compactionstats I 
> constantly see this -
> 
> pending tasks: 3
First check the logs for errors. 

Repair will first calculate the differences, you can see this as a validation 
compaction in nodetool compactionstats.
Then it will stream the data, you can watch that with nodetool netstats. 

Try to work out which part is taking the most time.   15 hours for 50Gb sounds 
like a long time (btw do you have compaction on ?)

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 3:14 AM, Raj N wrote:

> Hi experts,
> 
> I have a 6 node cluster spread across 2 DCs. 
> 
> DC  RackStatus State   LoadOwnsToken
>
> 113427455640312814857969558651062452225
> DC1 RAC13   Up Normal  95.98 GB33.33%  0
> DC2 RAC5Up Normal  50.79 GB0.00%   1
> DC1 RAC18   Up Normal  50.83 GB33.33%  
> 56713727820156407428984779325531226112
> DC2 RAC7Up Normal  50.74 GB0.00%   
> 56713727820156407428984779325531226113
> DC1 RAC19   Up Normal  61.72 GB33.33%  
> 113427455640312814857969558651062452224
> DC2 RAC9Up Normal  50.83 GB0.00%   
> 113427455640312814857969558651062452225
> 
> They are all replicas of each other. All reads and writes are done at 
> LOCAL_QUORUM. We are on Cassandra 0.8.4. I see that our weekend nodetool 
> repair runs for more than 12 hours. Especially on the first one which has 96 
> GB data. Is this usual? We are using 500 GB SAS drives with ext4 file system. 
> This gets worse every week. This week its even worse, the nodetool repair has 
> been running for the last 15 hours just on the first node and when I run 
> nodetool compactionstats I constantly see this -
> 
> pending tasks: 3
> 
> and nothing else. Looks like its just stuck. There's nothing substantial in 
> the logs as well. I also dont understand if all these nodes are replicas of 
> each other why is that the first node has almost double the data. Any help 
> will be really appreciated.
> 
> Thanks
> -Raj

Re: Data aggregation - averages, sums, etc.

[aaron morton]

Continuous computation is the sort of thing Storm 
(https://github.com/nathanmarz/storm) can help with. 

And good news everybody, storing the output from Storm is the sort of thing 
Cassandra can help with http://www.youtube.com/watch?v=cF8a_FZwULI

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 6:17 AM, Janne Jalkanen wrote:

>> 2. I know I have counter columns. I can do sums. But can I do averages ?
> 
> One counter column for the sum, one counter column for the count. Divide for 
> average :-)
> 
> /Janne

Re: Repair Process Taking too long

[aaron morton]

It repairs the ranges they have in common. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 4:05 PM, Raj N wrote:

> Can I infer from this that if I have 3 replicas, then running repair without 
> -pr won 1 node will repair the other 2 replicas as well.
> 
> -Raj
> 
> On Sat, Apr 14, 2012 at 2:54 AM, Zhu Han  wrote:
> 
> On Sat, Apr 14, 2012 at 1:57 PM, Igor  wrote:
> Hi!
> 
> What is the difference between 'repair' and '-pr repair'? Simple repair touch 
> all token ranges (for all nodes) and -pr touch only range for which given 
> node responsible?
> 
> 
> -pr only touches the primary range of the node.  If you executes -pr against 
> all nodes in replica groups,  then all ranges are repaired.
> 
> 
> On 04/12/2012 05:59 PM, Sylvain Lebresne wrote:
> On Thu, Apr 12, 2012 at 4:06 PM, Frank Ng  wrote:
> I also noticed that if I use the -pr option, the repair process went down
> from 30 hours to 9 hours.  Is the -pr option safe to use if I want to run
> repair processes in parallel on nodes that are not replication peers?
> There is pretty much two use case for repair:
> 1) to rebuild a node: if say a node has lost some data due to a hard
> drive corruption or the like and you want to to rebuild what's missing
> 2) the periodic repairs to avoid problem with deleted data coming back
> from the dead (basically:
> http://wiki.apache.org/cassandra/Operations#Frequency_of_nodetool_repair)
> 
> In case 1) you want to run 'nodetool repair' (without -pr) against the
> node to rebuild.
> In case 2) (which I suspect is the case your talking now), you *want*
> to use 'nodetool repair -pr' on *every* node of the cluster. I.e.
> that's the most efficient way to do it. The only reason not to use -pr
> in this case would be that it's not available because you're using an
> old version of Cassandra. And yes, it's is safe to run with -pr in
> parallel on nodes that are not replication peers.
> 
> --
> Sylvain
> 
> 
> thanks
> 
> 
> On Thu, Apr 12, 2012 at 12:06 AM, Frank Ng  wrote:
> Thank you for confirming that the per node data size is most likely
> causing the long repair process.  I have tried a repair on smaller column
> families and it was significantly faster.
> 
> On Wed, Apr 11, 2012 at 9:55 PM, aaron morton
> wrote:
> If you have 1TB of data it will take a long time to repair. Every bit of
> data has to be read and a hash generated. This is one of the reasons we
> often suggest that around 300 to 400Gb per node is a good load in the
> general case.
> 
> Look at nodetool compactionstats .Is there a validation compaction
> running ? If so it is still building the merkle  hash tree.
> 
> Look at nodetool netstats . Is it streaming data ? If so all hash trees
> have been calculated.
> 
> Cheers
> 
> 
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 12/04/2012, at 2:16 AM, Frank Ng wrote:
> 
> Can you expand further on your issue? Were you using Random Patitioner?
> 
> thanks
> 
> On Tue, Apr 10, 2012 at 5:35 PM, David Leimbach
> wrote:
> I had this happen when I had really poorly generated tokens for the
> ring.  Cassandra seems to accept numbers that are too big.  You get hot
> spots when you think you should be balanced and repair never ends (I think
> there is a 48 hour timeout).
> 
> 
> On Tuesday, April 10, 2012, Frank Ng wrote:
> I am not using tier-sized compaction.
> 
> 
> On Tue, Apr 10, 2012 at 12:56 PM, Jonathan Rhone
> wrote:
> Data size, number of nodes, RF?
> 
> Are you using size-tiered compaction on any of the column families
> that hold a lot of your data?
> 
> Do your cassandra logs say you are streaming a lot of ranges?
> zgrep -E "(Performing streaming repair|out of sync)"
> 
> 
> On Tue, Apr 10, 2012 at 9:45 AM, Igor  wrote:
> On 04/10/2012 07:16 PM, Frank Ng wrote:
> 
> Short answer - yes.
> But you are asking wrong question.
> 
> 
> I think both processes are taking a while.  When it starts up,
> netstats and compactionstats show nothing.  Anyone out there successfully
> using ext3 and their repair processes are faster than this?
> 
> On Tue, Apr 10, 2012 at 10:42 AM, Igor  wrote:
> Hi
> 
> You can check with nodetool  which part of repair process is slow -
> network streams or verify compactions. use nodetool netstats or
> compactionstats.
> 
> 
> On 04/10/2012 05:16 PM, Frank Ng wrote:
> Hello,
> 
> I am on Cassandra 1.0.7.  My repair processes are taking over 30
> hours to complete.  Is it normal for the repair process to take this long?
>  I wonder if it's because I am using the ext3 file system.
> 
> thanks
> 
> 
> 
> 
> --
> Jonathan Rhone
> Software Engineer
> 
> TinyCo
> 800 Market St., Fl 6
> San Francisco, CA 94102
> www.tinyco.com
> 
> 
> 
> 
> 

Re: unable to nodetool to remote EC2

[ramesh]

  
  
On 05/22/2012 12:45 AM, Tamar Fraenkel wrote:

  
  Thanks for the response. But it still does not work.

  I am running the script from a git bash on my windows 7.
  adding some debug prints, this is what I am
running
  
ssh -i key.pem -N -f -D8123 ubuntu@ec2-*.amazonaws.com
ssh pid = 11616
/c/PROGRA~2/Java/jdk1.7.0_02/bin/jconsole.exe
  -J-DsocksProxyHost=localhost -J-DsocksProxyPort=8123
  service:jmx:rmi:///jndi/rmi://ec2-*.amazonaws.com:7199/jmxrmi

  

Still getting "channel 2: open failed: connect failed:
  Connection timed out"
Any further idea? Where are you running the script.
Thanks



  
Tamar Fraenkel 
Senior Software Engineer, TOK Media 


  
ta...@tok-media.com
Tel:   +972 2 6409736 
Mob:  +972 54 8356490 
Fax:   +972 2 5612956 


  
  




On Mon, May 21, 2012 at 11:00 PM, ramesh 
  wrote:
  

  
 On 05/21/2012 03:55 AM, Tamar Fraenkel wrote:
  
Hi!
  I am trying the tunnel and it fails. Will
be gratefull for
some hints:
  
  
  I defined
  

  
proxy_host =
  ubuntu@my_ec2_cassandra_node_public_ip
proxy_port = 22
  
  I do:

ssh -N -f -i
/c/Users/tamar/.ssh/Amazon/tokey.openssh
-D22
ubuntu@my_ec2_cassandra_node_public_ip


 I put some debug prints and I can see
  that the ssh_pid is indeed the correct
  one.

  
I run
jconsole
-J-DsocksProxyHost=localhost
-J-DsocksProxyPort=22
service:jmx:rmi:///jndi/rmi://my_ec2_cassandra_node_public_ip:7199/jmxrmi


I get errors and it fails:
channel 2:
  open failed: connect failed: Connection
  timed out
 

One note though, I can ssh to that vm
  using 
ssh -i
  /c/Users/tamar/.ssh/Amazon/tokey.openssh
  -D22
  ubuntu@my_ec2_cassandra_node_public_ip
without being prompted for
  PW.


Any help appreciated



   Tamar Fraenkel 
Senior Software Engineer, TOK Media 


  
ta...@tok-media.com
Tel:   +972
  2 6409736 
Mob:  +972
  54 8356490 
Fax:   +972
  2 5612956 


  
  




On Fri, May 18, 2012 at 9:49 PM, ramesh
  
  wrote:
  

   On 05/18/2012 01:35 PM, Tyler
Hobbs wrote: 
   

Re: cassandra read latency help

[aaron morton]

With

> heap size = 4 gigs

I would check for GC activity in the logs and consider setting it to 8 given 
you have 16 GB.  You can also check if the IO system is saturated 
(http://spyced.blogspot.co.nz/2010/01/linux-performance-basics.html) Also take 
a look at nodetool cfhistogram perhaps to see how many sstables are involved. 


I would start by looking at the latency reported on the server, then work back 
to the client….

I may have missed it in the email but what recent latency for the CF is 
reported by nodetool cfstats ? That's latency for a single request on a single 
read thread. The default settings give you 32 read threads. 

If you know the latency for a single request, and you know you have 32 
concurrent read threads, you can get an idea of the max throughput for a single 
node. Once you get above that throughput the latency for a request will start 
to include wait time. 

It's a bit more complicated, because when you request 40 rows that turns into 
40 read tasks. So if two clients send a request for 40 rows at the same time 
there will be 80 read tasks to be processed by 32 threads. 

Hope that helps. 

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 4:10 PM, Radim Kolar wrote:

> Dne 19.5.2012 0:09, Gurpreet Singh napsal(a):
>> Thanks Radim.
>> Radim, actually 100 reads per second is achievable even with 2 disks.
> it will become worse as rows will get fragmented.
>> But achieving them with a really low avg latency per key is the issue.
>> 
>> I am wondering if anyone has played with index_interval, and how much of a 
>> difference would it make to reads on reducing the index_interval.
> close to zero. but try it yourself too and post your findings.

Re: Does Cassandra support parallel query processing?

[aaron morton]

In general read queries run on multiple nodes. But each node computes the 
complete result to the query. 

There is no support for aggregate queries. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 6:49 PM, Majid Azimi wrote:

> hi guys,
> 
> I'm going to build a warehouse with Cassandra. There are a lot of range and 
> aggregate queries. 
> Does Cassandra support parallel query processing?(both on single box and 
> cluster)

Re: exception when cleaning up...

[aaron morton]

CASSANDRA-3712 has not been applied to 0.8.X.  

If I understand the problem correctly the issue is 0.8.10. 

You may be able to avoid the race condition by:

1) Isolating the node from the cluster to stop write activity. You can either 
start the node with the -Dcassandra.join_ring=false  JVM option or use nodetool 
disablethrift and disablegossip to stop writes. Note that this will not stop 
existing Hinted Handoff sessions which target the node. 

2) Flushing all memtables with nodetool flush 

3) Running nodetool cleanup

Hope that helps. 
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/05/2012, at 10:20 PM, Boris Yen wrote:

> Hi,
> 
> We are currently running 0.8.10 with 4 nodes. We tried to re-balance the 
> range each node owns by using "nodetool move". After moving the node to the 
> assigned token, we run the cleanup command, then we saw the exceptions:
> 
>  Error occured during cleanup
> java.util.concurrent.ExecutionException: java.lang.AssertionError
> 
> at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
> 
> at java.util.concurrent.FutureTask.get(Unknown Source)
> 
> at 
> org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:226)
> 
> at 
> org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1073)
> 
> at 
> org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1423)
> 
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 
> at java.lang.reflect.Method.invoke(Unknown Source)
> 
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> 
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> 
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> 
> at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
> 
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
> 
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown 
> Source)
> 
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
> 
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> 
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
> 
> at 
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown 
> Source)
> 
> at 
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown 
> Source)
> 
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source)
> 
> at sun.reflect.GeneratedMethodAccessor236.invoke(Unknown Source)
> 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 
> at java.lang.reflect.Method.invoke(Unknown Source)
> 
> at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> 
> at sun.rmi.transport.Transport$1.run(Unknown Source)
> 
> at java.security.AccessController.doPrivileged(Native Method)
> 
> at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> 
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> 
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> 
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> 
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 
> at java.lang.Thread.run(Unknown Source)
> 
> Caused by: java.lang.AssertionError
> 
> at 
> org.apache.cassandra.db.ColumnFamilyStore.maybeSwitchMemtable(ColumnFamilyStore.java:762)
> 
> at 
> org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:844)
> 
> at 
> org.apache.cassandra.db.ColumnFamilyStore.forceBlockingFlush(ColumnFamilyStore.java:849)
> 
> at 
> org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:928)
> 
> at 
> org.apache.cassandra.db.compaction.CompactionManager.access$500(CompactionManager.java:66)
> 
> at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:205)
> 
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> 
> at java.util.concurrent.FutureTask.run(Unknown Source)
> 
> ... 3 more
> 
> 
> 
> It seems to be relative to CASSANDRA-3712. However, it only got fixed on 1.0. 
> I wonder if this will also be fixed on 0.8? One other question, if we do not 
> run the cleanup, is there any other way that the unnecessary data can get 
> cleaned?
> 
> 
> 
> Regards.
> 
> Boris
> 

Re: Exception when truncate

[aaron morton]

The first part of the name is the current system time in milliseconds. 

If you run it twice do you get log messages about failing to create the same 
directory twice ?

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 21/05/2012, at 5:09 AM, ruslan usifov wrote:

> I think as you, but this is not true, there are not any permissions
> issue. And as i said before, cassandra try to create directory for
> snapshort that already exists
> 
> 2012/5/19 Jonathan Ellis :
>> Sounds like you have a permissions problem.  Cassandra creates a
>> subdirectory for each snapshot.
>> 
>> On Thu, May 17, 2012 at 4:57 AM, ruslan usifov  
>> wrote:
>>> Hello
>>> 
>>> I have follow situation on our test server:
>>> 
>>> from cassandra-cli i try to use
>>> 
>>> truncate purchase_history;
>>> 
>>> 3 times i got:
>>> 
>>> [default@township_6waves] truncate purchase_history;
>>> null
>>> UnavailableException()
>>>at 
>>> org.apache.cassandra.thrift.Cassandra$truncate_result.read(Cassandra.java:20212)
>>>at 
>>> org.apache.cassandra.thrift.Cassandra$Client.recv_truncate(Cassandra.java:1077)
>>>at 
>>> org.apache.cassandra.thrift.Cassandra$Client.truncate(Cassandra.java:1052)
>>>at 
>>> org.apache.cassandra.cli.CliClient.executeTruncate(CliClient.java:1445)
>>>at 
>>> org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:272)
>>>at 
>>> org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:220)
>>>at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)
>>> 
>>> 
>>> So this looks that truncate goes very slow and too long, than
>>> rpc_timeout_in_ms: 1 (this can happens because we have very slow
>>> disck on test machine)
>>> 
>>> But in in cassandra system log i see follow exception:
>>> 
>>> 
>>> ERROR [MutationStage:7022] 2012-05-17 12:19:14,356
>>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>>> Thread[MutationStage:7022,5,main]
>>> java.io.IOError: java.io.IOException: unable to mkdirs
>>> /home/cassandra/1.0.0/data/township_6waves/snapshots/1337242754356-purchase_history
>>>at 
>>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1433)
>>>at 
>>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1462)
>>>at 
>>> org.apache.cassandra.db.ColumnFamilyStore.truncate(ColumnFamilyStore.java:1657)
>>>at 
>>> org.apache.cassandra.db.TruncateVerbHandler.doVerb(TruncateVerbHandler.java:50)
>>>at 
>>> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>>>at 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>at 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.io.IOException: unable to mkdirs
>>> /home/cassandra/1.0.0/data/township_6waves/snapshots/1337242754356-purchase_history
>>>at 
>>> org.apache.cassandra.io.util.FileUtils.createDirectory(FileUtils.java:140)
>>>at 
>>> org.apache.cassandra.io.util.FileUtils.createDirectory(FileUtils.java:131)
>>>at 
>>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1409)
>>>... 7 more
>>> 
>>> 
>>> Also i see that in snapshort dir already exists
>>> 1337242754356-purchase_history directory, so i think that snapshort
>>> names that generate cassandra not uniquely.
>>> 
>>> PS: We use cassandra 1.0.10 on Ubuntu 10.0.4-LTS
>> 
>> 
>> 
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com

Re: endless hinted handoff with 1.1

[aaron morton]

kinds of like  https://issues.apache.org/jira/browse/CASSANDRA-3733 but maybe 
different. 

Have you recently dropped as CF ? it looks like the hints CF is only compacted 
if hints are replayed. If they are dropped because the CF no longer exists 
compaction is not forced ( 
https://github.com/apache/cassandra/blob/cassandra-1.1/src/java/org/apache/cassandra/db/HintedHandOffManager.java#L370
 ) 

Can you create a ticket on https://issues.apache.org/jira/browse/CASSANDRA with 
the details and update the thread. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 21/05/2012, at 8:36 PM, Arend-Jan Wijtzes wrote:

> Hi,
> 
> We are running a small test cluster and recently installed Cassandra 1.1 and
> started with a new clean database. We keep seeing these messages in the log 
> on just one of our nodes:
> 
> INFO [HintedHandoff:1] 2012-05-21 09:49:56,757 HintedHandOffManager.java 
> (line 294) Started hinted handoff for token: 
> 85070591730234615865843651857942052864 with IP: /10.0.0.73
> INFO [HintedHandoff:1] 2012-05-21 09:49:56,775 HintedHandOffManager.java 
> (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73
> INFO [HintedHandoff:1] 2012-05-21 09:59:56,756 HintedHandOffManager.java 
> (line 294) Started hinted handoff for token: 
> 42535295865117307932921825928971026432 with IP: /10.0.0.69
> INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java 
> (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.69
> INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java 
> (line 294) Started hinted handoff for token: 
> 85070591730234615865843651857942052864 with IP: /10.0.0.73
> INFO [HintedHandoff:1] 2012-05-21 09:59:56,775 HintedHandOffManager.java 
> (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73
> INFO [HintedHandoff:1] 2012-05-21 10:09:56,757 HintedHandOffManager.java 
> (line 294) Started hinted handoff for token: 
> 42535295865117307932921825928971026432 with IP: /10.0.0.69
> INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java 
> (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.69
> INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java 
> (line 294) Started hinted handoff for token: 
> 85070591730234615865843651857942052864 with IP: /10.0.0.73
> INFO [HintedHandoff:1] 2012-05-21 10:09:56,879 HintedHandOffManager.java 
> (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73
> 
> 
> All four nodes are up:
> 
> -bash-4.1$ nodetool ring -h localhost
> Note: Ownership information does not include topology, please specify a 
> keyspace. 
> Address DC  RackStatus State   LoadOwns   
>  Token   
>   
> 127605887595351923798765477786913079296 
> 10.0.0.65   datacenter1 rack1   Up Normal  244.41 MB   25.00% 
>  0   
> 10.0.0.69   datacenter1 rack1   Up Normal  155.39 MB   25.00% 
>  42535295865117307932921825928971026432  
> 10.0.0.73   datacenter1 rack1   Up Normal  220.42 MB   25.00% 
>  85070591730234615865843651857942052864  
> 10.0.0.77   datacenter1 rack1   Up Normal  296.14 MB   25.00% 
>  127605887595351923798765477786913079296 
> 
> 
> This has been going on for days. Note that it's just two key's in the log
> that keep repeating. No recent messages about HintedHandOff in the logs 
> on the other nodes.
> 
> Let me know if you need more info.
> 
> Arend-Jan
> 
> -- 
> Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl

Re: Tuning cassandra (compactions overall)

[aaron morton]

not sure what you mean by 
> And after restarting the second one I have lost all the consistency of
> my data. All my statistics since September are totally false now in
> production

Can you give some examples?
Counter are not idempotent so if the client app retries TimedOut requests you 
can get an over count. That should not result in lost data.

> As reminder I'm using a 2 node cluster RF=2, CL.ONE

Have you been running repair ? 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 1:27 AM, Alain RODRIGUEZ wrote:

> Hi Aaron.
> 
> I wanted to try the new config. After doing a rolling restart I have
> all my counters false, with wrong values. I stopped my servers with
> the following :
> 
> nodetool -h localhost disablegossip
> nodetool -h localhost disablethrift
> nodetool -h localhost drain
> kill cassandra sigterm (15) via htop
> 
> And after restarting the second one I have lost all the consistency of
> my data. All my statistics since September are totally false now in
> production.
> 
> As reminder I'm using a 2 node cluster RF=2, CL.ONE
> 
> 1 - How to fix it ? (I have a backup from this morning, but I will
> lose all the data after this date if I restore this backup)
> 2 - What happened ? How to avoid it ?
> 
> Any Idea would be greatly appreciated, I'm quite desperated.
> 
> Alain
> 
> 2012/5/17 aaron morton :
>> What is the the benefit of having more memory ? I mean, I don't
>> 
>> understand why having 1, 2, 4, 8 or 16 GB of memory is so different.
>> 
>> Less frequent and less aggressive garbage collection frees up CPU resources
>> to run the database.
>> 
>> Less memory results in frequent and aggressive (i.e. stop the world) GC, and
>> increase IO pressure. Which reduces read performance and in the extreme can
>> block writes.
>> 
>> The memory used inside
>> 
>> the heap will remains close to the max memory available, therefore
>> having more or less memory doesn't matter.
>> 
>> Not an ideal situation. Becomes difficult to find an contiguous region of
>> memory to allocate.
>> 
>> Can you enlighten me about this point ?
>> 
>> It's a database server, it is going to work better with more memory. Also
>> it's Java and it's designed to run on multiple machines with many GB's of
>> ram available. There are better arguments
>> here http://wiki.apache.org/cassandra/CassandraHardware
>> 
>> 
>> I'm interested a lot in learning about some configuration I can use to
>> reach better peformance/stability as well as in learning about how
>> Cassandra works.
>> 
>> Turn off all caches.
>> 
>> In the schema increase the bloom filter false positive rate (see help in the
>> cli for Create column family)
>> 
>> In the yaml experiment with these changes:
>> * reduce sliced_buffer_size_in_kb
>> * reduce column_index_size_in_kb
>> * reduce in_memory_compaction_limit_in_mb
>> * increase index_interval
>> * set concurrent_compactors to 2
>> 
>> Cheers
>> 
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> 
>> On 17/05/2012, at 12:40 AM, Alain RODRIGUEZ wrote:
>> 
>> Using c1.medium, we are currently able to deliver the service.
>> 
>> What is the the benefit of having more memory ? I mean, I don't
>> understand why having 1, 2, 4, 8 or 16 GB of memory is so different.
>> In my mind, Cassandra will fill the heap and from then, start to flush
>> and compact to avoid OOMing and fill it again. The memory used inside
>> the heap will remains close to the max memory available, therefore
>> having more or less memory doesn't matter.
>> 
>> I'm pretty sure I misunderstand or forget something about how the
>> memory is used but not sure about what.
>> 
>> Can you enlighten me about this point ?
>> 
>> If I understand why the memory size is that important I will probably
>> be able to argue about the importance of having more memory and my
>> boss will probably allow me to spend more money to get better servers.
>> 
>> "There are some changes you can make to mitigate things (let me know
>> if you need help), but this is essentially a memory problem."
>> 
>> I'm interested a lot in learning about some configuration I can use to
>> reach better peformance/stability as well as in learning about how
>> Cassandra works.
>> 
>> Thanks for the help you give to people and for sharing your knowledge
>> with us. I appreciate a lot the Cassandra community and the most
>> active people keeping it alive. It's worth being said :).
>> 
>> Alain
>> 
>> 

Re: Ordering counters in Cassandra

[aaron morton]

> What's the best approach to perform this task? 
get the columns in slices of a 100 or so and order on the client. Then write a 
new row that is a pivot so the column name is the aggregate count and the 
column value is the old column name. 

To slice the row, make the first call with no start_col, then use the last col 
read as the next start col. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 2:59 AM, Filippo Diotalevi wrote:

> Hi,
> I'm trying to understand what's the best design for a simple "ranking" use 
> cases.
> I have, in a row, a good number (10k - a few 100K) of counters; each one is 
> counting the occurrence of an event. At the end of day, I want to create a 
> ranking of the most occurred event.
> 
> What's the best approach to perform this task? 
> The brute force approach of retrieving the row and ordering it doesn't work 
> well (the call usually goes timeout, especially is Cassandra is also under 
> load); I also don't know in advance the full set of event names (column 
> names), so it's difficult to slice the get call.
> 
> Is there any trick to solve this problem? Maybe a way to retrieve the row 
> ordering for counter values?
> 
> Thanks,
> -- 
> Filippo Diotalevi
> 
> 

Cassandra 0.8.5: Column name mystery in create column family command

[Roshan Dawrani]

Hi,

I use Cassandra 0.8.5 and am suddenly noticing some strange behavior. I run
a "create column family" command with some column meta-data and it runs
fine, but when I do "describe keyspace", it shows me different column names
for those index columns.

a) Here is what I run:
"create column family UserTemplate with comparator=BytesType and
column_metadata=[{*column_name: userid*, validation_class: UTF8Type,
index_type: KEYS, index_name: TemplateUserIdIdx}, {*column_name: type*,
validation_class: UTF8Type, index_type: KEYS, index_name:
TemplateTypeIdx}];"

b) This is what "describe keyspace" shows:
ColumnFamily: UserTemplate
  Key Validation Class: org.apache.cassandra.db.marshal.BytesType
  ...
  ...
  Built indexes: [UserTemplate.TemplateTypeIdx,
UserTemplate.TemplateUserIdIdx]
  Column Metadata:
*Column Name: ff*
  Validation Class: org.apache.cassandra.db.marshal.UTF8Type
  Index Name: TemplateUserIdIdx
  Index Type: KEYS
*Column Name: 0dfffaff*
  Validation Class: org.apache.cassandra.db.marshal.UTF8Type
  Index Name: TemplateTypeIdx
  Index Type: KEYS

Does anyone see why this must be happening? I have created many such column
families before and never run into this issue.

-- 
Roshan
http://roshandawrani.wordpress.com/

Re: 1.1 not removing commit log files?

[aaron morton]

4096 is also the internal hard coded default for commitlog_total_space_in_mb

If you are seeing more that 4GB of commit log files let us know. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 6:35 AM, Bryce Godfrey wrote:

> Thanks, I'll give it a try.
> 
> -Original Message-
> From: Alain RODRIGUEZ [mailto:arodr...@gmail.com] 
> Sent: Monday, May 21, 2012 2:12 AM
> To: user@cassandra.apache.org
> Subject: Re: 1.1 not removing commit log files?
> 
> commitlog_total_space_in_mb: 4096
> 
> By default this line is commented in 1.0.x if I remember well. I guess it is 
> the same in 1.1. You really should remove this comment or your commit logs 
> will entirely fill up your disk as it happened to me a while ago.
> 
> Alain
> 
> 2012/5/21 Pieter Callewaert :
>> Hi,
>> 
>> 
>> 
>> In 1.1 the commitlog files are pre-allocated with files of 128MB.
>> (https://issues.apache.org/jira/browse/CASSANDRA-3411) This should 
>> however not exceed your commitlog size in Cassandra.yaml.
>> 
>> 
>> 
>> commitlog_total_space_in_mb: 4096
>> 
>> 
>> 
>> Kind regards,
>> 
>> Pieter Callewaert
>> 
>> 
>> 
>> From: Bryce Godfrey [mailto:bryce.godf...@azaleos.com]
>> Sent: maandag 21 mei 2012 9:52
>> To: user@cassandra.apache.org
>> Subject: 1.1 not removing commit log files?
>> 
>> 
>> 
>> The commit log drives on my nodes keep slowly filling up.  I don't see 
>> any errors in my logs that are indicating any issues that I can map to 
>> this issue.
>> 
>> 
>> 
>> Is this how 1.1 is supposed to work now?  Previous versions seemed to 
>> keep this drive at a minimum as it flushed.
>> 
>> 
>> 
>> /dev/mapper/mpathf 25G   21G  4.2G  83% /opt/cassandra/commitlog
>> 
>> 

Cassandra 0.8.5: Column name mystery in create column family command

[Roshan Dawrani]

Hi,

I use Cassandra 0.8.5 and am suddenly noticing some strange behavior. I run
a "create column family" command with some column meta-data and it runs
fine, but when I do "describe keyspace", it shows me different column names
for those index columns.

a) Here is what I run:
"create column family UserTemplate with comparator=BytesType and
column_metadata=[{*column_name: userid*, validation_class: UTF8Type,
index_type: KEYS, index_name: TemplateUserIdIdx}, {*column_name: type*,
validation_class: UTF8Type, index_type: KEYS, index_name:
TemplateTypeIdx}];"

b) This is what "describe keyspace" shows:
ColumnFamily: UserTemplate
  Key Validation Class: org.apache.cassandra.db.marshal.BytesType
  ...
  ...
  Built indexes: [UserTemplate.TemplateTypeIdx,
UserTemplate.TemplateUserIdIdx]
  Column Metadata:
*Column Name: ff*
  Validation Class: org.apache.cassandra.db.marshal.UTF8Type
  Index Name: TemplateUserIdIdx
  Index Type: KEYS
*Column Name: 0dfffaff*
  Validation Class: org.apache.cassandra.db.marshal.UTF8Type
  Index Name: TemplateTypeIdx
  Index Type: KEYS

Does anyone see why this must be happening? I have created many such column
families before and never run into this issue.

-- 
Roshan
http://roshandawrani.wordpress.com/

Re: how can we get (a lot) more performance from cassandra

[aaron morton]

> I would look into the problems you are having with GC...


When ParNew runs the jvm pauses 
https://blogs.oracle.com/jonthecollector/entry/our_collectors . If it's pausing 
for 4 seconds it's not processing queries. 

> Then check the throughput on the san and the steal on the VM's.


Check to see if the IO system is overloaded / has a long latencies see 
http://spyced.blogspot.co.nz/2010/01/linux-performance-basics.html

Steal time is the time your virtual cpu is waiting to get the physical cpu, you 
can see it in top (http://manpages.ubuntu.com/manpages/lucid/man1/top.1.html) 
and vmstat

> Also try to isolate the issue to "it takes this long for a single thread to 
> make this call"
Things like "3-4 MB/sec" don't explain what you are doing. If you say "we are 
making this sort of query, on a single thread, and it takes this long or we 
only get this many a second" it's easier. 

Cheers


-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 8:15 AM, Yiming Sun wrote:

> Hi Aaron,
> 
> I don't know if you could elaborate a bit more on each of the points you 
> suggested.  Thanks.
> 
> -- Y.
> 
> On Sun, May 20, 2012 at 7:29 PM, aaron morton  wrote:
> I would look into the problems you are having with GC...
> 
>> The server log shows the GC ParNew frequently gets longer than 200ms, often 
>> in the range of 4-5seconds.  But nowhere near 15 seconds (which is an 
>> indication that JVM heap is being swapped out).
> 
> Then check the throughput on the san and the steal on the VM's.
> 
> Also try to isolate the issue to "it takes this long for a single thread to 
> make this call"
> 
> In a low write environment reads should be flying along. 
> 
> Cheers
> 
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 17/05/2012, at 1:44 PM, Yiming Sun wrote:
> 
>> Hi Aaron T.,  No, actually we haven't, but this sounds like a good 
>> suggestion.  I can definitely try THIS before jumping into other things such 
>> as enabling row cache etc. Thanks!
>> 
>> -- Y.
>> 
>> On Wed, May 16, 2012 at 9:38 PM, Aaron Turner  wrote:
>> On Wed, May 16, 2012 at 12:59 PM, Yiming Sun  wrote:
>> > Hello,
>> >
>> > I asked the question as a follow-up under a different thread, so I figure I
>> > should ask here instead in case the other one gets buried, and besides, I
>> > have a little more information.
>> >
>> > "We find the lack of performance disturbing" as we are only able to get
>> > about 3-4MB/sec read performance out of Cassandra.
>> >
>> > We are using cassandra as the backend for an IR repository of digital 
>> > texts.
>> > It is a read-mostly repository with occasional writes.  Each row represents
>> > a book volume, and each column of a row represents a page of the volume.
>> >  Granted the data size is small -- the average size of a column text is
>> > 2-3KB, and each row has about 250 columns (varies quite a bit from one
>> > volume to another).
>> >
>> > Currently we are running a 3-node cluster, and will soon be upgraded to a
>> > 6-node setup.  Each node is a VM with 4 cores and 16GB of memory.  All VMs
>> > use SAN as disk storage.
>> >
>> > To retrieve a volume, a slice query is used via Hector that specifies the
>> > row key (the volume), and a list of column keys (pages), and the 
>> > consistency
>> > level is set to ONE.  It is typical to retrieve multiple volumes per
>> > request.
>> >
>> > The read rate that I have been seeing is about 3-4 MB/sec, and that is
>> > reading the raw bytes... using string serializer the rate is even lower,
>> > about 2.2MB/sec.
>> >
>> > The server log shows the GC ParNew frequently gets longer than 200ms, often
>> > in the range of 4-5seconds.  But nowhere near 15 seconds (which is an
>> > indication that JVM heap is being swapped out).
>> >
>> > Currently we have not added JNA.  From a blog post, it seems JNA is able to
>> > increase the performance by 13%, and we are hoping to increase the
>> > performance by something more like 1300% (3-4 MB/sec is just disturbingly
>> > low).  And we are hesitant to disable swap entirely since one of the nodes
>> > is running a couple other services
>> >
>> > Do you have any suggestions on how we may boost the performance?  Thanks!
>> 
>> Have you tried using more threads on the client side?  Generally
>> speaking, when I need faster read/write performance I look for ways to
>> parallelize my requests and it scales pretty much linearly.
>> 
>> 
>> --
>> Aaron Turner
>> http://synfin.net/ Twitter: @synfinatic
>> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & 
>> Windows
>> Those who would give up essential Liberty, to purchase a little temporary
>> Safety, deserve neither Liberty nor Safety.
>> -- Benjamin Franklin
>> "carpe diem quam minimum credula postero"
>> 
> 
> 

Re: Number of keyspaces

[aaron morton]

It's more the number of CF's than keyspaces.

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 6:58 PM, R. Verlangen wrote:

> Yes, it does. However there's no real answer what's the limit: it depends on 
> your hardware and cluster configuration. 
> 
> You might even want to search the archives of this mailinglist, I remember 
> this has been asked before.
> 
> Cheers!
> 
> 2012/5/21 Luís Ferreira 
> Hi,
> 
> Does the number of keyspaces affect the overall cassandra performance?
> 
> 
> Cumprimentos,
> Luís Ferreira
> 
> 
> 
> 
> 
> 
> -- 
> With kind regards,
> 
> Robin Verlangen
> www.robinverlangen.nl
> 

Re: Number of keyspaces

[R. Verlangen]

Hmm, you got me on that. I assumed (~ wrong) that more keyspaces would mean
more CF's.

2012/5/22 aaron morton 

> It's more the number of CF's than keyspaces.
>
> Cheers
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
>
> Yes, it does. However there's no real answer what's the limit: it depends
> on your hardware and cluster configuration.
>
> You might even want to search the archives of this mailinglist, I remember
> this has been asked before.
>
> Cheers!
>
> 2012/5/21 Luís Ferreira 
>
>> Hi,
>>
>> Does the number of keyspaces affect the overall cassandra performance?
>>
>>
>> Cumprimentos,
>> Luís Ferreira
>>
>>
>>
>>
>
>
> --
> With kind regards,
>
> Robin Verlangen
> www.robinverlangen.nl
>
>
>


-- 
With kind regards,

Robin Verlangen
www.robinverlangen.nl

Re: Number of keyspaces

[Franc Carter]

On Tue, May 22, 2012 at 9:19 PM, aaron morton wrote:

> It's more the number of CF's than keyspaces.
>

Oh - does increasing the number of Column Families affect performance ?

The design we are working on at the moment is considering using a Column
Family per year. We were thinking this would isolate compactions to a more
manageable size as we don't update previous years.

cheers


>
> Cheers
>
>   -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
>
> Yes, it does. However there's no real answer what's the limit: it depends
> on your hardware and cluster configuration.
>
> You might even want to search the archives of this mailinglist, I remember
> this has been asked before.
>
> Cheers!
>
> 2012/5/21 Luís Ferreira 
>
>> Hi,
>>
>> Does the number of keyspaces affect the overall cassandra performance?
>>
>>
>> Cumprimentos,
>> Luís Ferreira
>>
>>
>>
>>
>
>
> --
> With kind regards,
>
> Robin Verlangen
> www.robinverlangen.nl
>
>
>


-- 

*Franc Carter* | Systems architect | Sirca Ltd
 

franc.car...@sirca.org.au | www.sirca.org.au

Tel: +61 2 9236 9118

Level 9, 80 Clarence St, Sydney NSW 2000

PO Box H58, Australia Square, Sydney NSW 1215

Re: Number of keyspaces

[samal]

Not ideally, now cass has global memtable tuning. Each cf correspond to
memory  in ram. Year wise cf means it will be in read only state for next
year, memtable  will still consume ram.
On 22-May-2012 5:01 PM, "Franc Carter"  wrote:

> On Tue, May 22, 2012 at 9:19 PM, aaron morton wrote:
>
>> It's more the number of CF's than keyspaces.
>>
>
> Oh - does increasing the number of Column Families affect performance ?
>
> The design we are working on at the moment is considering using a Column
> Family per year. We were thinking this would isolate compactions to a more
> manageable size as we don't update previous years.
>
> cheers
>
>
>>
>> Cheers
>>
>>   -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
>>
>> Yes, it does. However there's no real answer what's the limit: it depends
>> on your hardware and cluster configuration.
>>
>> You might even want to search the archives of this mailinglist, I
>> remember this has been asked before.
>>
>> Cheers!
>>
>> 2012/5/21 Luís Ferreira 
>>
>>> Hi,
>>>
>>> Does the number of keyspaces affect the overall cassandra performance?
>>>
>>>
>>> Cumprimentos,
>>> Luís Ferreira
>>>
>>>
>>>
>>>
>>
>>
>> --
>> With kind regards,
>>
>> Robin Verlangen
>> www.robinverlangen.nl
>>
>>
>>
>
>
> --
>
> *Franc Carter* | Systems architect | Sirca Ltd
>  
>
> franc.car...@sirca.org.au | www.sirca.org.au
>
> Tel: +61 2 9236 9118
>
> Level 9, 80 Clarence St, Sydney NSW 2000
>
> PO Box H58, Australia Square, Sydney NSW 1215
>
>

Re: Astyanax Error

[samal]

Host not found in client.
On 22-May-2012 4:34 PM, "Abhijit Chanda"  wrote:

> Hi All,
>
> Can any one suggest me why i am getting this error in Astyanax
> NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0),
> attempts=0] No hosts to borrow from
>
>
> Thanks In Advance
> Abhijit
>

Re: RE Ordering counters in Cassandra

[Filippo Diotalevi]

Thanks for all the answers, they definitely helped.  

Just out of curiosity, is there any underlying architectural reason why it's 
not possible to order a row based on its counters values? or is it something 
that might be in the roadmap in the future?  

--  
Filippo Diotalevi


On Tuesday, 22 May 2012 at 08:48, Romain HARDOUIN wrote:

>  
> I mean iterate over each column -- more precisly: *bunches of columns* using 
> slices -- and write new columns in the inversed index.  
> Tamar's data model is made for real time analysis. It's maybe overdesigned 
> for a daily ranking.  
> I agree with Samal, you should split your data across the space of tokens. 
> Only CF Ranking feeding would be affected, not the "top N" queries.  
>  
> Filippo Diotalevi mailto:fili...@ntoklo.com)> a écrit 
> sur 21/05/2012 19:05:28 :
>  
> > Hi Romain,  
> > thanks for your suggestion.  
> >  
> > When you say " build every day a ranking in a dedicated CF by  
> > iterating over events:" do you mean  
> > - load all the columns for the specified row key  
> > - iterate over each column, and write a new column in the inversed index  
> > ?  
> >  
> > That's my current approach, but since I have many of these wide rows
> > (1 per day), the process is extremely slow as it involves moving an  
> > entire row from Cassandra to client, inverting every column, and  
> > sending the data back to create the inversed index.  

supercolumns with TTL columns not being compacted correctly

[Pieter Callewaert]

Hi,

I've had my suspicions some months, but I think I am sure about it.
Data is being written by the SSTableSimpleUnsortedWriter and loaded by the 
sstableloader.
The data should be alive for 31 days, so I use the following logic:

int ttl = 2678400;
long timestamp = System.currentTimeMillis() * 1000;
long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl * 1000));

And using this to write it:

sstableWriter.newRow(bytes(entry.id));
sstableWriter.newSuperColumn(bytes(superColumn));
sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs), timestamp, 
ttl, expirationTimestampMS);
sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage), 
timestamp, ttl, expirationTimestampMS);
sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp, ttl, 
expirationTimestampMS);

This works perfectly, data can be queried until 31 days are passed, then no 
results are given, as expected.
But the data is still on disk until the sstables are being recompacted:

One of our nodes (we got 6 total) has the following sstables:
[cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
-rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19 
/data/MapData007/HOS-hc-125620-Data.db
-rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17 
/data/MapData007/HOS-hc-163141-Data.db
-rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17 
/data/MapData007/HOS-hc-172106-Data.db
-rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50 
/data/MapData007/HOS-hc-181902-Data.db
-rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37 
/data/MapData007/HOS-hc-191448-Data.db
-rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41 
/data/MapData007/HOS-hc-193842-Data.db
-rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03 
/data/MapData007/HOS-hc-196210-Data.db
-rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20 
/data/MapData007/HOS-hc-196779-Data.db
-rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33 
/data/MapData007/HOS-hc-58572-Data.db
-rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59 
/data/MapData007/HOS-hc-61630-Data.db
-rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46 
/data/MapData007/HOS-hc-63857-Data.db
-rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41 
/data/MapData007/HOS-hc-87900-Data.db

As you can see, the following files should be invalid:
/data/MapData007/HOS-hc-58572-Data.db
/data/MapData007/HOS-hc-61630-Data.db
/data/MapData007/HOS-hc-63857-Data.db

Because they are all written more than an moth ago. gc_grace is 0 so this 
should also not be a problem.

As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
Expected behavior should be an empty file is being written because all data in 
the sstable should be invalid:

Compactionstats is giving:
compaction typekeyspace   column family bytes compacted bytes total 
 progress
   Compaction  MapData007 HOS 11518215662
532355279724 2.16%

And when I ls the directory I find this:
-rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12 
/data/MapData007/HOS-tmp-hc-196898-Data.db

The sstable is being 1-on-1 copied to a new one. What am I missing here?
TTL works perfectly, but is it giving a problem because it is in a super 
column, and so never to be deleted from disk?

Kind regards
Pieter Callewaert | Web & IT engineer
 Be-Mobile NV | 
TouringMobilis
 Technologiepark 12b - 9052 Ghent - Belgium
Tel + 32 9 330 51 80 | Fax + 32 9 330 51 81 |  Cell + 32 473 777 121

Re: RE Ordering counters in Cassandra

[samal]

Secondary index is not supported for counters plus you must know column
name to support secondary index on regular column.
On 22-May-2012 5:34 PM, "Filippo Diotalevi"  wrote:

>  Thanks for all the answers, they definitely helped.
>
> Just out of curiosity, is there any underlying architectural reason why
> it's not possible to order a row based on its counters values? or is it
> something that might be in the roadmap in the future?
>
> --
> Filippo Diotalevi
>
> On Tuesday, 22 May 2012 at 08:48, Romain HARDOUIN wrote:
>
>
> I mean iterate over each column -- more precisly: *bunches of columns*
> using slices -- and write new columns in the inversed index.
> Tamar's data model is made for real time analysis. It's maybe overdesigned
> for a daily ranking.
> I agree with Samal, you should split your data across the space of tokens.
> Only CF Ranking feeding would be affected, not the "top N" queries.
>
> Filippo Diotalevi  a écrit sur 21/05/2012 19:05:28 :
>
> > Hi Romain,
> > thanks for your suggestion.
> >
> > When you say " build every day a ranking in a dedicated CF by
> > iterating over events:" do you mean
> > - load all the columns for the specified row key
> > - iterate over each column, and write a new column in the inversed index
> > ?
> >
> > That's my current approach, but since I have many of these wide rows
> > (1 per day), the process is extremely slow as it involves moving an
> > entire row from Cassandra to client, inverting every column, and
> > sending the data back to create the inversed index.
>
>
>

Re: Astyanax Error

[Abhijit Chanda]

Samal,


But I am setting up the Host.

On Tue, May 22, 2012 at 5:30 PM, samal  wrote:

> Host not found in client.
> On 22-May-2012 4:34 PM, "Abhijit Chanda" 
> wrote:
>
>> Hi All,
>>
>> Can any one suggest me why i am getting this error in Astyanax
>> NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0),
>> attempts=0] No hosts to borrow from
>>
>>
>> Thanks In Advance
>> Abhijit
>>
>


-- 
Abhijit Chanda
Software Developer
VeHere Interactive Pvt. Ltd.
+91-974395

Re: supercolumns with TTL columns not being compacted correctly

[samal]

Data will remain till next compaction but won't be available. Compaction
will delete old sstable create new one.
On 22-May-2012 5:47 PM, "Pieter Callewaert" 
wrote:

>  Hi,
>
> ** **
>
> I’ve had my suspicions some months, but I think I am sure about it.
>
> Data is being written by the SSTableSimpleUnsortedWriter and loaded by the
> sstableloader.
>
> The data should be alive for 31 days, so I use the following logic:
>
> ** **
>
> int ttl = 2678400;
>
> long timestamp = System.currentTimeMillis() * 1000;
>
> long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl *
> 1000));
>
> ** **
>
> And using this to write it:
>
> ** **
>
> sstableWriter.newRow(bytes(entry.id));
>
> sstableWriter.newSuperColumn(bytes(superColumn));
>
> sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs),
> timestamp, ttl, expirationTimestampMS);
>
> sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage),
> timestamp, ttl, expirationTimestampMS);
>
> sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp,
> ttl, expirationTimestampMS);
>
> ** **
>
> This works perfectly, data can be queried until 31 days are passed, then
> no results are given, as expected.
>
> But the data is still on disk until the sstables are being recompacted:***
> *
>
> ** **
>
> One of our nodes (we got 6 total) has the following sstables:
>
> [cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
>
> -rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19
> /data/MapData007/HOS-hc-125620-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17
> /data/MapData007/HOS-hc-163141-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17
> /data/MapData007/HOS-hc-172106-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50
> /data/MapData007/HOS-hc-181902-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37
> /data/MapData007/HOS-hc-191448-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41
> /data/MapData007/HOS-hc-193842-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03
> /data/MapData007/HOS-hc-196210-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20
> /data/MapData007/HOS-hc-196779-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33
> /data/MapData007/HOS-hc-58572-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59
> /data/MapData007/HOS-hc-61630-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46
> /data/MapData007/HOS-hc-63857-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41
> /data/MapData007/HOS-hc-87900-Data.db
>
> ** **
>
> As you can see, the following files should be invalid:
>
> /data/MapData007/HOS-hc-58572-Data.db
>
> /data/MapData007/HOS-hc-61630-Data.db
>
> /data/MapData007/HOS-hc-63857-Data.db
>
> ** **
>
> Because they are all written more than an moth ago. gc_grace is 0 so this
> should also not be a problem.
>
> ** **
>
> As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
> 
>
> Expected behavior should be an empty file is being written because all
> data in the sstable should be invalid:
>
> ** **
>
> Compactionstats is giving:
>
> compaction typekeyspace   column family bytes compacted bytes
> total  progress
>
>Compaction  MapData007 HOS
> 11518215662532355279724 2.16%
>
> ** **
>
> And when I ls the directory I find this:
>
> -rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12
> /data/MapData007/HOS-tmp-hc-196898-Data.db
>
> ** **
>
> The sstable is being 1-on-1 copied to a new one. What am I missing here?**
> **
>
> TTL works perfectly, but is it giving a problem because it is in a super
> column, and so never to be deleted from disk?
>
> ** **
>
> Kind regards
>
> Pieter Callewaert | Web & IT engineer
>
>  Be-Mobile NV  | 
> TouringMobilis
> 
>
>  Technologiepark 12b - 9052 Ghent - Belgium
>
> Tel + 32 9 330 51 80 | Fax + 32 9 330 51 81 |  Cell + 32 473 777 121
>
> ** **
>

RE: supercolumns with TTL columns not being compacted correctly

[Pieter Callewaert]

Hi Samal,

Thanks for your time looking into this.

I force the compaction by using forceUserDefinedCompaction on only that 
particular sstable. This gurantees me the new sstable being written only 
contains the data from the old sstable.
The data in the sstable is more than 31 days old and gc_grace is 0, but still 
the data from the sstable is being written to the new one, while I am 100% sure 
all the data is invalid.

Kind regards,
Pieter Callewaert

From: samal [mailto:samalgo...@gmail.com]
Sent: dinsdag 22 mei 2012 14:33
To: user@cassandra.apache.org
Subject: Re: supercolumns with TTL columns not being compacted correctly


Data will remain till next compaction but won't be available. Compaction will 
delete old sstable create new one.
On 22-May-2012 5:47 PM, "Pieter Callewaert" 
mailto:pieter.callewa...@be-mobile.be>> wrote:
Hi,

I’ve had my suspicions some months, but I think I am sure about it.
Data is being written by the SSTableSimpleUnsortedWriter and loaded by the 
sstableloader.
The data should be alive for 31 days, so I use the following logic:

int ttl = 2678400;
long timestamp = System.currentTimeMillis() * 1000;
long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl * 1000));

And using this to write it:

sstableWriter.newRow(bytes(entry.id));
sstableWriter.newSuperColumn(bytes(superColumn));
sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs), timestamp, 
ttl, expirationTimestampMS);
sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage), 
timestamp, ttl, expirationTimestampMS);
sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp, ttl, 
expirationTimestampMS);

This works perfectly, data can be queried until 31 days are passed, then no 
results are given, as expected.
But the data is still on disk until the sstables are being recompacted:

One of our nodes (we got 6 total) has the following sstables:
[cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
-rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19 
/data/MapData007/HOS-hc-125620-Data.db
-rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17 
/data/MapData007/HOS-hc-163141-Data.db
-rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17 
/data/MapData007/HOS-hc-172106-Data.db
-rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50 
/data/MapData007/HOS-hc-181902-Data.db
-rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37 
/data/MapData007/HOS-hc-191448-Data.db
-rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41 
/data/MapData007/HOS-hc-193842-Data.db
-rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03 
/data/MapData007/HOS-hc-196210-Data.db
-rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20 
/data/MapData007/HOS-hc-196779-Data.db
-rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33 
/data/MapData007/HOS-hc-58572-Data.db
-rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59 
/data/MapData007/HOS-hc-61630-Data.db
-rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46 
/data/MapData007/HOS-hc-63857-Data.db
-rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41 
/data/MapData007/HOS-hc-87900-Data.db

As you can see, the following files should be invalid:
/data/MapData007/HOS-hc-58572-Data.db
/data/MapData007/HOS-hc-61630-Data.db
/data/MapData007/HOS-hc-63857-Data.db

Because they are all written more than an moth ago. gc_grace is 0 so this 
should also not be a problem.

As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
Expected behavior should be an empty file is being written because all data in 
the sstable should be invalid:

Compactionstats is giving:
compaction typekeyspace   column family bytes compacted bytes total 
 progress
   Compaction  MapData007 HOS 11518215662
532355279724 2.16%

And when I ls the directory I find this:
-rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12 
/data/MapData007/HOS-tmp-hc-196898-Data.db

The sstable is being 1-on-1 copied to a new one. What am I missing here?
TTL works perfectly, but is it giving a problem because it is in a super 
column, and so never to be deleted from disk?

Kind regards
Pieter Callewaert | Web & IT engineer
 Be-Mobile NV | 
TouringMobilis
 Technologiepark 12b - 9052 Ghent - Belgium
Tel + 32 9 330 51 80 | Fax + 32 9 330 51 81 |  Cell + 32 473 777 121

Re: Astyanax Error

[samal]

Are you able to connect through cli?
Can you share your client code?
On 22-May-2012 5:59 PM, "Abhijit Chanda"  wrote:

> Samal,
>
>
> But I am setting up the Host.
>
> On Tue, May 22, 2012 at 5:30 PM, samal  wrote:
>
>> Host not found in client.
>> On 22-May-2012 4:34 PM, "Abhijit Chanda" 
>> wrote:
>>
>>> Hi All,
>>>
>>> Can any one suggest me why i am getting this error in Astyanax
>>> NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0),
>>> attempts=0] No hosts to borrow from
>>>
>>>
>>> Thanks In Advance
>>> Abhijit
>>>
>>
>
>
> --
> Abhijit Chanda
> Software Developer
> VeHere Interactive Pvt. Ltd.
> +91-974395
>
>

Re: Tuning cassandra (compactions overall)

[Alain RODRIGUEZ]

"not sure what you mean by
And after restarting the second one I have lost all the consistency of
my data. All my statistics since September are totally false now in
production

Can you give some examples?"

After restarting my 2 nodes (one after the other), All my counters
have become wrong. The counters values were modified by the restart.
Let's say I had a counter column called 20120101#click that value was
569, after the restart the value has become 751. I think that all the
values have increased (I'm not sure) but all counters have increased
in differents way, some values have increased a lot other just a bit.

"Counter are not idempotent so if the client app retries TimedOut
requests you can get an over count. That should not result in lost
data."

Some of these counters haven't be written since September and have
still been modified by the restart.

"Have you been running repair ?"

Yes, Repair didn't helped. I have the feeling that repairing doesn't
work on counters.

I have restored the data now, but I am afraid of restarting any node.
I can remain in this position too long...

Re: Cassandra 0.8.5: Column name mystery in create column family command

[samal]

Change your comparator to utf8type.
On 22-May-2012 4:32 PM, "Roshan Dawrani"  wrote:

> Hi,
>
> I use Cassandra 0.8.5 and am suddenly noticing some strange behavior. I
> run a "create column family" command with some column meta-data and it runs
> fine, but when I do "describe keyspace", it shows me different column names
> for those index columns.
>
> a) Here is what I run:
> "create column family UserTemplate with comparator=BytesType and
> column_metadata=[{*column_name: userid*, validation_class: UTF8Type,
> index_type: KEYS, index_name: TemplateUserIdIdx}, {*column_name: type*,
> validation_class: UTF8Type, index_type: KEYS, index_name:
> TemplateTypeIdx}];"
>
> b) This is what "describe keyspace" shows:
> ColumnFamily: UserTemplate
>   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
>   ...
>   ...
>   Built indexes: [UserTemplate.TemplateTypeIdx,
> UserTemplate.TemplateUserIdIdx]
>   Column Metadata:
> *Column Name: ff*
>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>   Index Name: TemplateUserIdIdx
>   Index Type: KEYS
> *Column Name: 0dfffaff*
>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>   Index Name: TemplateTypeIdx
>   Index Type: KEYS
>
> Does anyone see why this must be happening? I have created many such
> column families before and never run into this issue.
>
> --
> Roshan
> http://roshandawrani.wordpress.com/
>
>

Re: supercolumns with TTL columns not being compacted correctly

[Yuki Morishita]

Data will not be deleted when those keys appear in other stables outside of 
compaction. This is to prevent obsolete data from appearing again.

yuki


On Tuesday, May 22, 2012 at 7:37 AM, Pieter Callewaert wrote:

>  
> Hi Samal,
>  
>  
>   
>  
>  
> Thanks for your time looking into this.
>  
>  
>   
>  
>  
> I force the compaction by using forceUserDefinedCompaction on only that 
> particular sstable. This gurantees me the new sstable being written only 
> contains the data from the old sstable.
>  
>  
> The data in the sstable is more than 31 days old and gc_grace is 0, but still 
> the data from the sstable is being written to the new one, while I am 100% 
> sure all the data is invalid.
>  
>  
>   
>  
>  
> Kind regards,
>  
>  
> Pieter Callewaert
>  
>  
>   
>  
>  
> From: samal [mailto:samalgo...@gmail.com]  
> Sent: dinsdag 22 mei 2012 14:33
> To: user@cassandra.apache.org (mailto:user@cassandra.apache.org)
> Subject: Re: supercolumns with TTL columns not being compacted correctly
>  
>  
>   
>  
> Data will remain till next compaction but won't be available. Compaction will 
> delete old sstable create new one.
>  
> On 22-May-2012 5:47 PM, "Pieter Callewaert"  (mailto:pieter.callewa...@be-mobile.be)> wrote:
>  
>  
> Hi,
>  
>  
>   
>  
>  
> I’ve had my suspicions some months, but I think I am sure about it.
>  
>  
> Data is being written by the SSTableSimpleUnsortedWriter and loaded by the 
> sstableloader.
>  
>  
> The data should be alive for 31 days, so I use the following logic:
>  
>  
>   
>  
>  
> int ttl = 2678400;
>  
>  
> long timestamp = System.currentTimeMillis() * 1000;
>  
>  
> long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl * 
> 1000));
>  
>  
>   
>  
>  
> And using this to write it:
>  
>  
>   
>  
>  
> sstableWriter.newRow(bytes(entry.id (http://entry.id)));
>  
>  
> sstableWriter.newSuperColumn(bytes(superColumn));
>  
>  
> sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs), 
> timestamp, ttl, expirationTimestampMS);
>  
>  
> sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage), 
> timestamp, ttl, expirationTimestampMS);
>  
>  
> sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp, ttl, 
> expirationTimestampMS);
>  
>  
>   
>  
>  
> This works perfectly, data can be queried until 31 days are passed, then no 
> results are given, as expected.
>  
>  
> But the data is still on disk until the sstables are being recompacted:
>  
>  
>   
>  
>  
> One of our nodes (we got 6 total) has the following sstables:
>  
>  
> [cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19 
> /data/MapData007/HOS-hc-125620-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17 
> /data/MapData007/HOS-hc-163141-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17 
> /data/MapData007/HOS-hc-172106-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50 
> /data/MapData007/HOS-hc-181902-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37 
> /data/MapData007/HOS-hc-191448-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41 
> /data/MapData007/HOS-hc-193842-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03 
> /data/MapData007/HOS-hc-196210-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20 
> /data/MapData007/HOS-hc-196779-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33 
> /data/MapData007/HOS-hc-58572-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59 
> /data/MapData007/HOS-hc-61630-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46 
> /data/MapData007/HOS-hc-63857-Data.db
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41 
> /data/MapData007/HOS-hc-87900-Data.db
>  
>  
>   
>  
>  
> As you can see, the following files should be invalid:
>  
>  
> /data/MapData007/HOS-hc-58572-Data.db
>  
>  
> /data/MapData007/HOS-hc-61630-Data.db
>  
>  
> /data/MapData007/HOS-hc-63857-Data.db
>  
>  
>   
>  
>  
> Because they are all written more than an moth ago. gc_grace is 0 so this 
> should also not be a problem.
>  
>  
>   
>  
>  
> As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
>  
>  
> Expected behavior should be an empty file is being written because all data 
> in the sstable should be invalid:
>  
>  
>   
>  
>  
> Compactionstats is giving:
>  
>  
> compaction typekeyspace   column family bytes compacted bytes 
> total  progress
>  
>  
>Compaction  MapData007 HOS 11518215662
> 532355279724 2.16%
>  
>  
>   
>  
>  
> And when I ls the directory I find this:
>  
>  
> -rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12 
> /data/MapData007/HOS-tmp-hc-196898-Data.db
>  
>  
>   
>  
>  
> The sstable is being 1-on-1 copied to a new one. What

Re: supercolumns with TTL columns not being compacted correctly

[Jonathan Ellis]

Additionally, it will always take at least two compaction passes to
purge an expired column: one to turn it into a tombstone, and a second
(after gcgs) to remove it.

On Tue, May 22, 2012 at 9:21 AM, Yuki Morishita  wrote:
> Data will not be deleted when those keys appear in other stables outside of
> compaction. This is to prevent obsolete data from appearing again.
>
> yuki
>
> On Tuesday, May 22, 2012 at 7:37 AM, Pieter Callewaert wrote:
>
> Hi Samal,
>
>
>
> Thanks for your time looking into this.
>
>
>
> I force the compaction by using forceUserDefinedCompaction on only that
> particular sstable. This gurantees me the new sstable being written only
> contains the data from the old sstable.
>
> The data in the sstable is more than 31 days old and gc_grace is 0, but
> still the data from the sstable is being written to the new one, while I am
> 100% sure all the data is invalid.
>
>
>
> Kind regards,
>
> Pieter Callewaert
>
>
>
> From: samal [mailto:samalgo...@gmail.com]
> Sent: dinsdag 22 mei 2012 14:33
> To: user@cassandra.apache.org
> Subject: Re: supercolumns with TTL columns not being compacted correctly
>
>
>
> Data will remain till next compaction but won't be available. Compaction
> will delete old sstable create new one.
>
> On 22-May-2012 5:47 PM, "Pieter Callewaert" 
> wrote:
>
> Hi,
>
>
>
> I’ve had my suspicions some months, but I think I am sure about it.
>
> Data is being written by the SSTableSimpleUnsortedWriter and loaded by the
> sstableloader.
>
> The data should be alive for 31 days, so I use the following logic:
>
>
>
> int ttl = 2678400;
>
> long timestamp = System.currentTimeMillis() * 1000;
>
> long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl *
> 1000));
>
>
>
> And using this to write it:
>
>
>
> sstableWriter.newRow(bytes(entry.id));
>
> sstableWriter.newSuperColumn(bytes(superColumn));
>
> sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs),
> timestamp, ttl, expirationTimestampMS);
>
> sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage),
> timestamp, ttl, expirationTimestampMS);
>
> sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp, ttl,
> expirationTimestampMS);
>
>
>
> This works perfectly, data can be queried until 31 days are passed, then no
> results are given, as expected.
>
> But the data is still on disk until the sstables are being recompacted:
>
>
>
> One of our nodes (we got 6 total) has the following sstables:
>
> [cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
>
> -rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19
> /data/MapData007/HOS-hc-125620-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17
> /data/MapData007/HOS-hc-163141-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17
> /data/MapData007/HOS-hc-172106-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50
> /data/MapData007/HOS-hc-181902-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37
> /data/MapData007/HOS-hc-191448-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41
> /data/MapData007/HOS-hc-193842-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03
> /data/MapData007/HOS-hc-196210-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20
> /data/MapData007/HOS-hc-196779-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33
> /data/MapData007/HOS-hc-58572-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59
> /data/MapData007/HOS-hc-61630-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46
> /data/MapData007/HOS-hc-63857-Data.db
>
> -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41
> /data/MapData007/HOS-hc-87900-Data.db
>
>
>
> As you can see, the following files should be invalid:
>
> /data/MapData007/HOS-hc-58572-Data.db
>
> /data/MapData007/HOS-hc-61630-Data.db
>
> /data/MapData007/HOS-hc-63857-Data.db
>
>
>
> Because they are all written more than an moth ago. gc_grace is 0 so this
> should also not be a problem.
>
>
>
> As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
>
> Expected behavior should be an empty file is being written because all data
> in the sstable should be invalid:
>
>
>
> Compactionstats is giving:
>
> compaction type    keyspace   column family bytes compacted bytes
> total  progress
>
>    Compaction  MapData007 HOS 11518215662
> 532355279724 2.16%
>
>
>
> And when I ls the directory I find this:
>
> -rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12
> /data/MapData007/HOS-tmp-hc-196898-Data.db
>
>
>
> The sstable is being 1-on-1 copied to a new one. What am I missing here?
>
> TTL works perfectly, but is it giving a problem because it is in a super
> column, and so never to be deleted from disk?
>
>
>
> Kind regards
>
> Pieter Callewaert | Web & IT engineer
>
>  Be-Mobile NV | TouringMobilis
>
>  Technologiepark 12b - 9052 Ghent - Belgium
>
> Tel + 32 9 330 51 80 | Fax + 32 9 330 51 81 |  

Re: Cassandra 0.8.5: Column name mystery in create column family command

[Roshan Dawrani]

Can you please let me know why? Because I have created very similar column
familes many times with comparator = BytesType, and never run into this
issue before.

Here is an example:


ColumnFamily: Client
  Key Validation Class: org.apache.cassandra.db.marshal.BytesType
  Default column value validator: org.apache.cassandra.db.marshal.BytesType
  Columns sorted by: org.apache.cassandra.db.marshal.BytesType
  ...
  ...
  Built indexes: [Client.ACUserIdIdx]
  Column Metadata:
Column Name: userid (757365726964)
  Validation Class: org.apache.cassandra.db.marshal.LexicalUUIDType
  Index Name: ACUserIdIdx
  Index Type: KEYS


On Tue, May 22, 2012 at 6:16 PM, samal  wrote:

> Change your comparator to utf8type.
> On 22-May-2012 4:32 PM, "Roshan Dawrani"  wrote:
>
>> Hi,
>>
>> I use Cassandra 0.8.5 and am suddenly noticing some strange behavior. I
>> run a "create column family" command with some column meta-data and it runs
>> fine, but when I do "describe keyspace", it shows me different column names
>> for those index columns.
>>
>> a) Here is what I run:
>> "create column family UserTemplate with comparator=BytesType and
>> column_metadata=[{*column_name: userid*, validation_class: UTF8Type,
>> index_type: KEYS, index_name: TemplateUserIdIdx}, {*column_name: type*,
>> validation_class: UTF8Type, index_type: KEYS, index_name:
>> TemplateTypeIdx}];"
>>
>> b) This is what "describe keyspace" shows:
>> ColumnFamily: UserTemplate
>>   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
>>   ...
>>   ...
>>   Built indexes: [UserTemplate.TemplateTypeIdx,
>> UserTemplate.TemplateUserIdIdx]
>>   Column Metadata:
>> *Column Name: ff*
>>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>>   Index Name: TemplateUserIdIdx
>>   Index Type: KEYS
>> *Column Name: 0dfffaff*
>>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>>   Index Name: TemplateTypeIdx
>>   Index Type: KEYS
>>
>> Does anyone see why this must be happening? I have created many such
>> column families before and never run into this issue.
>>
>> --
>> Roshan
>> http://roshandawrani.wordpress.com/
>>
>>


-- 
Roshan
http://roshandawrani.wordpress.com/

Re: Welcome committers Dave Brosius and Yuki Morishita!

[Edward Capriolo]

Congrats!

On Tue, May 22, 2012 at 10:43 AM, Jonathan Ellis  wrote:
> Thanks to both of you for your help!
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com

Correct way to set strategy options in cqlsh?

[Damick, Jeffrey]

What's the correct way to set the strategy options for the 
networktopologystrategy with cqlsh?

I've tried several variations, but what's expected way to escape the hyphen in 
"us-east" ?

Thanks,
-jeff

CREATE KEYSPACE something
   ...   WITH strategy_class = 'NetworkTopologyStrategy'
   ...   AND strategy_options:us-east=1
   ...   AND strategy_options:us-west=1;
Bad Request: line 3:25 mismatched input '-' expecting '='

Re: Correct way to set strategy options in cqlsh?

[Yiming Sun]

AND strategy_options={us-east:1, us-west:1};



On Tue, May 22, 2012 at 11:10 AM, Damick, Jeffrey <
jeffrey.dam...@neustar.biz> wrote:

>  What’s the correct way to set the strategy options for the
> networktopologystrategy with cqlsh?
>
> I’ve tried several variations, but what’s expected way to escape the
> hyphen in “us-east” ?
>
> Thanks,
> -jeff
>
> CREATE KEYSPACE something
>...   WITH strategy_class = 'NetworkTopologyStrategy'
>...   AND strategy_options:us-east=1
>...   AND strategy_options:us-west=1;
> Bad Request: line 3:25 mismatched input '-' expecting '='
>
>
>
>

Re: Correct way to set strategy options in cqlsh?

[Damick, Jeffrey]

Thanks, but that would be for the cli, not cqlsh

CREATE KEYSPACE something
   ...   WITH strategy_class = 'NetworkTopologyStrategy' AND 
strategy_options={us-east:1};
Invalid syntax at line 2, char 72
WITH strategy_class = 'NetworkTopologyStrategy' AND 
strategy_options={us-east:1};



On 5/22/12 11:18 AM, "Yiming Sun"  wrote:

AND strategy_options={us-east:1, us-west:1};



On Tue, May 22, 2012 at 11:10 AM, Damick, Jeffrey  
wrote:
What's the correct way to set the strategy options for the 
networktopologystrategy with cqlsh?

I've tried several variations, but what's expected way to escape the hyphen in 
"us-east" ?

Thanks,
-jeff

CREATE KEYSPACE something
   ...   WITH strategy_class = 'NetworkTopologyStrategy'
   ...   AND strategy_options:us-east=1
   ...   AND strategy_options:us-west=1;
Bad Request: line 3:25 mismatched input '-' expecting '='

Re: Number of keyspaces

[Luís Ferreira]

I have 24 keyspaces, each with a columns family and am considering changing it 
to 1 keyspace with 24 CFs. Would this be beneficial?
On May 22, 2012, at 12:56 PM, samal wrote:

> Not ideally, now cass has global memtable tuning. Each cf correspond to 
> memory  in ram. Year wise cf means it will be in read only state for next 
> year, memtable  will still consume ram.
> 
> On 22-May-2012 5:01 PM, "Franc Carter"  wrote:
> On Tue, May 22, 2012 at 9:19 PM, aaron morton  wrote:
> It's more the number of CF's than keyspaces.
> 
> Oh - does increasing the number of Column Families affect performance ?
> 
> The design we are working on at the moment is considering using a Column 
> Family per year. We were thinking this would isolate compactions to a more 
> manageable size as we don't update previous years.
> 
> cheers
>  
> 
> Cheers
> 
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
> 
>> Yes, it does. However there's no real answer what's the limit: it depends on 
>> your hardware and cluster configuration. 
>> 
>> You might even want to search the archives of this mailinglist, I remember 
>> this has been asked before.
>> 
>> Cheers!
>> 
>> 2012/5/21 Luís Ferreira 
>> Hi,
>> 
>> Does the number of keyspaces affect the overall cassandra performance?
>> 
>> 
>> Cumprimentos,
>> Luís Ferreira
>> 
>> 
>> 
>> 
>> 
>> 
>> -- 
>> With kind regards,
>> 
>> Robin Verlangen
>> www.robinverlangen.nl
>> 
> 
> 
> 
> 
> -- 
> Franc Carter | Systems architect | Sirca Ltd
> franc.car...@sirca.org.au | www.sirca.org.au
> Tel: +61 2 9236 9118 
> Level 9, 80 Clarence St, Sydney NSW 2000
> PO Box H58, Australia Square, Sydney NSW 1215
> 

Cumprimentos,
Luís Ferreira

Re: exception when cleaning up...

[Rob Coli]

On Tue, May 22, 2012 at 3:00 AM, aaron morton  wrote:
> 1) Isolating the node from the cluster to stop write activity. You can
> either start the node with the -Dcassandra.join_ring=false  JVM option or
> use nodetool disablethrift and disablegossip to stop writes. Note that this
> will not stop existing Hinted Handoff sessions which target the node.

As a result of the last caveat here, I recommend either restarting the
node with the join_ring as false or using iptables to firewall off
ports 7000 and 9160. If you want to be sure that you have stopped
write activity right now, nuking these ports from orbit is the only
way to be sure. disablethrift/disablegossip as currently implemented
are not sufficient for this goal.

https://issues.apache.org/jira/browse/CASSANDRA-4162

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: Welcome committers Dave Brosius and Yuki Morishita!

[Sylvain Lebresne]

Thanks for all the work and congratulations to both of you.

--
Sylvain

On Tue, May 22, 2012 at 5:06 PM, Edward Capriolo  wrote:
> Congrats!
>
> On Tue, May 22, 2012 at 10:43 AM, Jonathan Ellis  wrote:
>> Thanks to both of you for your help!
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com

unknown exception with hector

[Deno Vichas]

could somebody clue me in to the cause of this exception?  i see these 
randomly.


AnalyzerService-2 2012-05-22 13:28:00,385 :: WARN  
cassandra.connection.HConnectionManager  - Exception:
me.prettyprint.hector.api.exceptions.HectorTransportException: 
org.apache.thrift.transport.TTransportException
at 
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:39)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:851)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:840)
at 
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:99)
at 
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:243)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.getColumn(KeyspaceServiceImpl.java:857)
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:57)
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:52)
at 
me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
at 
me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery.execute(ThriftColumnQuery.java:51)
at 
com.stocktouch.dao.StockDaoImpl.getHistorical(StockDaoImpl.java:365)
at 
com.stocktouch.dao.StockDaoImpl.getHistoricalQuote(StockDaoImpl.java:433)
at 
com.stocktouch.service.StockHistoryServiceImpl.getHistoricalQuote(StockHistoryServiceImpl.java:480)
at 
com.stocktouch.service.AnalyzerServiceImpl.getClose(AnalyzerServiceImpl.java:180)
at 
com.stocktouch.service.AnalyzerServiceImpl.calcClosingPrices(AnalyzerServiceImpl.java:90)
at 
com.stocktouch.service.AnalyzerServiceImpl.nightlyRollup(AnalyzerServiceImpl.java:66)
at 
com.stocktouch.service.AnalyzerServiceImpl$2.run(AnalyzerServiceImpl.java:55)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.thrift.transport.TTransportException
at 
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at 
org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at 
org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at 
org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at 
org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at 
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at 
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at 
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at 
org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:509)
at 
org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:492)
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:846)

... 20 more


thanks,
deno

Re: Astyanax Error

[Eran Landau]

This is normally the result of not having the client properly set up to talk to 
Cassandra.  Can you send a code snippet of how you are initializing Astyanax?

- Eran

From: Abhijit Chanda 
mailto:abhijit.chan...@gmail.com>>
Reply-To: mailto:user@cassandra.apache.org>>
Date: Tue, 22 May 2012 16:33:32 +0530
To: mailto:user@cassandra.apache.org>>
Subject: Astyanax Error

Hi All,

Can any one suggest me why i am getting this error in Astyanax
NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0), attempts=0] No 
hosts to borrow from


Thanks In Advance
Abhijit

RE: 1.1 not removing commit log files?

[Bryce Godfrey]

The nodes appear to be holding steady at the 8G that I set it to in the config 
file now.  I'll keep an eye on them.

From: aaron morton [mailto:aa...@thelastpickle.com]
Sent: Tuesday, May 22, 2012 4:08 AM
To: user@cassandra.apache.org
Subject: Re: 1.1 not removing commit log files?

4096 is also the internal hard coded default for commitlog_total_space_in_mb

If you are seeing more that 4GB of commit log files let us know.

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 22/05/2012, at 6:35 AM, Bryce Godfrey wrote:


Thanks, I'll give it a try.

-Original Message-
From: Alain RODRIGUEZ 
[mailto:arodr...@gmail.com]
Sent: Monday, May 21, 2012 2:12 AM
To: user@cassandra.apache.org
Subject: Re: 1.1 not removing commit log files?

commitlog_total_space_in_mb: 4096

By default this line is commented in 1.0.x if I remember well. I guess it is 
the same in 1.1. You really should remove this comment or your commit logs will 
entirely fill up your disk as it happened to me a while ago.

Alain

2012/5/21 Pieter Callewaert 
mailto:pieter.callewa...@be-mobile.be>>:

Hi,



In 1.1 the commitlog files are pre-allocated with files of 128MB.
(https://issues.apache.org/jira/browse/CASSANDRA-3411) This should
however not exceed your commitlog size in Cassandra.yaml.



commitlog_total_space_in_mb: 4096



Kind regards,

Pieter Callewaert



From: Bryce Godfrey 
[mailto:bryce.godf...@azaleos.com]
Sent: maandag 21 mei 2012 9:52
To: user@cassandra.apache.org
Subject: 1.1 not removing commit log files?



The commit log drives on my nodes keep slowly filling up.  I don't see
any errors in my logs that are indicating any issues that I can map to
this issue.



Is this how 1.1 is supposed to work now?  Previous versions seemed to
keep this drive at a minimum as it flushed.



/dev/mapper/mpathf 25G   21G  4.2G  83% /opt/cassandra/commitlog

Re: supercolumns with TTL columns not being compacted correctly

[Jonathan Ellis]

Correction: the first compaction after expiration + gcgs can remove
it, even if it hasn't been turned into a tombstone previously.

On Tue, May 22, 2012 at 9:37 AM, Jonathan Ellis  wrote:
> Additionally, it will always take at least two compaction passes to
> purge an expired column: one to turn it into a tombstone, and a second
> (after gcgs) to remove it.
>
> On Tue, May 22, 2012 at 9:21 AM, Yuki Morishita  wrote:
>> Data will not be deleted when those keys appear in other stables outside of
>> compaction. This is to prevent obsolete data from appearing again.
>>
>> yuki
>>
>> On Tuesday, May 22, 2012 at 7:37 AM, Pieter Callewaert wrote:
>>
>> Hi Samal,
>>
>>
>>
>> Thanks for your time looking into this.
>>
>>
>>
>> I force the compaction by using forceUserDefinedCompaction on only that
>> particular sstable. This gurantees me the new sstable being written only
>> contains the data from the old sstable.
>>
>> The data in the sstable is more than 31 days old and gc_grace is 0, but
>> still the data from the sstable is being written to the new one, while I am
>> 100% sure all the data is invalid.
>>
>>
>>
>> Kind regards,
>>
>> Pieter Callewaert
>>
>>
>>
>> From: samal [mailto:samalgo...@gmail.com]
>> Sent: dinsdag 22 mei 2012 14:33
>> To: user@cassandra.apache.org
>> Subject: Re: supercolumns with TTL columns not being compacted correctly
>>
>>
>>
>> Data will remain till next compaction but won't be available. Compaction
>> will delete old sstable create new one.
>>
>> On 22-May-2012 5:47 PM, "Pieter Callewaert" 
>> wrote:
>>
>> Hi,
>>
>>
>>
>> I’ve had my suspicions some months, but I think I am sure about it.
>>
>> Data is being written by the SSTableSimpleUnsortedWriter and loaded by the
>> sstableloader.
>>
>> The data should be alive for 31 days, so I use the following logic:
>>
>>
>>
>> int ttl = 2678400;
>>
>> long timestamp = System.currentTimeMillis() * 1000;
>>
>> long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl *
>> 1000));
>>
>>
>>
>> And using this to write it:
>>
>>
>>
>> sstableWriter.newRow(bytes(entry.id));
>>
>> sstableWriter.newSuperColumn(bytes(superColumn));
>>
>> sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs),
>> timestamp, ttl, expirationTimestampMS);
>>
>> sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverage),
>> timestamp, ttl, expirationTimestampMS);
>>
>> sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp, ttl,
>> expirationTimestampMS);
>>
>>
>>
>> This works perfectly, data can be queried until 31 days are passed, then no
>> results are given, as expected.
>>
>> But the data is still on disk until the sstables are being recompacted:
>>
>>
>>
>> One of our nodes (we got 6 total) has the following sstables:
>>
>> [cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
>>
>> -rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19
>> /data/MapData007/HOS-hc-125620-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17
>> /data/MapData007/HOS-hc-163141-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17
>> /data/MapData007/HOS-hc-172106-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50
>> /data/MapData007/HOS-hc-181902-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37
>> /data/MapData007/HOS-hc-191448-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41
>> /data/MapData007/HOS-hc-193842-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03
>> /data/MapData007/HOS-hc-196210-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20
>> /data/MapData007/HOS-hc-196779-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33
>> /data/MapData007/HOS-hc-58572-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59
>> /data/MapData007/HOS-hc-61630-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46
>> /data/MapData007/HOS-hc-63857-Data.db
>>
>> -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41
>> /data/MapData007/HOS-hc-87900-Data.db
>>
>>
>>
>> As you can see, the following files should be invalid:
>>
>> /data/MapData007/HOS-hc-58572-Data.db
>>
>> /data/MapData007/HOS-hc-61630-Data.db
>>
>> /data/MapData007/HOS-hc-63857-Data.db
>>
>>
>>
>> Because they are all written more than an moth ago. gc_grace is 0 so this
>> should also not be a problem.
>>
>>
>>
>> As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.db.
>>
>> Expected behavior should be an empty file is being written because all data
>> in the sstable should be invalid:
>>
>>
>>
>> Compactionstats is giving:
>>
>> compaction type    keyspace   column family bytes compacted bytes
>> total  progress
>>
>>    Compaction  MapData007 HOS 11518215662
>> 532355279724 2.16%
>>
>>
>>
>> And when I ls the directory I find this:
>>
>> -rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12
>> /data/MapData007/HOS-tmp-hc-196898-Data.db
>>
>>
>>
>> The sstable is being 1-o

schema fail to load on some nodes

[Yiming Sun]

Hi,

We are setting up a 6-node cassandra cluster within one data center.  3 in
rack1 and the other 3 in rack2.  The tokens are assigned alternating
between rack 1 and rack 2.  There is one seed node in each rack.  Below is
the ring:

r1-node1DC1 r1   0  (seed)
r2-node1DC1 r2  28356863910078205288614550619314017621
r1-node2DC1 r1  56713727820156410577229101238628035242
r2-node2DC1 r2  85070591730234615865843651857942052863
r1-node3DC1 r1  113427455640312821154458202477256070484
r2-node3DC1 r2  141784319550391026443072753096570088105
 (seed)

for our schema, we are using NetworkTopologyStrategy, setting DC1
replication factor to 2.   We loaded the schema via cassandra-cli on
r1-node1, and cassandra-cli seemed to have succeeded, but we later
discovered that some nodes didn't load the complete keyspace schema
definition.

On these nodes, show keyspaces from cassandra-cli would show the keyspace,
but column families are empty.  The system.log file also shows the
following stack trace:

 INFO [FlushWriter:4] 2012-05-22 16:56:13,677 Memtable.java (line 307)
Completed flushing
/r1-node2/cassandra-data/system/schema_columnfamilies/system-schema_columnfamilies-hc-6-Data.db
(1428 bytes)
ERROR [MigrationStage:1] 2012-05-22 16:56:13,706
AbstractCassandraDaemon.java (line 134) Exception in thread
Thread[MigrationStage:1,5,main]
java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
length = 1
at
org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:50)
at
org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
at
org.apache.cassandra.cql3.CFDefinition.(CFDefinition.java:59)
at
org.apache.cassandra.config.CFMetaData.updateCfDef(CFMetaData.java:1278)
at
org.apache.cassandra.config.CFMetaData.keyAlias(CFMetaData.java:221)
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1162)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1190)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:395)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
at
org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:248)
at
org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:163)
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:120)
at
org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:46)
... 18 more


Has anyone seen this before?  Thanks.

-- Y.

Replication factor

[Daning Wang]

Hello,

What is the pros and cons to choose different number of replication factor
in term of performance? if space is not a concern.

for example, if I have 4 nodes cluster in one data center, how can RF=2 vs
RF=4 affect read performance? If consistency level is ONE, looks reading
does not need to go to another hop to get data if RF=4, but it would do
more work on read repair in the background.

Can you share some insights about this?

Thanks in advance,

Daning

Re: Number of keyspaces

[aaron morton]

1 KS with 24 CF's will use roughly the same resources as 24 KS's with 1 CF. 
Each CF:

* loads the bloom filter for each SSTable
* samples the index for each sstable
* uses row and key cache
* has a current memtable and potentially memtables waiting to flush.
* had secondary index CF's

I would generally avoid a data model that calls for CF's to be added in 
response to new entities or new data. Older data will move moved to larger 
files, and not included in compaction for newer data.

Hope that helps. 

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 23/05/2012, at 3:31 AM, Luís Ferreira wrote:

> I have 24 keyspaces, each with a columns family and am considering changing 
> it to 1 keyspace with 24 CFs. Would this be beneficial?
> On May 22, 2012, at 12:56 PM, samal wrote:
> 
>> Not ideally, now cass has global memtable tuning. Each cf correspond to 
>> memory  in ram. Year wise cf means it will be in read only state for next 
>> year, memtable  will still consume ram.
>> 
>> On 22-May-2012 5:01 PM, "Franc Carter"  wrote:
>> On Tue, May 22, 2012 at 9:19 PM, aaron morton  
>> wrote:
>> It's more the number of CF's than keyspaces.
>> 
>> Oh - does increasing the number of Column Families affect performance ?
>> 
>> The design we are working on at the moment is considering using a Column 
>> Family per year. We were thinking this would isolate compactions to a more 
>> manageable size as we don't update previous years.
>> 
>> cheers
>>  
>> 
>> Cheers
>> 
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> 
>> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
>> 
>>> Yes, it does. However there's no real answer what's the limit: it depends 
>>> on your hardware and cluster configuration. 
>>> 
>>> You might even want to search the archives of this mailinglist, I remember 
>>> this has been asked before.
>>> 
>>> Cheers!
>>> 
>>> 2012/5/21 Luís Ferreira 
>>> Hi,
>>> 
>>> Does the number of keyspaces affect the overall cassandra performance?
>>> 
>>> 
>>> Cumprimentos,
>>> Luís Ferreira
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> With kind regards,
>>> 
>>> Robin Verlangen
>>> www.robinverlangen.nl
>>> 
>> 
>> 
>> 
>> 
>> -- 
>> Franc Carter | Systems architect | Sirca Ltd
>> franc.car...@sirca.org.au | www.sirca.org.au
>> Tel: +61 2 9236 9118 
>> Level 9, 80 Clarence St, Sydney NSW 2000
>> PO Box H58, Australia Square, Sydney NSW 1215
>> 
> 
> Cumprimentos,
> Luís Ferreira
> 
> 
> 

Re: Using EC2 ephemeral 4disk raid0 cause high iowait trouble

[Deno Vichas]

for what it's worth i've been having pretty good success using the 
Datastax AMIs.



On 5/17/2012 6:59 PM, koji Lin wrote:


Hi

We use amazon ami 3.2.12-3.2.4.amzn1.x86_64

and some of our data file are more than 10G

thanks

koji

2012-5-16 下午6:00 於 "aaron morton" > 寫道：


On Ubuntu ? Sounds like
http://wiki.apache.org/cassandra/FAQ#ubuntu_hangs

Cheers


-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 16/05/2012, at 2:13 PM, koji Lin wrote:


Hi

Our service already run cassandra 1.0 on 1x ec2 instances(with
ebs), and we saw lots of discussion talk about using 
ephemeral raid for better performance and consistent performance.


So we want to create new instance using 4 ephemeral raid0, and
copy the data from ebs to finally replace the old instance and
reduce some .

we create the xlarge instance with -b '/dev/sdb=ephemeral0' -b
'/dev/sdc=ephemeral1' -b '/dev/sdd=ephemeral2' -b
'/dev/sde=ephemeral3',

and use mdadm command like this  mdadm --create /dev/md0
--level=0 -c256 --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde

after copying file and start the cassandra(same token as old
instance it replaced).

we saw the read is really fast always keep 2xxm/sec, but system
load exceed 40, with high iowait, and lots of client get timeout
result. We guess maybe it's the problem of ec2 instance, so we
create another one with same setting to replace other machine
,but the result is same . Then we rollback to ebs with single
disk ,read speed keeps at 1xmb/sec but system becomes well
.(using ebs with 2 disks raid0 will keep at 2xmb/sec and higher
iowait then single disk ,but still works)

Is there anyone meet the same problem too ? or do we forget
something to configure?

thank you

koji

Re: schema fail to load on some nodes

[Tyler Hobbs]

Looks like this: https://issues.apache.org/jira/browse/CASSANDRA-4269

On Tue, May 22, 2012 at 4:10 PM, Yiming Sun  wrote:

> Hi,
>
> We are setting up a 6-node cassandra cluster within one data center.  3 in
> rack1 and the other 3 in rack2.  The tokens are assigned alternating
> between rack 1 and rack 2.  There is one seed node in each rack.  Below is
> the ring:
>
> r1-node1DC1 r1   0  (seed)
> r2-node1DC1 r2  28356863910078205288614550619314017621
> r1-node2DC1 r1  56713727820156410577229101238628035242
> r2-node2DC1 r2  85070591730234615865843651857942052863
> r1-node3DC1 r1  113427455640312821154458202477256070484
> r2-node3DC1 r2  141784319550391026443072753096570088105
>  (seed)
>
> for our schema, we are using NetworkTopologyStrategy, setting DC1
> replication factor to 2.   We loaded the schema via cassandra-cli on
> r1-node1, and cassandra-cli seemed to have succeeded, but we later
> discovered that some nodes didn't load the complete keyspace schema
> definition.
>
> On these nodes, show keyspaces from cassandra-cli would show the keyspace,
> but column families are empty.  The system.log file also shows the
> following stack trace:
>
>  INFO [FlushWriter:4] 2012-05-22 16:56:13,677 Memtable.java (line 307)
> Completed flushing
> /r1-node2/cassandra-data/system/schema_columnfamilies/system-schema_columnfamilies-hc-6-Data.db
> (1428 bytes)
> ERROR [MigrationStage:1] 2012-05-22 16:56:13,706
> AbstractCassandraDaemon.java (line 134) Exception in thread
> Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException:
> Input length = 1
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:50)
> at
> org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
> at
> org.apache.cassandra.cql3.CFDefinition.(CFDefinition.java:59)
> at
> org.apache.cassandra.config.CFMetaData.updateCfDef(CFMetaData.java:1278)
> at
> org.apache.cassandra.config.CFMetaData.keyAlias(CFMetaData.java:221)
> at
> org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1162)
> at
> org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1190)
> at
> org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
> at
> org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:395)
> at
> org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
> at
> org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:248)
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at
> java.nio.charset.CoderResult.throwException(CoderResult.java:260)
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:163)
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:120)
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:46)
> ... 18 more
>
>
> Has anyone seen this before?  Thanks.
>
> -- Y.
>



-- 
Tyler Hobbs
DataStax 

Re: schema fail to load on some nodes

[Yiming Sun]

It indeed looks almost the same, except in our case, we are only using
UTF8Type.  Hopefully when they release 1.1.1, all will be fixed.  Thanks
for making me aware of this issue, Tyler.

-- Y.

On Tue, May 22, 2012 at 7:28 PM, Tyler Hobbs  wrote:

> Looks like this: https://issues.apache.org/jira/browse/CASSANDRA-4269
>
>
> On Tue, May 22, 2012 at 4:10 PM, Yiming Sun  wrote:
>
>> Hi,
>>
>> We are setting up a 6-node cassandra cluster within one data center.  3
>> in rack1 and the other 3 in rack2.  The tokens are assigned alternating
>> between rack 1 and rack 2.  There is one seed node in each rack.  Below is
>> the ring:
>>
>> r1-node1DC1 r1   0  (seed)
>> r2-node1DC1 r2  28356863910078205288614550619314017621
>> r1-node2DC1 r1  56713727820156410577229101238628035242
>> r2-node2DC1 r2  85070591730234615865843651857942052863
>> r1-node3DC1 r1  113427455640312821154458202477256070484
>> r2-node3DC1 r2  141784319550391026443072753096570088105
>>  (seed)
>>
>> for our schema, we are using NetworkTopologyStrategy, setting DC1
>> replication factor to 2.   We loaded the schema via cassandra-cli on
>> r1-node1, and cassandra-cli seemed to have succeeded, but we later
>> discovered that some nodes didn't load the complete keyspace schema
>> definition.
>>
>> On these nodes, show keyspaces from cassandra-cli would show the
>> keyspace, but column families are empty.  The system.log file also shows
>> the following stack trace:
>>
>>  INFO [FlushWriter:4] 2012-05-22 16:56:13,677 Memtable.java (line 307)
>> Completed flushing
>> /r1-node2/cassandra-data/system/schema_columnfamilies/system-schema_columnfamilies-hc-6-Data.db
>> (1428 bytes)
>> ERROR [MigrationStage:1] 2012-05-22 16:56:13,706
>> AbstractCassandraDaemon.java (line 134) Exception in thread
>> Thread[MigrationStage:1,5,main]
>> java.lang.RuntimeException: java.nio.charset.MalformedInputException:
>> Input length = 1
>> at
>> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:50)
>> at
>> org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
>> at
>> org.apache.cassandra.cql3.CFDefinition.(CFDefinition.java:59)
>> at
>> org.apache.cassandra.config.CFMetaData.updateCfDef(CFMetaData.java:1278)
>> at
>> org.apache.cassandra.config.CFMetaData.keyAlias(CFMetaData.java:221)
>> at
>> org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1162)
>> at
>> org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1190)
>> at
>> org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
>> at
>> org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:395)
>> at
>> org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>> at
>> org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:248)
>> at
>> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:48)
>> at
>> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>> at
>> java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>> at
>> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:163)
>> at
>> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:120)
>> at
>> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:46)
>> ... 18 more
>>
>>
>> Has anyone seen this before?  Thanks.
>>
>> -- Y.
>>
>
>
>
> --
> Tyler Hobbs
> DataStax 
>
>

Re: Using EC2 ephemeral 4disk raid0 cause high iowait trouble

[koji Lin]

Hi
I think amazon ami is based on RHEL.

thank you

2012/5/21 aaron morton 

> Are you using the Ubuntu operating system ?
>
> Cheers
>
>   -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 18/05/2012, at 1:59 PM, koji Lin wrote:
>
> Hi
>
> We use amazon ami 3.2.12-3.2.4.amzn1.x86_64
>
> and some of our data file are more than 10G
>
> thanks
>
> koji
> 2012-5-16 下午6:00 於 "aaron morton"  寫道：
>
>> On Ubuntu ? Sounds like http://wiki.apache.org/cassandra/FAQ#ubuntu_hangs
>>
>> Cheers
>>
>>
>>   -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 16/05/2012, at 2:13 PM, koji Lin wrote:
>>
>> Hi
>>
>> Our service already run cassandra 1.0 on 1x ec2 instances(with ebs), and
>> we saw lots of discussion talk about using  ephemeral raid for better
>> performance and consistent performance.
>>
>> So we want to create new instance using 4 ephemeral raid0, and copy the
>> data from ebs to finally replace the old instance and reduce some .
>>
>> we create the xlarge instance with -b '/dev/sdb=ephemeral0' -b
>> '/dev/sdc=ephemeral1' -b '/dev/sdd=ephemeral2' -b '/dev/sde=ephemeral3',
>>
>> and use mdadm command like this  mdadm --create /dev/md0 --level=0 -c256
>> --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
>>
>> after copying file and start the cassandra(same token as old instance it
>> replaced).
>>
>> we saw the read is really fast always keep 2xxm/sec, but system load
>> exceed 40, with high iowait, and lots of client get timeout result. We
>> guess maybe it's the problem of ec2 instance, so we create another one with
>> same setting to replace other machine ,but the result is same . Then we
>> rollback to ebs with single disk ,read speed keeps at 1xmb/sec but system
>> becomes well .(using ebs with 2 disks raid0 will keep at 2xmb/sec and
>> higher iowait then single disk ,but still works)
>>
>> Is there anyone meet the same problem too ? or do we forget something to
>> configure?
>>
>> thank you
>>
>> koji
>>
>>
>>
>


-- 
Koji Lin
blog: http://www.javaworld.com.tw/roller/koji/

Re: Using EC2 ephemeral 4disk raid0 cause high iowait trouble

[koji Lin]

Hi
Thanks for your information, we will try that.

koji

2012/5/23 Deno Vichas 

>  for what it's worth i've been having pretty good success using the
> Datastax AMIs.
>
>
>
> On 5/17/2012 6:59 PM, koji Lin wrote:
>
> Hi
>
> We use amazon ami 3.2.12-3.2.4.amzn1.x86_64
>
> and some of our data file are more than 10G
>
> thanks
>
> koji
> 2012-5-16 下午6:00 於 "aaron morton"  寫道：
>
>> On Ubuntu ? Sounds like http://wiki.apache.org/cassandra/FAQ#ubuntu_hangs
>>
>>  Cheers
>>
>>
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>>  On 16/05/2012, at 2:13 PM, koji Lin wrote:
>>
>>  Hi
>>
>> Our service already run cassandra 1.0 on 1x ec2 instances(with ebs), and
>> we saw lots of discussion talk about using  ephemeral raid for better
>> performance and consistent performance.
>>
>> So we want to create new instance using 4 ephemeral raid0, and copy the
>> data from ebs to finally replace the old instance and reduce some .
>>
>> we create the xlarge instance with -b '/dev/sdb=ephemeral0' -b
>> '/dev/sdc=ephemeral1' -b '/dev/sdd=ephemeral2' -b '/dev/sde=ephemeral3',
>>
>> and use mdadm command like this  mdadm --create /dev/md0 --level=0 -c256
>> --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
>>
>> after copying file and start the cassandra(same token as old instance it
>> replaced).
>>
>> we saw the read is really fast always keep 2xxm/sec, but system load
>> exceed 40, with high iowait, and lots of client get timeout result. We
>> guess maybe it's the problem of ec2 instance, so we create another one with
>> same setting to replace other machine ,but the result is same . Then we
>> rollback to ebs with single disk ,read speed keeps at 1xmb/sec but system
>> becomes well .(using ebs with 2 disks raid0 will keep at 2xmb/sec and
>> higher iowait then single disk ,but still works)
>>
>> Is there anyone meet the same problem too ? or do we forget something to
>> configure?
>>
>> thank you
>>
>> koji
>>
>>
>>
>


-- 
Koji Lin
blog: http://www.javaworld.com.tw/roller/koji/

Re: exception when cleaning up...

[Boris Yen]

Hi Aaron, Rob,

Thanks for the information, I will try it.

Regards,
Boris

On Tue, May 22, 2012 at 11:47 PM, Rob Coli  wrote:

> On Tue, May 22, 2012 at 3:00 AM, aaron morton 
> wrote:
> > 1) Isolating the node from the cluster to stop write activity. You can
> > either start the node with the -Dcassandra.join_ring=false  JVM option or
> > use nodetool disablethrift and disablegossip to stop writes. Note that
> this
> > will not stop existing Hinted Handoff sessions which target the node.
>
> As a result of the last caveat here, I recommend either restarting the
> node with the join_ring as false or using iptables to firewall off
> ports 7000 and 9160. If you want to be sure that you have stopped
> write activity right now, nuking these ports from orbit is the only
> way to be sure. disablethrift/disablegossip as currently implemented
> are not sufficient for this goal.
>
> https://issues.apache.org/jira/browse/CASSANDRA-4162
>
> =Rob
>
> --
> =Robert Coli
> AIM>ALK - rc...@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb
>

Re: Number of keyspaces

[Franc Carter]

On Wed, May 23, 2012 at 7:42 AM, aaron morton wrote:

> 1 KS with 24 CF's will use roughly the same resources as 24 KS's with 1
> CF. Each CF:
>
> * loads the bloom filter for each SSTable
> * samples the index for each sstable
> * uses row and key cache
> * has a current memtable and potentially memtables waiting to flush.
> * had secondary index CF's
>
> I would generally avoid a data model that calls for CF's to be added in
> response to new entities or new data. Older data will move moved to larger
> files, and not included in compaction for newer data.
>

We were thinking of doing a major compaction after each year is 'closed
off'. This would mean that compactions for the current year were dealing
with a smaller amount of data and hence be faster and have less impact on a
day-to-day basis. Our query patterns will only infrequently cross year
boundaries.

Are we being naive ?

cheers


>
> Hope that helps.
>
>   -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 23/05/2012, at 3:31 AM, Luís Ferreira wrote:
>
> I have 24 keyspaces, each with a columns family and am considering
> changing it to 1 keyspace with 24 CFs. Would this be beneficial?
> On May 22, 2012, at 12:56 PM, samal wrote:
>
> Not ideally, now cass has global memtable tuning. Each cf correspond to
> memory  in ram. Year wise cf means it will be in read only state for next
> year, memtable  will still consume ram.
> On 22-May-2012 5:01 PM, "Franc Carter"  wrote:
>
>> On Tue, May 22, 2012 at 9:19 PM, aaron morton wrote:
>>
>>> It's more the number of CF's than keyspaces.
>>>
>>
>> Oh - does increasing the number of Column Families affect performance ?
>>
>> The design we are working on at the moment is considering using a Column
>> Family per year. We were thinking this would isolate compactions to a more
>> manageable size as we don't update previous years.
>>
>> cheers
>>
>>
>>>
>>> Cheers
>>>
>>>   -
>>> Aaron Morton
>>> Freelance Developer
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 22/05/2012, at 6:58 PM, R. Verlangen wrote:
>>>
>>> Yes, it does. However there's no real answer what's the limit: it
>>> depends on your hardware and cluster configuration.
>>>
>>> You might even want to search the archives of this mailinglist, I
>>> remember this has been asked before.
>>>
>>> Cheers!
>>>
>>> 2012/5/21 Luís Ferreira 
>>>
 Hi,

 Does the number of keyspaces affect the overall cassandra performance?


 Cumprimentos,
 Luís Ferreira




>>>
>>>
>>> --
>>> With kind regards,
>>>
>>> Robin Verlangen
>>> www.robinverlangen.nl
>>>
>>>
>>>
>>
>>
>> --
>> *Franc Carter* | Systems architect | Sirca Ltd
>>  
>> franc.car...@sirca.org.au | www.sirca.org.au
>> Tel: +61 2 9236 9118
>>  Level 9, 80 Clarence St, Sydney NSW 2000
>> PO Box H58, Australia Square, Sydney NSW 1215
>>
>>
>  Cumprimentos,
> Luís Ferreira
>
>
>
>
>


-- 

*Franc Carter* | Systems architect | Sirca Ltd
 

franc.car...@sirca.org.au | www.sirca.org.au

Tel: +61 2 9236 9118

Level 9, 80 Clarence St, Sydney NSW 2000

PO Box H58, Australia Square, Sydney NSW 1215

Confusion regarding the terms "replica" and "replication factor"

[java jalwa]

Hi all,
  I am a bit confused regarding the terms "replica" and
"replication factor". Assume that I am using RandomPartitioner and
NetworkTopologyStrategy for replica placement.
>From what I understand, with a RandomPartitioner, a row key will
always be hashed and be stored on the node that owns the range to
which the key is mapped.
http://www.datastax.com/docs/1.0/cluster_architecture/replication#networktopologystrategy.
The example here, talks about having 2 data centers and a replication
factor of 4 with 2 replicas in each datacenter, so the strategy is
configured as DC1:2 and DC2:2. Now suppose I add another datacenter
DC3, and do not change the NetworkTopologyStrategy.
Now if a row key hash is mapped to a range owned by a node in DC3,
will the Node in DC3 still store the key as determined by the
partitioner and then walk the ring and store 2 replicas each in DC1
and DC2 ? Will that mean that I will then have 5 replicas in the
cluster and not 4 ? Or will the co-ordinator node be aware of the
replica placement strategy,
and override the partitioner's decision and walk the ring until it
first encounters a node in DC1 or DC2 ? and then place the remaining
replicas ?

Thanks.

Re: Cassandra 0.8.5: Column name mystery in create column family command

[samal]

I an not able to reproduce this in cli.
On 22-May-2012 8:12 PM, "Roshan Dawrani"  wrote:

> Can you please let me know why? Because I have created very similar column
> familes many times with comparator = BytesType, and never run into this
> issue before.
>
> Here is an example:
>
> 
> ColumnFamily: Client
>   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
>   Default column value validator: org.apache.cassandra.db.marshal.BytesType
>   Columns sorted by: org.apache.cassandra.db.marshal.BytesType
>   ...
>   ...
>   Built indexes: [Client.ACUserIdIdx]
>   Column Metadata:
> Column Name: userid (757365726964)
>   Validation Class: org.apache.cassandra.db.marshal.LexicalUUIDType
>   Index Name: ACUserIdIdx
>   Index Type: KEYS
> 
>
> On Tue, May 22, 2012 at 6:16 PM, samal  wrote:
>
>> Change your comparator to utf8type.
>> On 22-May-2012 4:32 PM, "Roshan Dawrani"  wrote:
>>
>>> Hi,
>>>
>>> I use Cassandra 0.8.5 and am suddenly noticing some strange behavior. I
>>> run a "create column family" command with some column meta-data and it runs
>>> fine, but when I do "describe keyspace", it shows me different column names
>>> for those index columns.
>>>
>>> a) Here is what I run:
>>> "create column family UserTemplate with comparator=BytesType and
>>> column_metadata=[{*column_name: userid*, validation_class: UTF8Type,
>>> index_type: KEYS, index_name: TemplateUserIdIdx}, {*column_name: type*,
>>> validation_class: UTF8Type, index_type: KEYS, index_name:
>>> TemplateTypeIdx}];"
>>>
>>> b) This is what "describe keyspace" shows:
>>> ColumnFamily: UserTemplate
>>>   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
>>>   ...
>>>   ...
>>>   Built indexes: [UserTemplate.TemplateTypeIdx,
>>> UserTemplate.TemplateUserIdIdx]
>>>   Column Metadata:
>>> *Column Name: ff*
>>>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>>>   Index Name: TemplateUserIdIdx
>>>   Index Type: KEYS
>>> *Column Name: 0dfffaff*
>>>   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>>>   Index Name: TemplateTypeIdx
>>>   Index Type: KEYS
>>>
>>> Does anyone see why this must be happening? I have created many such
>>> column families before and never run into this issue.
>>>
>>> --
>>> Roshan
>>> http://roshandawrani.wordpress.com/
>>>
>>>
>
>
> --
> Roshan
> http://roshandawrani.wordpress.com/
>
>

Re: supercolumns with TTL columns not being compacted correctly

[samal]

Thanks I didn't knew  two stage removal process.
On 23-May-2012 2:20 AM, "Jonathan Ellis"  wrote:

> Correction: the first compaction after expiration + gcgs can remove
> it, even if it hasn't been turned into a tombstone previously.
>
> On Tue, May 22, 2012 at 9:37 AM, Jonathan Ellis  wrote:
> > Additionally, it will always take at least two compaction passes to
> > purge an expired column: one to turn it into a tombstone, and a second
> > (after gcgs) to remove it.
> >
> > On Tue, May 22, 2012 at 9:21 AM, Yuki Morishita 
> wrote:
> >> Data will not be deleted when those keys appear in other stables
> outside of
> >> compaction. This is to prevent obsolete data from appearing again.
> >>
> >> yuki
> >>
> >> On Tuesday, May 22, 2012 at 7:37 AM, Pieter Callewaert wrote:
> >>
> >> Hi Samal,
> >>
> >>
> >>
> >> Thanks for your time looking into this.
> >>
> >>
> >>
> >> I force the compaction by using forceUserDefinedCompaction on only that
> >> particular sstable. This gurantees me the new sstable being written only
> >> contains the data from the old sstable.
> >>
> >> The data in the sstable is more than 31 days old and gc_grace is 0, but
> >> still the data from the sstable is being written to the new one, while
> I am
> >> 100% sure all the data is invalid.
> >>
> >>
> >>
> >> Kind regards,
> >>
> >> Pieter Callewaert
> >>
> >>
> >>
> >> From: samal [mailto:samalgo...@gmail.com]
> >> Sent: dinsdag 22 mei 2012 14:33
> >> To: user@cassandra.apache.org
> >> Subject: Re: supercolumns with TTL columns not being compacted correctly
> >>
> >>
> >>
> >> Data will remain till next compaction but won't be available. Compaction
> >> will delete old sstable create new one.
> >>
> >> On 22-May-2012 5:47 PM, "Pieter Callewaert" <
> pieter.callewa...@be-mobile.be>
> >> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I’ve had my suspicions some months, but I think I am sure about it.
> >>
> >> Data is being written by the SSTableSimpleUnsortedWriter and loaded by
> the
> >> sstableloader.
> >>
> >> The data should be alive for 31 days, so I use the following logic:
> >>
> >>
> >>
> >> int ttl = 2678400;
> >>
> >> long timestamp = System.currentTimeMillis() * 1000;
> >>
> >> long expirationTimestampMS = (long) ((timestamp / 1000) + ((long) ttl *
> >> 1000));
> >>
> >>
> >>
> >> And using this to write it:
> >>
> >>
> >>
> >> sstableWriter.newRow(bytes(entry.id));
> >>
> >> sstableWriter.newSuperColumn(bytes(superColumn));
> >>
> >> sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs),
> >> timestamp, ttl, expirationTimestampMS);
> >>
> >> sstableWriter.addExpiringColumn(nameCov,
> bytes(entry.observationCoverage),
> >> timestamp, ttl, expirationTimestampMS);
> >>
> >> sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp,
> ttl,
> >> expirationTimestampMS);
> >>
> >>
> >>
> >> This works perfectly, data can be queried until 31 days are passed,
> then no
> >> results are given, as expected.
> >>
> >> But the data is still on disk until the sstables are being recompacted:
> >>
> >>
> >>
> >> One of our nodes (we got 6 total) has the following sstables:
> >>
> >> [cassandra@bemobile-cass3 ~]$ ls -hal /data/MapData007/HOS-* | grep G
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 103G May  3 03:19
> >> /data/MapData007/HOS-hc-125620-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17
> >> /data/MapData007/HOS-hc-163141-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra  25G May 15 06:17
> >> /data/MapData007/HOS-hc-172106-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra  25G May 17 19:50
> >> /data/MapData007/HOS-hc-181902-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra  21G May 21 07:37
> >> /data/MapData007/HOS-hc-191448-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41
> >> /data/MapData007/HOS-hc-193842-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03
> >> /data/MapData007/HOS-hc-196210-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20
> >> /data/MapData007/HOS-hc-196779-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33
> >> /data/MapData007/HOS-hc-58572-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59
> >> /data/MapData007/HOS-hc-61630-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46
> >> /data/MapData007/HOS-hc-63857-Data.db
> >>
> >> -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41
> >> /data/MapData007/HOS-hc-87900-Data.db
> >>
> >>
> >>
> >> As you can see, the following files should be invalid:
> >>
> >> /data/MapData007/HOS-hc-58572-Data.db
> >>
> >> /data/MapData007/HOS-hc-61630-Data.db
> >>
> >> /data/MapData007/HOS-hc-63857-Data.db
> >>
> >>
> >>
> >> Because they are all written more than an moth ago. gc_grace is 0 so
> this
> >> should also not be a problem.
> >>
> >>
> >>
> >> As a test, I use forceUserSpecifiedCompaction on the
> HOS-hc-61630-Data.db.
> >>
> >> Expected behavior should be an empty file is being written because all
> data

how to get list of snapshots

[Илья Шипицин]

Hello!

I'm about to schedule backups in the following way

a) snapshots are done daily
b) increment backups are enabled

so, backup will be consistent, very old snapshots must be removed (I guess,
a week depth should be enough).

couple of questions:

1) is there any good guide for scheduling backups ?
2) is there way to get list of snapshots ? (without "ls" in directory)

Cheers,
Ilya Shipitsin

Re: Using EC2 ephemeral 4disk raid0 cause high iowait trouble

[Tamar Fraenkel]

Did you upgrade DataStax AMIs? Did you add a node to an existing ring?
Thanks
*Tamar Fraenkel *
Senior Software Engineer, TOK Media

[image: Inline image 1]

ta...@tok-media.com
Tel:   +972 2 6409736
Mob:  +972 54 8356490
Fax:   +972 2 5612956





On Wed, May 23, 2012 at 2:00 AM, Deno Vichas  wrote:

>  for what it's worth i've been having pretty good success using the
> Datastax AMIs.
>
>
>
> On 5/17/2012 6:59 PM, koji Lin wrote:
>
> Hi
>
> We use amazon ami 3.2.12-3.2.4.amzn1.x86_64
>
> and some of our data file are more than 10G
>
> thanks
>
> koji
> 2012-5-16 下午6:00 於 "aaron morton"  寫道：
>
>> On Ubuntu ? Sounds like http://wiki.apache.org/cassandra/FAQ#ubuntu_hangs
>>
>>  Cheers
>>
>>
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>>  On 16/05/2012, at 2:13 PM, koji Lin wrote:
>>
>>  Hi
>>
>> Our service already run cassandra 1.0 on 1x ec2 instances(with ebs), and
>> we saw lots of discussion talk about using  ephemeral raid for better
>> performance and consistent performance.
>>
>> So we want to create new instance using 4 ephemeral raid0, and copy the
>> data from ebs to finally replace the old instance and reduce some .
>>
>> we create the xlarge instance with -b '/dev/sdb=ephemeral0' -b
>> '/dev/sdc=ephemeral1' -b '/dev/sdd=ephemeral2' -b '/dev/sde=ephemeral3',
>>
>> and use mdadm command like this  mdadm --create /dev/md0 --level=0 -c256
>> --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
>>
>> after copying file and start the cassandra(same token as old instance it
>> replaced).
>>
>> we saw the read is really fast always keep 2xxm/sec, but system load
>> exceed 40, with high iowait, and lots of client get timeout result. We
>> guess maybe it's the problem of ec2 instance, so we create another one with
>> same setting to replace other machine ,but the result is same . Then we
>> rollback to ebs with single disk ,read speed keeps at 1xmb/sec but system
>> becomes well .(using ebs with 2 disks raid0 will keep at 2xmb/sec and
>> higher iowait then single disk ,but still works)
>>
>> Is there anyone meet the same problem too ? or do we forget something to
>> configure?
>>
>> thank you
>>
>> koji
>>
>>
>>
>
<>

Re: Correct way to set strategy options in cqlsh?

[Romain HARDOUIN]

You *must* remove the hyphen.
According to the csql 2.0 documentation, here is the correct syntax to 
create keyspace:

 ::= "CREATE" "KEYSPACE" 
 "WITH"  "=" 
 ( "AND"  "="  )*
;
 ::= 
   |  ":" 
   |  ":" 
   ;
 ::= 
  | 
  | 
  ;

The string "strategy_options:us-west=1;" matches the following syntax:

 ":"  = 

Thus, "us-west" is an *identifier*, and again according to the 
documentation:
An  is a letter followed by any sequence of letters, digits, 
or the underscore (_).