We are running Cassandra 1.2.5
We have 8 nodes cluster, and we removed one machine from cluster and try to
add it back(the purpose is we are using vnodes, some node has more tokens
so by rejoining this machine we hope it could get some loads from the busy
machines). But we got following exception
Does Cassandra put keys in key cache during the write path?
If I have two tables, the key cache for the first table was warmed up
nicely, and I want to insert millions rows in the second table, and there
is no read on the second table yet, will that affect cache hit ratio for
the first table?
Tha
How to move a token to another node on 1.2.x? I have tried move command,
[cassy@dsat103.e1a ~]$ nodetool move 168755834953206242653616795390304335559
Exception in thread "main" java.io.IOException: target token
168755834953206242653616795390304335559 is already owned by another node.
at
org.apache
Thanks. actually I forgot to mention it is multi-center environment and we
have dynamic snitch disabled. because we saw some performance impact on the
multi-center environment.
On Wed, Oct 30, 2013 at 11:12 AM, Piavlo wrote:
> On 10/30/2013 02:06 AM, Daning Wang wrote:
>
>> We
We are running 1.2.5 on 8 nodes(256 tokens). all the nodes are running on
same type of machine. and db size is about same. but recently we checked
ReadCount stats through jmx, and found that some nodes got 3 times change
rate(we have calculated the changes per minute) than others.
We are using h
We noticed that key cache could not be fully populated, we have set the key
cache size to 1024M.
key_cache_size_in_mb: 1024
But none of nodes showed the cache capacity is 1G, we have recently
upgraded to 1.2.5, could be an issue in that version?
Token: (invoke with -T/--tokens to see
How does dynamic snitch work with EC2MultiRegionSnitch? Can dynamic routing
only happen in one data center? We don't wan to have the requests routed to
another center even nodes are idle in other side since the network could be
slow.
Thanks in advance,
Daning
gt;>
>>> On Wed, Jun 12, 2013 at 4:56 AM, Alain RODRIGUEZ wrote:
>>>
>>>> "counter will replicate to all replicas during write regardless the
>>>> consistency level"
>>>>
>>>> I that the normal behavior or a bug ?
>>>&g
gt; Thanks !
>
>
> On Fri, Jun 7, 2013 at 11:49 PM, Daning Wang wrote:
>
>> We have deployed multi-center but got performance issue. When the nodes
>> on other center are up, the read response time from clients is 4 or 5 times
>> higher. when we take those nodes down,
We have deployed multi-center but got performance issue. When the nodes on
other center are up, the read response time from clients is 4 or 5 times
higher. when we take those nodes down, the response time becomes
normal(compare to the time before we changed to multi-center).
We have high volume on
We have multi-center deployment. data from some tables we don't want to
sync to other center. could we set replication factor to 0 on other data
center? what is the best to way for not syncing some data in a cluster?
Thanks in advance,
Daning
our mutation
> requests will dictate how long it will take the second datacentre to get in
> sync with the primary datacentre.
>
>
> I've probably missed something but there are plenty of intelligent people
> in this mailing list to fill the blanks :)
>
> Thanks
>
>
Hi all,
We are deploying Cassandra on two data centers. there is slower network
connection between data centers.
Looks casandra should use internal ip to communicate with nodes in the same
data center, and public ip to talk to nodes in other data center. We know
VPN is a solution, but want to kno
Hi Guys,
What is the cassandra solution for remote backup besides multi-center? I
hope I can do incremental backup to remote database center.
Thanks,
Daning
Hi All,
We have 8 nodes cluster(replication factor is 3), about 50G data on each
node. we need to change the cluster to multi-center environment(to EC2).
the data need to have one replica on ec2.
Here is the plan,
- Change cluster config to mult-center.
- Add 2 or 3 nodes in another center, whic
I tried to add new node to ring, it is supposed to be fast in 1.2(256
tokens on each node), but it is 8+ hours now. after showing bootstraping,
now cpu usage is very low, I turned on debug, it shows applying mutation.
is that normal?
INFO [main] 2013-03-15 08:36:44,530 StorageService.java (line
node".
>
> So I would let "#num_tokens" commented in the cassandra.yaml and would
> set the initial_token at the same value than in the pre-C*1.2.x-uprage
> configuration.
>
> Alain
>
>
> 2013/2/14 Daning Wang
>
>> Thanks Aaron and Manu.
>>
>> Si
ble virtual nodes. Which are not necessary to run 1.2.
>>
>> Cheers
>>
>>
>>-
>> Aaron Morton
>> Freelance Cassandra Developer
>> New Zealand
>>
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 13/02/2013, at 8:02 AM, Daning W
to 1.2 first, then do the shuffle when things are
> stable.
>
> Cheers
>
> -
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 12/02/2013, at 2:55 PM, Daning Wang wrote:
>
---
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 3/02/2013, at 11:32 PM, Manu Zhang wrote:
>
> On Sun 03 Feb 2013 05:45:56 AM CST, Daning Wang wrote:
>
> I'd like to upgrade from 1.
We have 8 nodes cluster in Casandra 1.1.0, with replication factor is 3. We
found that when you just insert data, not only WriteCount increases, the
ReadCount also increases.
How could this happen? I am under the impression that readCount only counts
the reads from client.
Thanks,
Daning
I'd like to upgrade from 1.1.6 to 1.2.1, one big feature in 1.2 is that it
can have multiple tokens in one node. but there is only one token in 1.1.6.
how can I upgrade to 1.2.1 then breaking the token to take advantage of
this feature? I went through this doc but it does not say how to change the
ou are not using QUOURM / QUOURM you maybe getting inconsistent results
> now.
>
> Cheers
>
> -
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 29/01/2013, at 9:51 AM,
We were using SimpleAuthenticator on 1.1.x, it worked fine.
While testing 1.2, I have put classes under example/simple_authentication
in a jar and copy to lib directory, the class is loaded. however, when I
try to connect with correct user/password, it gives me error
./cqlsh s2.dsat103-e1a -u xxx
I add a new node to ring(version 1.1.6), after more than 30 hours, it is
still in the 'Joining' state
Address DC RackStatus State Load
Effective-Ownership Token
141784319550391026443072753096570088105
10.28.78.123datacenter1 rack1 Up Normal
d you read at CL ONE you will always be
> reading locally. But with a low consistency.
>
> If you read with QUORUM when RF == number of nodes you will still get some
> performance benefit from the data being read locally.
>
> ** **
>
> Cheers****
>
Hello,
What is the pros and cons to choose different number of replication factor
in term of performance? if space is not a concern.
for example, if I have 4 nodes cluster in one data center, how can RF=2 vs
RF=4 affect read performance? If consistency level is ONE, looks reading
does not need to
tool repair as hints for other
> CF's may have been dropped.
>
> Cheers
>
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 16/05/2012, at 2:27 AM, Daning Wang wrote:
>
> We got exception &qu
We got exception "UnserializableColumnFamilyException: Couldn't find
cfId=1075" in the log of one node, describe cluster showed all the nodes in
same schema version. how to fix this problem? did repair but looks does not
work, haven't try scrub yet.
We are on v1.0.3
ERROR [HintedHandoff:1631] 201
gt; 0 0
>
> Looks fine.
>
> Hope that helps.
>
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 10/04/2012, at 8:08 AM, Daning Wan
gt; Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 6/04/2012, at 5:30 AM, Daning Wang wrote:
>
> > Hi all,
> >
> > We are using Hector and ofter we see lots of timeout exception in the
> log, I know that the hector can failover to othe
Hi all,
We are using Hector and ofter we see lots of timeout exception in the log,
I know that the hector can failover to other node, but I want to reduce the
number of timeouts.
any hector parameter I should change to reduce this error?
also, on the server side, any kind of tunning need to do f
be marked as UNREACHABLE if it is DOWN or if it did not
> respond in time.
>
> Cheers
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 23/03/2012, at 11:29 AM, Daning Wang wrote:
>
> Thanks Aaron. when I
gt;
> use cassandra-cli and run describe cluster; to see how many schema
> versions you have.
>
> Cheers
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 22/03/2012, at 6:27 AM, Daning Wang wrote:
>
> an
Hi,
How to find a column family from a cfId? I got a bunch of exception, want
to find out which CF has problem.
java.io.IOError:
org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find
cfId=1744830464
at
org.apache.cassandra.service.AbstractRowResolver.preprocess(Abstra
and we are on 0.8.6.
On Wed, Mar 21, 2012 at 10:24 AM, Daning Wang wrote:
> Hi All,
>
>
> We got lots of Exception in the log, and later the server crashed. any
> idea what is happening and how to fix it?
>
> ERROR [RequestResponseStage:4] 20
Hi All,
We got lots of Exception in the log, and later the server crashed. any idea
what is happening and how to fix it?
ERROR [RequestResponseStage:4] 2012-03-21 04:16:30,482
AbstractCassandraDaemon.java (line 139) Fatal exception in thread
Thread[RequestResponseStage:4,5,main]
java.io.IOError:
hosts every xx seconds and recover connection.
>
> Regards,
> Maciej
>
>
> On Mon, Mar 5, 2012 at 8:13 PM, Daning Wang wrote:
>
>> I just got this error ": All host pools marked down. Retry burden pushed
>> out to client." in a few clients recently, client co
I just got this error ": All host pools marked down. Retry burden pushed
out to client." in a few clients recently, client could not recover, we
have to restart client application. we are using 0.8.0.3 hector.
At that time we did compaction for a CF, it takes several hours, server
was busy. But
> nodetool -h hostname move newtoken
>>
>> Then, once you've update each of your tokens that you want to move,
>> you'll want to run
>> nodetool -h hostname cleanup
>>
>> That will remove the no-longer necessary tokens from your smaller
>> machine
Hi All,
We have 5 nodes cluster(on 0.8.6), but two machines are slower and have
less memory, so the performance was not good on those two machines for
large volume traffic.I want to move some data from slower machine to faster
machine to ease some load, the token ring will not be equally balanced
using RandomPartitioner? Are you reading using indexes?
>
> First thing you can do is compare iostat -x output between the 2 nodes
> to rule out any io issues assuming your read requests are equally
> balanced.
>
> On Fri, Jan 6, 2012 at 10:11 AM, Daning Wang wrote:
> > Hi all,
&
Hi all,
We have 5 nodes cluster(0.8.6), but the performance from one node is way
behind others, I checked tpstats, It always show non-zero pending
ReadStage, I don't see this problem on other nodes.
What caused the problem? I/O? Memory? Cpu usage is still low. How to fix
this problem?
~/bin/node
Hi All,
We are getting TimedOutException() when inserting data into Cassandra, it
was working fine for a few months, but suddenly got this problem. I have
increase rpc_timout_in_ms to 3, but it still timed out in 30 secs.
I turned on debug, I saw many of this error in the log
DEBUG [pool-2-t
I have Cassandra server which has JVM setting -Xms4G -Xmx4G, but why top
reports 15G RES memory and 11G SHR memory usage? I understand that -Xmx4G
is only for the heap size. but it is strange that OS reports 2.5 times
memory usage. Are there a lot of memory used by JNI? Please help to explain
this.
gt; Cheers
>
>
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/09/2011, at 3:27 AM, Daning Wang wrote:
>
> Jonathan/Aaron,
>
> Thank you guy's reply, I will change GCGracePeriod
Jonathan/Aaron,
Thank you guy's reply, I will change GCGracePeriod to 1 day to see what will
happen.
Is there a way to purge tombstones at anytime? because if tombstones affect
performance, we want them to be purged right away, not after GCGracePeriod.
We know all the nodes are up, and we can do
How is the performance of ByteOrderedPartitioner, compared to
RandomPartitioner? the perforamnce when getting data with single key, does
it use same algorithm?
I have read that the downside of ByteOrderedPartitioner is creating hotspot.
But if I have 4 nodes and I set RF to 4, that will replicate
We try to implement an ordered queue system in Cassandra(ver 0.8.5). In
initial design we use a row as queue, a column for each item in queue.
that means creating new column when inserting item and delete column when
top item is popped. Since columns are sorted in Cassandra we got the ordered
qu
49 matches
Mail list logo
