unable to find sufficient sources for streaming range

2014-07-02 Thread Daning Wang
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:362) at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88) at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:513) Daning

Bulk writes and key cache

2014-02-03 Thread Daning Wang
? Thanks, Daning

Move token to another node on 1.2.x

2013-11-07 Thread Daning Wang
lhost move 168755834953206242653616795390304335560 This node has more than one token and cannot be moved thusly We don't want to use cassandra-shuffle, because it put too much load on the server. we just want to move some tokens. Thanks, Daning

Re: ReadCount change rate is different across nodes

2013-10-30 Thread Daning Wang
Thanks. actually I forgot to mention it is multi-center environment and we have dynamic snitch disabled. because we saw some performance impact on the multi-center environment. On Wed, Oct 30, 2013 at 11:12 AM, Piavlo wrote: > On 10/30/2013 02:06 AM, Daning Wang wrote: > >> We

ReadCount change rate is different across nodes

2013-10-29 Thread Daning Wang
hector on client side, and clients are connecting to all the servers, we checked open connections on each server, the numbers are about same. What could cause this problem, how to debug this? Thanks in advance, Daning

Key cache size

2013-09-04 Thread Daning Wang
We noticed that key cache could not be fully populated, we have set the key cache size to 1024M. key_cache_size_in_mb: 1024 But none of nodes showed the cache capacity is 1G, we have recently upgraded to 1.2.5, could be an issue in that version? Token: (invoke with -T/--tokens to see

Dynamic Snitch and EC2MultiRegionSnitch

2013-07-01 Thread Daning Wang
How does dynamic snitch work with EC2MultiRegionSnitch? Can dynamic routing only happen in one data center? We don't wan to have the requests routed to another center even nodes are idle in other side since the network could be slow. Thanks in advance, Daning

Re: Multiple data center performance

2013-06-12 Thread Daning Wang
other DC is shutdown. Thanks, Daning On Wed, Jun 12, 2013 at 7:48 AM, Sylvain Lebresne wrote: > > Is there something special of this kind regarding counters over multiDC ? >> > > No. Counters behave exactly as other writes as far the consistency level > is concerned. >

Re: Multiple data center performance

2013-06-11 Thread Daning Wang
ption replicate_on_write on table. If you turn that off for counter might have better performance. but you are on high risk to lose data and create inconsistency. I did not try this option. Daning On Sat, Jun 8, 2013 at 6:53 AM, srmore wrote: > I am seeing the similar behavior, in my case I have 2

Multiple data center performance

2013-06-07 Thread Daning Wang
-- Address Load Tokens Owns (effective) Host ID Rack DN xx.xx.xx.19958.39 GB 256 50.0% 6954754a-e9df-4b3c-aca7-146b938515d8 rac1 DN xx.xx.xx..61 33.79 GB 256 50.0% 91b8d510-966a-4f2d-a666-d7edbe986a1c rac1 Thank you in advance, Daning

replication factor is zero

2013-06-06 Thread Daning Wang
We have multi-center deployment. data from some tables we don't want to sync to other center. could we set replication factor to 0 on other data center? what is the best to way for not syncing some data in a cluster? Thanks in advance, Daning

Re: Cassandra multi-datacenter

2013-05-03 Thread Daning Wang
dress? or other solutions? Is that easy to write a new Snitch for this? Thanks, Daning On Thu, May 2, 2013 at 2:31 PM, aaron morton wrote: > Look at the broadcast_address in the yaml file > Cheers > - > Aaron Morton > Freelance Cassandra Consultant > New Z

Cassandra multi-datacenter

2013-05-02 Thread Daning Wang
know if there is other idea. Thanks in advance, Daning

Cassandra remote backup solution

2013-04-25 Thread Daning Wang
Hi Guys, What is the cassandra solution for remote backup besides multi-center? I hope I can do incremental backup to remote database center. Thanks, Daning

How to change existing cluster to multi-center

2013-04-25 Thread Daning Wang
before, please share some light, Thanks in advance, Daning

Cassandra 1.2.1 adding new node

2013-03-15 Thread Daning Wang
) Applying mutation DEBUG [MutationStage:602] 2013-03-15 14:32:33,643 RowMutationVerbHandler.java (line 40) Applying mutation DEBUG [MutationStage:602] 2013-03-15 14:32:33,643 AbstractSimplePerColumnSecondaryIndex.java (line 118) applying index row us.countryproducts.safestchina.com Thanks, Daning

Re: Upgrade to Cassandra 1.2

2013-02-14 Thread Daning Wang
node". > > So I would let "#num_tokens" commented in the cassandra.yaml and would > set the initial_token at the same value than in the pre-C*1.2.x-uprage > configuration. > > Alain > > > 2013/2/14 Daning Wang > >> Thanks Aaron and Manu. >> >> Si

Re: Upgrade to Cassandra 1.2

2013-02-14 Thread Daning Wang
Thanks Aaron and Manu. Since we are using 1.1, there is no num_taken parameter. when I upgrade to 1.2, should I set num_token=1 to start up, or I can set to other numbers? Daning On Tue, Feb 12, 2013 at 3:45 PM, Manu Zhang wrote: > num_tokens is only used at bootstrap > > I t

Re: Upgrade to Cassandra 1.2

2013-02-12 Thread Daning Wang
No, I did not run shuffle since the upgrade was not successful. what do you mean "reverting the changes to num_tokens and inital_token"? set num_tokens=1? initial_token should be ignored since it is not bootstrap. right? Thanks, Daning On Tue, Feb 12, 2013 at 10:52 AM, aaron mo

Re: Upgrade to Cassandra 1.2

2013-02-11 Thread Daning Wang
373 rack1 UN 10.210.101.12045.43 GB 256 0.4% 0fd912fb-3187-462b-8c8a-7d223751b649 rack1 UN 10.210.101.11127.08 GB 256 0.4% bd4c37bc-07dd-488b-bfab-e74e32c26f6e rack1 What was wrong? please help. I could provide more information if you need. Thanks, Daning On Mon, Feb

Cassandra jmx stats ReadCount

2013-02-07 Thread Daning Wang
We have 8 nodes cluster in Casandra 1.1.0, with replication factor is 3. We found that when you just insert data, not only WriteCount increases, the ReadCount also increases. How could this happen? I am under the impression that readCount only counts the reads from client. Thanks, Daning

Upgrade to Cassandra 1.2

2013-02-02 Thread Daning Wang
hange the num_token http://www.datastax.com/docs/1.2/install/upgrading Is there other doc about this upgrade path? Thanks, Daning

Re: Problem on node join the ring

2013-01-29 Thread Daning Wang
ou are not using QUOURM / QUOURM you maybe getting inconsistent results > now. > > Cheers > > - > Aaron Morton > Freelance Cassandra Developer > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 29/01/2013, at 9:51 AM,

1.2 Authentication

2013-01-28 Thread Daning Wang
cql.cassandra.ttypes.AuthenticationException: AuthenticationException(why="User doesn't exist - create it with CREATE USER query first") What does "create it with CREATE USER query first" mean? I put debug information in SimpleAuthenticator class, that showed authentication is passed in the authenticate() method. Thanks, Daning

Problem on node join the ring

2013-01-28 Thread Daning Wang
I add a new node to ring(version 1.1.6), after more than 30 hours, it is still in the 'Joining' state Address DC RackStatus State Load Effective-Ownership Token 141784319550391026443072753096570088105 10.28.78.123datacenter1 rack1 Up Normal

Re: Replication factor

2012-05-23 Thread Daning Wang
Thanks guys. Aaron, I am confused about this. from wiki http://wiki.apache.org/cassandra/ReadRepair, looks for any consistency level. Read Repair will be done either before or after responding data. >Read Repair does not run at CL ONE Daning On Wed, May 23, 2012 at 3:51 AM, Viktor Jevdoki

Replication factor

2012-05-22 Thread Daning Wang
to go to another hop to get data if RF=4, but it would do more work on read repair in the background. Can you share some insights about this? Thanks in advance, Daning

Re: Couldn't find cfId

2012-05-16 Thread Daning Wang
Thanks Aaron! We will upgrade to 1.0.9. Just curious, you said "removing the HintedHandoff files from data/system", what do the HintedHandoff files look like? Thanks, Daning On Wed, May 16, 2012 at 2:32 AM, aaron morton wrote: > Looks like this https://issues.apache.org/jira/bro

Couldn't find cfId

2012-05-15 Thread Daning Wang
ffManager.java:344) at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:248) at org.apache.cassandra.db.HintedHandOffManager.access$200(HintedHandOffManager.java:84) at org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:418) Thanks, Daning

Re: Request timeout and host marked down

2012-04-10 Thread Daning Wang
gt; 0 0 > > Looks fine. > > Hope that helps. > > > - > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 10/04/2012, at 8:08 AM, Daning Wan

Re: Request timeout and host marked down

2012-04-09 Thread Daning Wang
gt; Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 6/04/2012, at 5:30 AM, Daning Wang wrote: > > > Hi all, > > > > We are using Hector and ofter we see lots of timeout exception in the > log, I know that the hector can failover to othe

Request timeout and host marked down

2012-04-05 Thread Daning Wang
Hi all, We are using Hector and ofter we see lots of timeout exception in the log, I know that the hector can failover to other node, but I want to reduce the number of timeouts. any hector parameter I should change to reduce this error? also, on the server side, any kind of tunning need to do f

Re: Cassandra Exception

2012-03-28 Thread Daning Wang
We upgraded to 1.0.8, and looks the problem is gone. Thanks for your help, Daning On Sun, Mar 25, 2012 at 9:54 AM, aaron morton wrote: > Can you go to those nodes and run describe cluster ? Also check the logs > on the machines that are marked as UNREACHABLE . > > A node will

Re: Cassandra Exception

2012-03-22 Thread Daning Wang
10.28.20.200datacenter1 rack1 Up Normal 5.51 GB 16.67% 85070591730234615865843651857942052863 10.123.83.114 datacenter1 rack1 Up Normal 5.49 GB 16.67% 113427455640312821154458202477256070484 10.240.113.171 datacenter1 rack1 Up Normal 5.43 GB 16.67% 14178

How to find CF from cfId

2012-03-22 Thread Daning Wang
ponse.java:113) at org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:82) at org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:64) Daning

Re: Cassandra Exception

2012-03-21 Thread Daning Wang
and we are on 0.8.6. On Wed, Mar 21, 2012 at 10:24 AM, Daning Wang wrote: > Hi All, > > > We got lots of Exception in the log, and later the server crashed. any > idea what is happening and how to fix it? > > ERROR [RequestResponseStage:4] 20

Cassandra Exception

2012-03-21 Thread Daning Wang
at org.apache.cassandra.service.RowRepairResolver.resolve(RowRepairResolver.java:85) at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:54) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more Thank you in advance, Daning

Re: hector connection pool

2012-03-09 Thread Daning Wang
hosts every xx seconds and recover connection. > > Regards, > Maciej > > > On Mon, Mar 5, 2012 at 8:13 PM, Daning Wang wrote: > >> I just got this error ": All host pools marked down. Retry burden pushed >> out to client." in a few clients recently, client co

hector connection pool

2012-03-05 Thread Daning Wang
r was busy. But I think client should recover after server load was down. Any bug reported about this? I did search but could not find one. Thanks, Daning

Re: Rebalance cluster

2012-01-12 Thread Daning Wang
Thank you guys. very appreciated. How about just pulling the slow machines out of cluster? I think the most of reads should already from fast machine right now because of dynamic snitch. so removing two machines should not add much loads on the remaining nodes. How do you think? Thanks, Daning

Rebalance cluster

2012-01-11 Thread Daning Wang
hanks in advance, Daning

Re: Pending on ReadStage

2012-01-06 Thread Daning Wang
Thanks for your reply. Nodes are equally balanced. and it is RandomPartitioner. I think that machine is slower, Are you saying it is IO issue? Daning On Fri, Jan 6, 2012 at 10:25 AM, Mohit Anchlia wrote: > Are all your nodes equally balanced in terms of read requests? Are you >

Pending on ReadStage

2012-01-06 Thread Daning Wang
ATION 0 REQUEST_RESPONSE 0 Thanks you in advance. Daning

TimedOutException()

2012-01-03 Thread Daning Wang
? Your help is much appreciated. Daning

Cassandra memory usage

2012-01-03 Thread Daning Wang
11g *S 66.9 65.5 338:02.72 java Thank you in advance, Daning

Re: Key count mismatch in cluster for a column family

2011-11-07 Thread Daning
Sylvain - We have similar problem but the discrepancy is not that big. Do we have to do major compaction to fix it? We did not do 'nodetool compact', just did repair regularly, which triggers minor compaction. Thanks, Daning On 10/26/2011 03:23 AM, Sylvain Lebresne wrote: The es

Restore snapshots suggestion

2011-10-13 Thread Daning
meter DoConsistencyChecksBoolean(http://www.datastax.com/dev/blog/whats-new-cassandra-066) to disable read repair, what is the proper way to do it? I am on 0.8.6. Thank you in advance, Daning

Re: Weird problem with empty CF

2011-10-04 Thread Daning
Thanks. Do you have plan to improve this? I think tombstone should be separated with live data since it serves different purpose, built in separate SSTable or indexed differently. It is pretty costly to do filtering while reading. Daning On 10/04/2011 01:34 PM, aaron morton wrote: I would

Re: Weird problem with empty CF

2011-10-04 Thread Daning
nding deleted data will be synced back. In this case my data will be processed twice and it will not be a big deal to me. Thanks, Daning On 10/04/2011 01:27 AM, aaron morton wrote: Yes that's the slice query skipping past the tombstone columns. Cheers - Aaron Morton

Re: Weird problem with empty CF

2011-10-03 Thread Daning Wang
gt; Cheers > > > - > Aaron Morton > Freelance Cassandra Developer > @aaronmorton > http://www.thelastpickle.com > > On 30/09/2011, at 3:27 AM, Daning Wang wrote: > > Jonathan/Aaron, > > Thank you guy's reply, I will change GCGracePeriod

Re: Weird problem with empty CF

2011-09-29 Thread Daning Wang
an do repair first to make sure the consistency before purging. Thanks, Daning On Wed, Sep 28, 2011 at 5:22 PM, aaron morton wrote: > if I had to guess I would say it was spending time handling tombstones. If > you see it happen again, and are interested, turn the logging up to DEBUG >

Weird problem with empty CF

2011-09-28 Thread Daning
6. Thanks, Daning

Re: Lots of GC in log

2011-09-22 Thread Daning
Thanks Peter. I saw cpu was shooting much higher. I am not sure if frequent GCs are caused by improperly sized generations. I'd like to get some tunning tips, or good document about Cassandra tuning. Daning On 09/22/2011 12:23 PM, Peter Schuller wrote: We are testing Cassandra with p

Lots of GC in log

2011-09-22 Thread Daning
GCInspector.java (line 122) GC for ParNew: 534 ms for 1 collections, 3308763856 used; max is 8015314944 Thanks in advance. Daning

Re: Unable to create compaction marker

2011-09-22 Thread Daning
Thank you guys. I don't think have permission issue or out of space. might be ulimit set to low(1024), we will change ulimit -Hn and unlimit -Sn to see if that could solve the problem. Daning On 09/22/2011 01:12 AM, aaron morton wrote: It's in the yaml file… # directories where

Re: Unable to create compaction marker

2011-09-21 Thread Daning
Thanks Jonathan. where is the data directory configured? since I did not any permission problem. Daning On 09/21/2011 11:01 AM, Jonathan Ellis wrote: Means Cassandra couldn't create an empty file in the data directory designating a sstable as compacted. I'd look for permission

Unable to create compaction marker

2011-09-21 Thread Daning
Thank you in advance. Daning

ByteOrderedPartitioner

2011-09-16 Thread Daning Wang
data to all 4 nodes, that could avoid hot spot, right? Thank you in advance, Daning

Queue suggestion in Cassandra

2011-09-16 Thread Daning Wang
for other designs. I want to know if anybody has implemented a large ordered queue successfully. Let me know if you have suggestion, Thank you in advance. Daning

Disable hector stats

2011-09-09 Thread Daning
Avg(ms) Min Max Std Dev 95th Count (PeriodicalLog.java:221) INFO Thread-53040 2011-09-09 13:24:03,291 ( We have tried to set log4j like this but that does not work, log4j.logger.com.ecyrd.speed4j.log.PeriodicalLog=ERROR Thanks, Daning

Query indexed column with key filter‏

2011-06-28 Thread Daning
query is not supported, and what's the good work around for it? At this moment my workaound is to create a column which is exactly same as key. Thanks, Daning