Thanks Haebin, I scaled up to a 3 node system and it now behaves as expected. Was trying to simplify the test case but shot myself in the foot instead.
Mitchell From: [email protected] [mailto:[email protected]] On Behalf Of Frederick Haebin Na Sent: Tuesday, June 03, 2014 2:17 AM To: [email protected] Subject: Re: INSERT ... IF NOT EXISTS with some nodes unavailable Hello Mitchell, I think it is due to your replication factor, which, I assume, is 2 since you have only 2 nodes in the cluster. If you are using even number of nodes, Cassandra is impossible to run queries require QUORUM participants. So, I think you have to expand your cluster to 3 nodes and make replication factor 3. HTH Haebin 2014-06-03 12:09 GMT+09:00 Ackerman, Mitchell <[email protected]<mailto:[email protected]>>: Hi, I’m trying to get a query using INSERT ... IF NOT EXISTS working when not all of the nodes are available. As a test case I have 2 nodes, one in AWS us-west-1, another in AWS eu-west-1. The keyspace settings are described below. When I only have one of the nodes available, the insert fails with an UnavaliableException (via TokenRangeOfflineException, see below for details). From reading about the lightweight transactions (http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0) and the CQL expression, it looks like this scenario should be supported. Does anyone have any idea why it is not working? You will notice that I’m using Astyanax, could this be a source of problems for this use case? I presume that a Consistency Level of Serial is in use and that a quorum of 1 should suffice for a 2 node cache. Thanks, Mitchell conferenceassignmentcache replica_placement_strategy org.apache.cassandra.locator.NetworkTopologyStrategy Replication Strategy Options us-west 1 eu-west 1 CREATE TABLE conferenceassignmentcache.conferenceassignmentcache_cf ( id text PRIMARY KEY, value text ) WITH bloom_filter_fp_chance=0.010000 AND caching='KEYS_ONLY' AND comment='' AND dclocal_read_repair_chance=0.000000 AND gc_grace_seconds=864000 AND read_repair_chance=0.100000 AND replicate_on_write='true' AND populate_io_cache_on_flush='false' AND compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'LZ4Compressor'}; When I try to INSERT a row with only one node available I get the following exception: 2014-06-02 20:57:59,947 [eventTaskExecutor-15] DEBUG [CassandraConferenceAssignmentManager.getInsertStatement() 53] - INSERT INTO conferenceassignmentcache_cf (id, value) VALUES (?, ?) IF NOT EXISTS USING TTL 60; 2014-06-02 20:58:00,084 [eventTaskExecutor-15] DEBUG [ThriftConverter.ToConnectionPoolException() 157] - 2014-06-02 20:58:00,086 [eventTaskExecutor-15] ERROR [CountingConnectionPoolMonitor.trackError() 94] - com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=cache.alpha.us-west-1.bobdev.com<http://cache.alpha.us-west-1.bobdev.com>(10.89.0.37):9160, latency=137(137), attempts=1]UnavailableException() com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=cache.alpha.us-west-1.bobdev.com<http://cache.alpha.us-west-1.bobdev.com>(10.89.0.37):9160, latency=137(137), attempts=1]UnavailableException() Caused by: UnavailableException() at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result.read(Cassandra.java:41876) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_prepared_cql3_query(Cassandra.java:1689) at org.apache.cassandra.thrift.Cassandra$Client.execute_prepared_cql3_query(Cassandra.java:1674) at com.netflix.astyanax.thrift.ThriftCql3Query.execute_prepared_cql_query(ThriftCql3Query.java:29) at com.netflix.astyanax.thrift.AbstractThriftCqlQuery$3$1.internalExecute(AbstractThriftCqlQuery.java:93) at com.netflix.astyanax.thrift.AbstractThriftCqlQuery$3$1.internalExecute(AbstractThriftCqlQuery.java:83) at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60) ... 35 more
