Hi Jakub, Were you able to resolve the issue? For a multi data center setup I do believe some steps are different. You may need to set Networktopology as your replication strategy rather than simple strategy, and setup a snitch. And mention the rack/dc configurations in a config file. You can refer to the steps for a multi data center installation.
Regards, Roshni From: Jakub Glapa <jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: Unsuccessful attempt to add a second node to a ring. I found a similar thread from March : http://www.mail-archive.com/user@cassandra.apache.org/msg21007.html For me clearing the data and starting from the beginning didn't help. It's interesting because on my dev environment I was able to add another node without any problems. The only difference is that the second node now is in a different data center. (but I'm not using any different settings, SimpleSnitch) 7000,9160,7199 ports were open between those 2 nodes. How else can I check if the communication between those 2 nodes is working? In the logs I see that: DEBUG [WRITE-NODE1/node1.ip] 2012-07-31 13:50:39,642 OutboundTcpConnection.java (line 206) attempting to connect to NODE1/node1.ip So I assume that the communication is somehow established? -- regards, Jakub Glapa On Wed, Aug 1, 2012 at 11:36 AM, Jakub Glapa <jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com>> wrote: yes it's the same -- regards, pozdrawiam, Jakub Glapa On Wed, Aug 1, 2012 at 11:24 AM, Roshni Rajagopal <roshni.rajago...@wal-mart.com<mailto:roshni.rajago...@wal-mart.com>> wrote: Ok, sorry it may not be required, I was thinking of a configuration I had done on my local laptop, where I had aliased my IP address. In that case the directories and jmx port needed to be different. Cluster name is same right? From: Jakub Glapa <jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com><mailto:jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com>>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>> Subject: Re: Unsuccessful attempt to add a second node to a ring. Hi Roshni, no they are the same, my changes in cassandra.yaml were only in the listen_address, rpc_address, seeds and initial_token field. The rest is exactly the same as on node1. That's how the file looks on node2: cluster_name: 'Test Cluster' initial_token: 85070591730234615865843651857942052864 hinted_handoff_enabled: true hinted_handoff_throttle_delay_in_ms: 1 authenticator: org.apache.cassandra.auth.AllowAllAuthenticator authority: org.apache.cassandra.auth.AllowAllAuthority partitioner: org.apache.cassandra.dht.RandomPartitioner data_file_directories: - /data/servers/cassandra_sbe_edtool/cassandra_data/data commitlog_directory: /data/servers/cassandra_sbe_edtool/cassandra_data/commitlog saved_caches_directory: /data/servers/cassandra_sbe_edtool/cassandra_data/saved_caches commitlog_sync: periodic commitlog_sync_period_in_ms: 10000 seed_provider: - class_name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "NODE1" flush_largest_memtables_at: 0.75 reduce_cache_sizes_at: 0.85 reduce_cache_capacity_to: 0.6 concurrent_reads: 32 concurrent_writes: 32 memtable_flush_queue_size: 4 sliced_buffer_size_in_kb: 64 storage_port: 7000 ssl_storage_port: 7001 listen_address: NODE2 rpc_address: NODE2 rpc_port: 9160 rpc_keepalive: true rpc_server_type: sync thrift_framed_transport_size_in_mb: 15 thrift_max_message_length_in_mb: 16 incremental_backups: false snapshot_before_compaction: false column_index_size_in_kb: 64 in_memory_compaction_limit_in_mb: 64 multithreaded_compaction: false compaction_throughput_mb_per_sec: 16 compaction_preheat_key_cache: true rpc_timeout_in_ms: 10000 endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch dynamic_snitch_update_interval_in_ms: 100 dynamic_snitch_reset_interval_in_ms: 600000 dynamic_snitch_badness_threshold: 0.1 request_scheduler: org.apache.cassandra.scheduler.NoScheduler index_interval: 128 encryption_options: internode_encryption: none keystore: conf/.keystore keystore_password: cassandra truststore: conf/.truststore truststore_password: cassandra -- regards, pozdrawiam, Jakub Glapa On Wed, Aug 1, 2012 at 10:29 AM, Roshni Rajagopal <roshni.rajago...@wal-mart.com<mailto:roshni.rajago...@wal-mart.com><mailto:roshni.rajago...@wal-mart.com<mailto:roshni.rajago...@wal-mart.com>>> wrote: Jakub, Have you set the Data, commitlog, saved cache directories to different ones in each yaml file for each node? Regards, Roshni From: Jakub Glapa <jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com><mailto:jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com>><mailto:jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com><mailto:jakub.gl...@gmail.com<mailto:jakub.gl...@gmail.com>>>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>> Subject: Unsuccessful attempt to add a second node to a ring. Hi Everybody! I'm trying to add a second node to an already operating one node cluster. Some specs: - cassandra 1.0.7 - both nodes have a routable listen_address and rpc_address. - Ports are open: (from node2) telnet node1 7000 is successful - Seeds parameter on node2 points to node 1. [node1] nodetool -h localhost ring Address DC Rack Status State Load Owns Token node1.ip datacenter1 rack1 Up Normal 74.33 KB 100.00% 0 - initial token on node2 was specified I see something like that in the logs on node2: DEBUG [main] 2012-07-31 13:50:38,640 CollationController.java (line 76) collectTimeOrderedData INFO [main] 2012-07-31 13:50:38,641 StorageService.java (line 667) JOINING: waiting for ring and schema information DEBUG [WRITE-NODE1/node1.ip] 2012-07-31 13:50:39,642 OutboundTcpConnection.java (line 206) attempting to connect to NODE1/node1.ip DEBUG [ScheduledTasks:1] 2012-07-31 13:50:40,639 LoadBroadcaster.java (line 86) Disseminating load info ... INFO [main] 2012-07-31 13:51:08,641 StorageService.java (line 667) JOINING: schema complete, ready to bootstrap DEBUG [main] 2012-07-31 13:51:08,642 StorageService.java (line 554) ... got ring + schema info INFO [main] 2012-07-31 13:51:08,642 StorageService.java (line 667) JOINING: getting bootstrap token DEBUG [main] 2012-07-31 13:51:08,644 BootStrapper.java (line 138) token manually specified as 85070591730234615865843651857942052864 DEBUG [main] 2012-07-31 13:51:08,645 Table.java (line 387) applying mutation of row 4c but it doesn't join the ring: [node2] nodetool -h localhost ring Address DC Rack Status State Load Owns Token node2.ip datacenter1 rack1 Up Normal 13.49 KB 100.00% 85070591730234615865843651857942052864 I'm attaching the full log from node2 startup in debug mode. PS. When I didn't specified the initial token on node2 I ended up with exception like that: "Exception encountered during startup: No other nodes seen! Unable to bootstrap.If you intended to start a single-node cluster, you should make sure your broadcast_address (or listen_address) is listed as a seed. Otherwise, you need to determine why the seed being contacted has no knowledge of the rest of the cluster. Usually, this can be solved by giving all nodes the same seed list." I'm not sure how to proceed now. I found a couple of posts with problems like that but they weren't very useful. -- regards, Jakub Glapa This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. *** Walmart Confidential *** This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. *** Walmart Confidential *** This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. *** Walmart Confidential ***
