Hi,
#using cassandra 0.8.9
I used to have rpc_address set to 0.0.0.0 to bind cassandra to all
interfaces. After upgrading out Hadoop jobs to cassandra 0.8.9 (from
0.8.7) there are lots of these messages, and the jobs fails.
12/01/26 11:15:21 DEBUG hadoop.ColumnFamilyInputFormat: failed
connect to endpoint 0.0.0.0
java.io.IOException: unable to connect to server
at
org.apache.cassandra.hadoop.ConfigHelper.createConnection(ConfigHelper.java:389)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSubSplits(ColumnFamilyInputFormat.java:224)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat.access$200(ColumnFamilyInputFormat.java:73)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat$SplitCallable.call(ColumnFamilyInputFormat.java:193)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat$SplitCallable.call(ColumnFamilyInputFormat.java:178)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at
org.apache.cassandra.hadoop.ConfigHelper.createConnection(ConfigHelper.java:385)
... 9 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:211)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.thrift.transport.TSocket.open(TSocket.java:178)
... 11 more
...
Caused by: java.util.concurrent.ExecutionException:
java.io.IOException: failed connecting to all endpoints
10.0.18.129,10.0.18.99,10.0.18.98
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:156)
... 19 more
Caused by: java.io.IOException: failed connecting to all endpoints
10.0.18.129,10.0.18.99,10.0.18.98
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSubSplits(ColumnFamilyInputFormat.java:241)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat.access$200(ColumnFamilyInputFormat.java:73)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat$SplitCallable.call(ColumnFamilyInputFormat.java:193)
at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat$SplitCallable.call(ColumnFamilyInputFormat.java:178)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
describe_ring returns:
endpoints: 10.0.18.129,10.0.18.99,10.0.18.98
rpc_endpoints: 0.0.0.0,0.0.0.0,0.0.0.0
I've found CASSANDRA-3214 but there is "... then fall back to the
gossip endpoint if we don't get what we want." But the job just fails.
The job is run from outside of the cluster, which is valid. When I run
the job from one of the nodes, there is of course cassandra on 0.0.0.0
and the job runs. Where can be the problem?
Regards,
P.
