Re: Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jian Fang
I did not set the consistency level because I didn't find this option in the ConfigHelper class. I guess it should use level one by default. Actually, I only twisted the word count example a bit. Here is the code snippet, getConf().set(CONF_COLUMN_NAME, columnName); Job job = new

Re: Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jeremy Hanna
Just wondering - what consistency level are you using for hadoop reads? Also, do you have task trackers running on the cassandra nodes so that reads will be local? On Jul 28, 2011, at 2:46 PM, Jian Fang wrote: > I changed the rpc_timeout_in_ms to 3 and 4, then changed the > cassandra.

Re: Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jian Fang
I changed the rpc_timeout_in_ms to 3 and 4, then changed the *cassandra.range.batch.size from 4096 to 1024, but still 40% tasks got timeout exceptions. * Not sure if this is caused by Cassandra speed performance (8G heap size for about 100G of data) or the way how the Cassandra-hadoop integ

Re: Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jian Fang
My current setting is 1. I will try 3. Thanks, John On Thu, Jul 28, 2011 at 12:39 PM, Jeremy Hanna wrote: > See http://wiki.apache.org/cassandra/HadoopSupport#Troubleshooting - I > would probably start with setting your rpc_timeout_in_ms to something like > 3. > > On Jul 28, 2011, at

Re: Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jeremy Hanna
See http://wiki.apache.org/cassandra/HadoopSupport#Troubleshooting - I would probably start with setting your rpc_timeout_in_ms to something like 3. On Jul 28, 2011, at 11:09 AM, Jian Fang wrote: > Hi, > > I run Cassandra 0.8.2 and hadoop 0.20.2 on three nodes, each node includes a > Cassa

Cassandra timeout exception when works with hadoop

2011-07-28 Thread Jian Fang
Hi, I run Cassandra 0.8.2 and hadoop 0.20.2 on three nodes, each node includes a Cassandra instance and a hadoop data node. I created a simple hadoop job to scan a Cassandra column value in a column family and write it to a file system if it meets some conditions. I keep getting the following time