I did change the rpc_endpoint to endpoints and now the splits are
computed correctly. So it's a bug in cassandra to hadoop interface. I
suspect that it has something to do with wide rows with tens of
thousands of columns we have because the unpatched getSubSplits()
works with small test data we hav
If you want try a test, in the CFIF.getSubSplits(String, String,
TokenRange, Configuration) method, replace the loop on
'range.rpc_endpoints' by the same loop on 'range.endpoints'.
This method split token range of each node with describe_splits method, but
I think there is something wrong when you
You're right, I wasn't looking in the right logs. Unfortunately I'd
need to restart hadoop takstracker with loglevel DEBUG and that is not
possilbe at the moment. Pitty it happens only in the production with
terrabytes of data, not in the test...
Regards,
P.
On Tue, Mar 6, 2012 at 14:31, Florent
CFRR.getProgress() is called by child mapper tasks on each TastTracker
node, so the log must appear on
${hadoop_log_dir}/attempt_201202081707_0001_m_00_0/syslog (or
somethings like this) on TaskTrackers, not on client job logs.
Are you sure to see the good log file, I say that because in your f
I've added a debug message in the CFRR.getProgress() and I can't find
it in the debug output. Seems like the getProgress() has not been
called at all;
Regards,
P.
On Tue, Mar 6, 2012 at 09:49, Jeremy Hanna wrote:
> you may be running into this -
> https://issues.apache.org/jira/browse/CASSANDRA
I've tryied cassandra-all 0.8.10 with fixed the rpc_endpoints ==
"0.0.0.0" bug, but the result is the same, there are still tasks over
1000%. The only change is that there are real host names instead of
0.0.0.0 in the debug output.
Reconfiguring whole cluster is not possible, I can't test the
"rp
I remember a bug on the ColumnFamilyInputFormat class 0.8.10.
It was a test rpc_endpoints == "0.0.0.0" in place of
rpc_endpoint.equals("0.0.0.0"), may be it can help you
Le 6 mars 2012 12:18, Florent Lefillâtre a écrit :
> Excuse me, I had not understood.
> So, for me, the problem comes from the
Excuse me, I had not understood.
So, for me, the problem comes from the change of ColumnFamilyInputFormat
class between 0.8.7 and 0.8.10 where the splits are created (0.8.7 uses
endpoints and 0.8.10 uses rpc_endpoints).
With your config, splits fails, so Hadoop doesn't run a Map task on
approximtiv
Hi Florent,
I don't change the server version, it is the Cassandra 0.8.10. I
change just the version of cassandra-all in pom.xml of the mapreduce
job.
I have the 'rpc_address: 0.0.0.0' in cassandra.yaml, because I want
cassandra to bind RPC to all interfaces.
Regards,
P.
On Tue, Mar 6, 2012 at
you may be running into this -
https://issues.apache.org/jira/browse/CASSANDRA-3942 - I'm not sure if it
really affects the execution of the job itself though.
On Mar 6, 2012, at 2:32 AM, Patrik Modesto wrote:
> Hi,
>
> I was recently trying Hadoop job + cassandra-all 0.8.10 again and the
> Ti
Hi, I had the same problem on hadoop 0.20.2 and cassandra 1.0.5.
In my case the split of token range failed.
I have comment line 'rpc_address: 0.0.0.0' in cassandra.yaml.
May be see if you have not configuration changes between 0.8.7 and 0.8.10
Le 6 mars 2012 09:32, Patrik Modesto a écrit :
> H
Hi,
I was recently trying Hadoop job + cassandra-all 0.8.10 again and the
Timeouts I get are not because of the Cassandra can't handle the
requests. I've noticed there are several tasks that show proggess of
several thousands percents. Seems like they are looping their range of
keys. I've run the
I'll alter these settings and will let you know.
Regards,
P.
On Tue, Feb 28, 2012 at 09:23, aaron morton wrote:
> Have you tried lowering the batch size and increasing the time out? Even
> just to get it to work.
>
> If you get a TimedOutException it means CL number of servers did not respond
>
Have you tried lowering the batch size and increasing the time out? Even just
to get it to work.
If you get a TimedOutException it means CL number of servers did not respond in
time.
Cheers
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 28/0
Hi aaron,
this is our current settings:
cassandra.range.batch.size
1024
cassandra.input.split.size
16384
rpc_timeout_in_ms: 3
Regards,
P.
On Mon, Feb 27, 2012 at 21:54, aaron morton wrote:
> What settings do you have for
What settings do you have for cassandra.range.batch.size and rpc_timeout_in_ms
? Have you tried reducing the first and/or increasing the second ?
Cheers
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 27/02/2012, at 8:02 PM, Patrik Modesto wrote
On Sun, Feb 26, 2012 at 04:25, Edward Capriolo wrote:
> Did you see the notes here?
I'm not sure what do you mean by the notes?
I'm using the mapred.* settings suggested there:
mapred.max.tracker.failures
20
mapred.map.max.attempts
20
Did you see the notes here?
http://wiki.apache.org/cassandra/HadoopSupport#Troubleshooting
On Sat, Feb 25, 2012 at 2:04 AM, Patrik Modesto
wrote:
> No, we have our own servers, no cloud service.
>
> P.
>
> On Feb 24, 2012 5:57 PM, "Jeremy Hanna" wrote:
>>
>> By chance are you in EC2?
>>
>> On F
No, we have our own servers, no cloud service.
P.
On Feb 24, 2012 5:57 PM, "Jeremy Hanna" wrote:
> By chance are you in EC2?
>
> On Feb 24, 2012, at 8:33 AM, Patrik Modesto wrote:
>
> > Hi Jeremy,
> >
> > I've seen the page and tried the values but to no help.
> >
> > Here goes tcpdump of one fa
By chance are you in EC2?
On Feb 24, 2012, at 8:33 AM, Patrik Modesto wrote:
> Hi Jeremy,
>
> I've seen the page and tried the values but to no help.
>
> Here goes tcpdump of one failed TCP connection:
>
> 15:06:20.231421 IP 10.0.18.87.9160 > 10.0.18.87.39396: Flags [P.], seq
> 137891735:13790
Hi Jeremy,
I've seen the page and tried the values but to no help.
Here goes tcpdump of one failed TCP connection:
15:06:20.231421 IP 10.0.18.87.9160 > 10.0.18.87.39396: Flags [P.], seq
137891735:137904068, ack 708, win 282, options [nop,nop,TS val
4119981636 ecr 4119981636], length 12333
15:06:
Check out the troubleshooting section of the hadoop support - we ran into the
same thing and tried to update that with some info on how to get around it:
http://wiki.apache.org/cassandra/HadoopSupport#Troubleshooting
On Feb 24, 2012, at 7:20 AM, Patrik Modesto wrote:
> Hi,
>
> I can see some st
There are two more similar exceptions:
DEBUG 14:23:12,817 Thrift transport error occurred during processing of message.
org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transpor
Some more info, running the cluster with debug enabled shows this error:
DEBUG 14:23:05,749 Thrift transport error occurred during processing of message.
org.apache.thrift.transport.TTransportException:
java.net.SocketException: Broken pipe
at
org.apache.thrift.transport.TIOStreamTranspor
24 matches
Mail list logo
