Re: Help with MapReduce

2010-04-19 Thread Jonathan Ellis
http://wiki.apache.org/cassandra/FAQ#slows_down_after_lotso_inserts On Tue, Apr 20, 2010 at 12:48 AM, Joost Ouwerkerk wrote: > Ok.  This should be ok for now, although not optimal for some jobs. > > Next issue is node stability during the insert job.  The stacktrace below > occured on several nod

Re: Help with MapReduce

2010-04-19 Thread Joost Ouwerkerk
Ok. This should be ok for now, although not optimal for some jobs. Next issue is node stability during the insert job. The stacktrace below occured on several nodes while inserting 10 million rows. We're running on 4G machines, 1G of which is allocated to cassandra. What's the best config to p

Re: Help with MapReduce

2010-04-19 Thread Jonathan Ellis
TimedOutException() >> >> > at >> >> > >> >> > >> org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:11015) >> >> > at >> >> > >> >> > >&g

Re: Help with MapReduce

2010-04-19 Thread Joost Ouwerkerk
gt; >> > at > >> > > >> > > org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:597) > >> > at > >> > > >> > > org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.mayb

Re: Help with MapReduce

2010-04-19 Thread Jonathan Ellis
der$RowIterator.maybeInit(ColumnFamilyRecordReader.java:142) >> >       ... 11 more >> > >> > On Sun, Apr 18, 2010 at 6:01 PM, Stu Hood >> > wrote: >> >> >> >> In 0.6.0 and trunk, it is located at >> >> src/java/org/apach

Re: Help with MapReduce

2010-04-19 Thread Joost Ouwerkerk
u Hood > wrote: > >> > >> In 0.6.0 and trunk, it is located at > >> src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java > >> > >> You might be using a pre-release version of 0.6 if you are seeing a fat > >> client based InputFormat. &g

Re: Help with MapReduce

2010-04-19 Thread Jonathan Ellis
.6 if you are seeing a fat >> client based InputFormat. >> >> >> -Original Message- >> From: "Joost Ouwerkerk" >> Sent: Sunday, April 18, 2010 4:53pm >> To: user@cassandra.apache.org >> Subject: Re: Help with MapRed

Re: Help with MapReduce

2010-04-19 Thread Jesse McConnell
      ... 11 more >> >> On Sun, Apr 18, 2010 at 6:01 PM, Stu Hood wrote: >>> >>> In 0.6.0 and trunk, it is located at >>> src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java >>> >>> You might be using a pre-release version of 0.6 if you

Re: Help with MapReduce

2010-04-19 Thread Jesse McConnell
ColumnFamilyInputFormat.java >> >> You might be using a pre-release version of 0.6 if you are seeing a fat >> client based InputFormat. >> >> >> -Original Message- >> From: "Joost Ouwerkerk" >> Sent: Sunday, April 18, 2010 4:53pm

Re: Help with MapReduce

2010-04-19 Thread Joost Ouwerkerk
-Original Message- > From: "Joost Ouwerkerk" > Sent: Sunday, April 18, 2010 4:53pm > To: user@cassandra.apache.org > Subject: Re: Help with MapReduce > > Where is the ColumnFamilyInputFormat that uses Thrift? I don't actually > have a preference about client, I

Re: Help with MapReduce

2010-04-18 Thread Stu Hood
4:53pm To: user@cassandra.apache.org Subject: Re: Help with MapReduce Where is the ColumnFamilyInputFormat that uses Thrift? I don't actually have a preference about client, I just want to be consistent with ColumnInputFormat. On Sun, Apr 18, 2010 at 5:37 PM, Stu Hood wrote: > ColumnFa

Re: Help with MapReduce

2010-04-18 Thread Joost Ouwerkerk
Where is the ColumnFamilyInputFormat that uses Thrift? I don't actually have a preference about client, I just want to be consistent with ColumnInputFormat. On Sun, Apr 18, 2010 at 5:37 PM, Stu Hood wrote: > ColumnFamilyInputFormat no longer uses the fat client API, and instead uses > Thrift. T

RE: Help with MapReduce

2010-04-18 Thread Stu Hood
ColumnFamilyInputFormat no longer uses the fat client API, and instead uses Thrift. There are still some significant problems with the fat client, so it shouldn't be used without a good understanding of those problems. If you still want to use it, check out contrib/bmt_example, but I'd recommend