Re: hadoop consistency level

2012-10-18 Thread Andrey Ilinykh
On Thu, Oct 18, 2012 at 2:31 PM, Jeremy Hanna wrote: > > On Oct 18, 2012, at 3:52 PM, Andrey Ilinykh wrote: > >> On Thu, Oct 18, 2012 at 1:34 PM, Michael Kjellman >> wrote: >>> Not sure I understand your question (if there is one..) >>> >>> You are more than welcome to do CL ONE and assuming you

Re: hadoop consistency level

2012-10-18 Thread Jeremy Hanna
On Oct 18, 2012, at 3:52 PM, Andrey Ilinykh wrote: > On Thu, Oct 18, 2012 at 1:34 PM, Michael Kjellman > wrote: >> Not sure I understand your question (if there is one..) >> >> You are more than welcome to do CL ONE and assuming you have hadoop nodes >> in the right places on your ring things

Re: hadoop consistency level

2012-10-18 Thread Michael Kjellman
1. Yes, you can absolutely benefit from data locality, and the InputSplits will theoretically schedule the map task on Cassandra+Hadoop nodes that have the data locally. If your application doesn't require you to worry about that one pesky row that should be local to that node (and that node is res

Re: hadoop consistency level

2012-10-18 Thread Bryan Talbot
I believe that reading with CL.ONE will still cause read repair to be run (in the background) 'read_repair_chance' of the time. -Bryan On Thu, Oct 18, 2012 at 1:52 PM, Andrey Ilinykh wrote: > On Thu, Oct 18, 2012 at 1:34 PM, Michael Kjellman > wrote: > > Not sure I understand your question (i

Re: hadoop consistency level

2012-10-18 Thread Andrey Ilinykh
On Thu, Oct 18, 2012 at 1:34 PM, Michael Kjellman wrote: > Not sure I understand your question (if there is one..) > > You are more than welcome to do CL ONE and assuming you have hadoop nodes > in the right places on your ring things could work out very nicely. If you > need to guarantee that you

Re: hadoop consistency level

2012-10-18 Thread Michael Kjellman
Not sure I understand your question (if there is one..) You are more than welcome to do CL ONE and assuming you have hadoop nodes in the right places on your ring things could work out very nicely. If you need to guarantee that you have all the data in your job then you'll need to use QUORUM. If

Re: hadoop consistency level

2012-10-18 Thread Andrey Ilinykh
On Thu, Oct 18, 2012 at 1:24 PM, Michael Kjellman wrote: > Well there is *some* data locality, it's just not guaranteed. My > understanding (and someone correct me if I'm wrong) is that > ColumnFamilyInputFormat implements InputSplit and the getLocations() > method. > > http://hadoop.apache.org/do

Re: hadoop consistency level

2012-10-18 Thread Michael Kjellman
Well there is *some* data locality, it's just not guaranteed. My understanding (and someone correct me if I'm wrong) is that ColumnFamilyInputFormat implements InputSplit and the getLocations() method. http://hadoop.apache.org/docs/mapreduce/current/api/org/apache/hadoop/mapre duce/InputSplit.html

Re: hadoop consistency level

2012-10-18 Thread Andrey Ilinykh
On Thu, Oct 18, 2012 at 12:00 PM, Michael Kjellman wrote: > Unless you have Brisk (however as far as I know there was one fork that got > it working on 1.0 but nothing for 1.1 and is not being actively maintained > by Datastax) or go with CFS (which comes with DSE) you are not guaranteed > all dat

Re: hadoop consistency level

2012-10-18 Thread Michael Kjellman
; >> >> From: Jean-Nicolas Boulay Desjardins >> Reply-To: "user@cassandra.apache.org" >> Date: Thursday, October 18, 2012 11:49 AM >> To: "user@cassandra.apache.org" >> Subject: Re: hadoop consistency level >> >> Why don't you l

Re: hadoop consistency level

2012-10-18 Thread Jean-Nicolas Boulay Desjardins
-Nicolas Boulay Desjardins > Reply-To: "user@cassandra.apache.org" > Date: Thursday, October 18, 2012 11:49 AM > To: "user@cassandra.apache.org" > Subject: Re: hadoop consistency level > > Why don't you look into Brisk: > http://www.datastax.com/doc

Re: hadoop consistency level

2012-10-18 Thread Michael Kjellman
user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Re: hadoop consistency level Why don't you look into Brisk: http://www.datastax.com/docs/0.8/brisk/about_brisk On Thu, Oct 18, 2012 at 2:46 PM, Andrey Ilinykh mailto:ailin...@gmail.com>> wrote

Re: hadoop consistency level

2012-10-18 Thread William Oberman
A recent thread made it sound like Brisk was no longer a datastax supported thing (it's DataStax Enterpise, or DSE, now): http://www.mail-archive.com/user@cassandra.apache.org/msg24921.html In particular this response: http://www.mail-archive.com/user@cassandra.apache.org/msg25061.html On Thu, Oc

Re: hadoop consistency level

2012-10-18 Thread Jean-Nicolas Boulay Desjardins
Why don't you look into Brisk: http://www.datastax.com/docs/0.8/brisk/about_brisk On Thu, Oct 18, 2012 at 2:46 PM, Andrey Ilinykh wrote: > Hello, everybody! > I'm thinking about running hadoop jobs on the top of the cassandra > cluster. My understanding is - hadoop jobs read data from local node

hadoop consistency level

2012-10-18 Thread Andrey Ilinykh
Hello, everybody! I'm thinking about running hadoop jobs on the top of the cassandra cluster. My understanding is - hadoop jobs read data from local nodes only. Does it mean the consistency level is always ONE? Thank you, Andrey