Re: Hadoop over Cassandra

2010-05-18 Thread Mark Schnitzius
> > If anyone has "war stories" on the topic of Cassandra & Hadoop (or > even just Hadoop in general) let me know. Don't know if it counts as a war story, but I was successful recently in implementing something I got advice on in an earlier thread, namely feeding both a Cassandra table and a Had

Re: Updating (as opposed to just setting) Cassandra data via Hadoop

2010-05-06 Thread Mark Schnitzius
or hadoop run. If/when atomic increments are available, we'd be able > to run concurrent log processing jobs for but for now, this seems to work. > I > think the biggest risk is that a reduce task fails, hadoop restarts it and > the replacement task re-increments the values.

Re: Updating (as opposed to just setting) Cassandra data via Hadoop

2010-05-05 Thread Mark Schnitzius
erstanding what you're saying. > > On Tue, May 4, 2010 at 10:53 PM, Mark Schnitzius > wrote: > > I have a situation where I need to accumulate values in Cassandra on an > > ongoing basis. Atomic increments are still in the works apparently > > (see https://issues.apac

Updating (as opposed to just setting) Cassandra data via Hadoop

2010-05-04 Thread Mark Schnitzius
I have a situation where I need to accumulate values in Cassandra on an ongoing basis. Atomic increments are still in the works apparently (see https://issues.apache.org/jira/browse/CASSANDRA-721) so for the time being I'll be using Hadoop, and attempting to feed in both the existing values and th

Re: Feeding in specific Cassandra columns into Hadoop

2010-05-03 Thread Mark Schnitzius
> > You should test that getSlicePredicate(conf).equals(originalPredicate) > > That's it! The byte arrays are slightly different after setting it on the Hadoop config. Below is a simple test which demonstrates the bug -- it should print "true" but instead prints "false". Please let me know if a

Re: Feeding in specific Cassandra columns into Hadoop

2010-05-03 Thread Mark Schnitzius
de the Hadoop environment, i.e. w/ Thrift code? > > On Mon, May 3, 2010 at 5:49 AM, Mark Schnitzius > wrote: > > Hi all... I am trying to feed a specific list of Cassandra column names > in > > as input to a Hadoop process, but for some reason it only feeds in some &g

Feeding in specific Cassandra columns into Hadoop

2010-05-03 Thread Mark Schnitzius
Hi all... I am trying to feed a specific list of Cassandra column names in as input to a Hadoop process, but for some reason it only feeds in some of the columns I specify, not all. This is a short description of the problem - I'll see if anyone might have some insight before I dump a big load of