Hi -

I'm not sure if this should be asked on the Cassandra or Flume list, so I'm 
trying both - 

I am trying to do a proof of concept for logging into Cassandra.  We need to 
capture large volumes of audit events in a non-lossy manner at the same time 
being able to quickly access those events for various reports, so this 
Flume-Cassandra combination seems to be just the ticket.  For this POC, I've 
set up a ring/cluster of 4 Cassandra nodes and they are all running 
successfully it seems (version 0.6.8), and I can do the simple entry and access 
of data by hand using the cli.

Next I am trying to use Flume to push log data into Cassandra - I've set up the 
logsandra keyspace for logsandraSyslogSink and the extra column families for 
the simpleCassandraSink, set up the plugins for Flume, updated the 
flume-site.xml to add the plugin references, and set the $FLUME_CLASSPATH all 
as mentioned on the https://github.com/thobbs/flume-cassandra-plugin page.  
However, I always get an "Invalid sink" error when I try to configure a Flume 
node to have Cassandra as a sink.  Neither of these commands work:

172.16.199.166 : collectorSource(35853) | 
simpleCassandraSink("KeySpace1","data","FlumeIndexes","172.16.200.130:7005");
172.16.199.166 : collectorSource(35853) | 
logsandraSyslogSink("172.16.200.130:7005");

Can anyone please tell me if this is the correct command, or what the format 
should be?  It is not really mentioned in the instructions.

I do see the following when starting the flume nodes, but not on the flume 
master:
2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder 
simpleCassandraSink in org.apache.cassandra.plugins.SimpleCassandraSink
2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators 
found in org.apache.cassandra.plugins.SimpleCassandraSink
2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder 
logsandraSyslogSink in org.apache.cassandra.plugins.LogsandraSyslogSink
2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators 
found in org.apache.cassandra.plugins.LogsandraSyslogSink

So it looks like the plugins are being loaded - without going into the source I 
am at a loss at the 'Invalid Sink' error.  Any information appreciated -

            Thanks in advance,
            -Derek


--
Derek Deeter, Sr. Software Engineer           Intuit Financial Services
                                                      5601 Lindero Canyon Road.
derek.dee...@digitalinsight.com                Westlake, CA 91362


Reply via email to