Hi - I'm not sure if this should be asked on the Cassandra or Flume list, so I'm trying both -
I am trying to do a proof of concept for logging into Cassandra. We need to capture large volumes of audit events in a non-lossy manner at the same time being able to quickly access those events for various reports, so this Flume-Cassandra combination seems to be just the ticket. For this POC, I've set up a ring/cluster of 4 Cassandra nodes and they are all running successfully it seems (version 0.6.8), and I can do the simple entry and access of data by hand using the cli. Next I am trying to use Flume to push log data into Cassandra - I've set up the logsandra keyspace for logsandraSyslogSink and the extra column families for the simpleCassandraSink, set up the plugins for Flume, updated the flume-site.xml to add the plugin references, and set the $FLUME_CLASSPATH all as mentioned on the https://github.com/thobbs/flume-cassandra-plugin page. However, I always get an "Invalid sink" error when I try to configure a Flume node to have Cassandra as a sink. Neither of these commands work: 172.16.199.166 : collectorSource(35853) | simpleCassandraSink("KeySpace1","data","FlumeIndexes","172.16.200.130:7005"); 172.16.199.166 : collectorSource(35853) | logsandraSyslogSink("172.16.200.130:7005"); Can anyone please tell me if this is the correct command, or what the format should be? It is not really mentioned in the instructions. I do see the following when starting the flume nodes, but not on the flume master: 2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder simpleCassandraSink in org.apache.cassandra.plugins.SimpleCassandraSink 2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators found in org.apache.cassandra.plugins.SimpleCassandraSink 2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder logsandraSyslogSink in org.apache.cassandra.plugins.LogsandraSyslogSink 2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators found in org.apache.cassandra.plugins.LogsandraSyslogSink So it looks like the plugins are being loaded - without going into the source I am at a loss at the 'Invalid Sink' error. Any information appreciated - Thanks in advance, -Derek -- Derek Deeter, Sr. Software Engineer Intuit Financial Services 5601 Lindero Canyon Road. derek.dee...@digitalinsight.com Westlake, CA 91362
