Hi Derek, The Cassandra sink currently only works with Cassandra 0.7, although I don't suppose that is what is causing the 'InvalidSink' problem.
What version of Flume are you using? The plugin works with Flume 0.9.1, but for 0.9.2 they upgraded the Thrift library version that the project uses and I haven't had the time yet to update the plugin. This would normally be a compile-time error, so I imagine 'InvalidSink' is Flume's response to that. Adding support for Cassandra 0.6 and Flume 0.9.2 are both on my TODO list. If either of those are hard version requirements for you, I could make the effort sooner rather than later. I'm glad you're checking it out! Let me know if you think of any improvements once you get it working :) - Tyler On Thu, Jan 6, 2011 at 8:22 PM, Deeter, Derek < derek.dee...@digitalinsight.com> wrote: > Hi - > > I'm not sure if this should be asked on the Cassandra or Flume list, so I'm > trying both - > > I am trying to do a proof of concept for logging into Cassandra. We need > to capture large volumes of audit events in a non-lossy manner at the same > time being able to quickly access those events for various reports, so this > Flume-Cassandra combination seems to be just the ticket. For this POC, I've > set up a ring/cluster of 4 Cassandra nodes and they are all running > successfully it seems (version 0.6.8), and I can do the simple entry and > access of data by hand using the cli. > > Next I am trying to use Flume to push log data into Cassandra - I've set up > the logsandra keyspace for logsandraSyslogSink and the extra column families > for the simpleCassandraSink, set up the plugins for Flume, updated the > flume-site.xml to add the plugin references, and set the $FLUME_CLASSPATH > all as mentioned on the https://github.com/thobbs/flume-cassandra-pluginpage. > However, I always get an "Invalid sink" error when I try to configure > a Flume node to have Cassandra as a sink. Neither of these commands work: > > 172.16.199.166 : collectorSource(35853) | > simpleCassandraSink("KeySpace1","data","FlumeIndexes","172.16.200.130:7005 > "); > 172.16.199.166 : collectorSource(35853) | logsandraSyslogSink(" > 172.16.200.130:7005"); > > Can anyone please tell me if this is the correct command, or what the > format should be? It is not really mentioned in the instructions. > > I do see the following when starting the flume nodes, but not on the flume > master: > 2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink > builder simpleCassandraSink in > org.apache.cassandra.plugins.SimpleCassandraSink > 2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink > decorators found in org.apache.cassandra.plugins.SimpleCassandraSink > 2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink > builder logsandraSyslogSink in > org.apache.cassandra.plugins.LogsandraSyslogSink > 2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink > decorators found in org.apache.cassandra.plugins.LogsandraSyslogSink > > So it looks like the plugins are being loaded - without going into the > source I am at a loss at the 'Invalid Sink' error. Any information > appreciated - > > Thanks in advance, > -Derek > > > -- > Derek Deeter, Sr. Software Engineer Intuit Financial Services > 5601 Lindero Canyon > Road. > derek.dee...@digitalinsight.com Westlake, CA 91362 > > >
