Re: Issues in moving data from cassandra to elasticsearch in java.

2014-11-25 Thread William Arbaugh
Sounds like you're trying to use C* as a message broker. Perhaps try using Kafka or RabbitMQ as a front-end. Then have two subscribers - one pulls and places into elasticsearch and the other inserts into C*. Yes it is more complex front-end, but it will give you the functionality you want. > O

Re: Issues in moving data from cassandra to elasticsearch in java.

2014-11-25 Thread Eric Stevens
Consider adding log_bucket timestamp, and then indexing that column. Your data loader can SELECT * FROM logs WHERE log_bucket = ?. The value you supply there would be the timestamp log bucket you're processing - in your case logged_at % 5. However, I'll caution against writing data to Cassandra

Fwd: Issues in moving data from cassandra to elasticsearch in java.

2014-11-25 Thread Vinod Joseph
Hi, I am working on a java plugin which moves data from cassandra to elasticsearch. This plugin must run in the server for every 5 seconds. The data is getting moved, but the issue is that every time the plugin runs(ie after every 5 seconds) all the data, including data which has been