Thanks all for the pointers. @Joseph,
I have gone through the links shared by you. Also, I have been looking at the code base. I understand the fact that pushing the logs to ES or Solr is a lot easier to do. Having said that, the only reason I thought having something like this might help is, if I don't want to add more pieces and still provide a central piece of audit logging within Cassandra itself and still be queryable. In terms of usages, one of them could definitely be CDC related use cases. With data being stored in tables and being queryable, it can become a lot more easier to expose this data to external systems like Kafka Connect, Debezium which have the ability to push data to Kafka for example. Note that pushing data to Kafka is just an example, but what I mean is, if we can have data in tables, then instead of everyone writing custom custom loggers, they can hook into this table info and take action. Regarding the infinite loop question, I have done some analysis, and in my opinion, instead of tweaking the behaviour of Binlog and the way it functions currently, we can actually spin up another tailer thread to the same Chronicle Queue which can do the needful. This way the config options etc all remain the same(apart from the logger ofcourse). Let me know if any of it makes sense :D Thanks! Sagar. On Thu, Feb 28, 2019 at 1:09 AM Dinesh Joshi <djos...@icloud.com.invalid> wrote: > > > > On Feb 27, 2019, at 10:41 AM, Joseph Lynch <joe.e.ly...@gmail.com> > wrote: > > > > Vinay can confirm, but as far as I am aware we have no current plans to > > implement audit logging to a table directly, but the implementation is > > fully pluggable (like compaction, compression, etc ...). Check out the > blog > > post [1] and documentation [2] Vinay wrote for more details, but the > short > > +1. I am still curious as to why you'd want to store audit log entries > back in Cassandra? Depending on the scale it can generate a lot of load and > I think you'd end up in an infinite loop because as you're inserting the > audit log entry you'll generate a new one and so on unless you black list > audits to that table / keyspace. > > Ideally you'd insert this data into ElasticSearch / Solr or some other > place that can be then used for analytics or search. > > Dinesh > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
