Sophie Blee-Goldman created KAFKA-8295:
------------------------------------------
Summary: Optimize count() using RocksDB merge operator
Key: KAFKA-8295
URL: https://issues.apache.org/jira/browse/KAFKA-8295
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Sophie Blee-Goldman
In addition to regular put/get/delete RocksDB provides a fourth operation,
merge. This essentially provides an optimized read/update/write path in a
single operation. One of the built-in (C++) merge operators exposed over the
Java API is a counter. We should be able to leverage this for a more efficient
implementation of count()
(Note: Unfortunately it seems unlikely we can use this to optimize general
aggregations, even if RocksJava allowed for a custom merge operator, unless we
provide a way for the user to specify and connect a C++ implemented aggregator
– otherwise we incur too much cost crossing the jni for a net performance
benefit)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
