Matt,
If I understand correctly, you have an 8 node Kafka cluster and need to support
about 1 million requests/sec into the cluster from source servers and expect
to consume that for aggregation.
How big are your msgs?
I would suggest looking into batching multiple requests per single Kafka m
Hi,
There seems to be no way to commit custom metadata along with offsets from
within Kafka Streams.
Are there any plans to expose this functionality or have I missed something?
Best regards,
Stanislav.
Hello Adrien,
I was wrong. There is only one such file per data dir and not one per
topicpartition dir. It is a text file containing
- a format version number (0),
- number of following entries, and
- one entry for each topicpartition: topic name, partition and offset.
Yes, when the broker sta
