Our team is evaluating KStreams and Reactive Kafka (version 0.11-M4) on a confluent 3.0 cluster. Our testing is very simple (pulling from one topic, doing a simple transform) and then writing out to another topic.
The performance for the two systems is night and day. Both applications were running on a laptop and connecting to kafka over a wifi network. Here are the numbers: KStreams: ~14K messages per second Reactive Kafka: ~110 messages per second Both the input, and output topic had 54 partitions. I’m fairly certain I’m not using Reactive kafka with good configuration. Here is some stubbed out code: https://gist.github.com/anduill/2e17cd7a40d4a86fefe19870d1270f5b One note, I am using the confluent stack (hence the CachedSchemaRegistryClient) I like the flexibility of Reactive Kafka, so we’d really like to use it…but if performance is going to be like this, I can’t really justify it. I’m a scala/akka/streaming-akka newbie, so I’m sure there are better ways to use the API. Any help is appreciated. -David
