Using Kafka for distributing a log-structured temporal document store

2019-12-24 Thread Johannes Lichtenberger
Hello, as I've read Martin Kleppmans book "Designing Data Intensive Applications" I'm thinking about using Kafka to distribute a temporal document store I'm engineering called SirixDB[1]. SirixDB allows the creation of databases and therein resources, which are currently binary tree-representatio

Re: Missing internal topic (partitions) after kafka-streams client upgrade (2.3.1 -> 2.4.0)

2019-12-24 Thread Nitay Kufert
Hey, thanks for the response! Clarifying the feature flag: We have a flag that controls how our streams application is built (meaning, the topology is changing according to this flag). To be fair, the flag is set to TRUE for the last couple of months - so the fact that the streams application work

Re: Simplifying standalone mm2-connect config

2019-12-24 Thread Ryanne Dolan
Hello Karan. I agree the initial experience could be a lot friendlier. Most of the complexity there is inherited from Connect, but it's compounded when multiple clusters are involved. I don't think we want to change Connect's (or MM2's) defaults to assume a single broker cluster -- it'd be too eas

Simplifying standalone mm2-connect config

2019-12-24 Thread Karan Kumar
Hi One of the nice things about kafka is setting up in the local environment is really simple. I was giving a try to the latest feature ie MM2 and found it took me some time to get a minimal setup running. Default config provided assumes that there will already be 3 brokers running due to the defa