Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-28 Thread Guozhang Wang
Hi Matthias, We had another quick discussion on https://issues.apache.org/jira/browse/KAFKA-9925 regarding deprecating the old StreamsBuilder#build overload. Just copying it here: There's risk of course that the props passed in `build` is not the same as the one passed into the `KafkaStreams` con

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-14 Thread Guozhang Wang
Thanks for writing this up Matthias. I'm also thinking that maybe we can allow customized store types to be replaced as a single config knob. I'm even thinking if this can be extended to processor API besides DSL as well: e.g. if we allow classes like "KeyValueStoreBuilder" to take a constructor l

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-14 Thread Matthias J. Sax
While writing the KIP, the idea about custom stores also came to my mind, but I thought it could be done in a follow up KIP. However, given that two people asked about it, it might be better to just do it right away. Using a "super supplier interface" instead of an String config seems to be the mor

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-14 Thread John Roesler
Hi all, Thanks for starting this, Matthias! I've had multiple people mention this feature request to me. Actually, the most recent such request was from someone developing an LMDB-backed set of store implementations, as a drop-in replacement for RocksDB, so Sophie's suggestion seems relevant. Wha

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-13 Thread Sophie Blee-Goldman
Hey Matthias, Thanks for picking this up! This'll be really nice for testing in particular. My only question is, do we want to make this available for use with custom state stores as well? I'm not sure how common custom stores are in practice, but I imagine when they *are* used, they're likely to

[DISCUSS] KIP-591: Add Kafka Streams config to set default store type

2020-04-09 Thread Matthias J. Sax
Hi, I would like to propose a small KIP to simplify the switch from RocksDB to in-memory stores in Kafka Stream: https://cwiki.apache.org/confluence/display/KAFKA/KIP-591%3A+Add+Kafka+Streams+config+to+set+default+store+type Looking forward to your feedback. -Matthias signature.asc Descript