Well, the store must be byte-based to get fault-tolerance working.
If we loose a global-state store and recover it, it's a different code
path that does not go through the processor, but we take the plain bytes
from the Kafka topic and put them directly into the state store. This
only work if the
Hi Matthias,
Thanks for your response. The Kafka issue you linked provides me lots of
useful context.
> why do you want to write a custom state store?
I really only wrote a custom store because I was struggling to understand
the behaviour when using the existing stores.
I initially started with
* Successful Jenkins builds for the 2.7 branches:
Unit/integration tests:
https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-2.7-jdk8/detail/kafka-2.7-jdk8/53/
On Wed, Nov 4, 2020 at 11:34 AM Bill Bejeck wrote:
> Hello Kafka users, developers, and client-developers,
>
> This i
What you say is correct. The API is not idea unfortunately (something we
hope to improve on eventually).
An important thing to keep in mind is, that the Processor should not
modify the data but only `put` them into the state store as-is.
Otherwise, store recovery breaks (cf
https://issues.apache.o
if your new topics are not named "topic1" or "topic2", maybe you want to use
regex * to allow more topics to be considered by Mm2
# regex which defines which topics gets replicated. For eg "foo-.*"
src-cluster->dst-cluster.topics = topic1,topic2
On 2020/10/30 01:48:00, "Devaki, Srinivas" wrote:
Hello Kafka users, developers, and client-developers,
This is the second candidate for the release of Apache Kafka 2.7.0.
Some blockers were discovered after I cut the first RC, but I had not
announced it yet. So that is why you're seeing this email for the first
time, but
it's the second RC.
T
Am new to Kafka. We are using apache kafka library to produce messages
(KafkaProducer), the ask is to connect via socks proxy from apache library.
Is it possible to connect to Kafka via socks proxy in Java?
When I tried with below params, even after that it didn't took these values.
-DsocksProx
I think I figured it out after digging through the source code ...
- The Processor is called for each new message in the global topic, and
*must* write updated values to the state store
- The deserializers are invoked to decode each message key/value bytes
before passing the decoded v
Hi,
I am trying to implement some functionality using a global state store, and
would like some help so that I can understand it.
I am using the Processor API with Kafka 2.6.
The code I have so far is :
final StoreBuilder> storeBuilder =
Stores.keyValueStoreBuilder(Stores.inMemoryKeyValueStore(
