to be more clear, does using transactions and providing the
transactional.id, makes it possible for
a producer to be idempotent even across sessions ?
Thanks :)
On Fri, May 15, 2020 at 9:29 PM Raffaele Esposito
wrote:
> Hi Boynag,
> Yeah what you wrote is clear to me, that's why I asked this qu
Hi Boynag,
Yeah what you wrote is clear to me, that's why I asked this question.
What I don't understand if in the case of a producer we build a producer
sourcing the data from whatever source (MySql for example) we need to take
care explicitly of
the mentioned idempotency per session problem.
Do
Hey Raffaele,
the producer id is getting assigned upon receiving the
producer.initTransaction call at the broker side. It guarantees the
uniqueness of a producer for current lifecycle, which you don't have to
configure manually.
Transactional API on the other hand, includes idempotent produce for
In relation to producer idempotency Kafka documentation says:
... Since each new instance of a producer is assigned a new, unique, PID,
we can only guarantee idempotent production within a single producer
session.
Does it mean that when we build a producer sourcing the data from whatever
source
It is the topic the record will be send to.
Most serializers ignore this parameter. If could be used to pick a
different serialization format for different topics.
-Matthias
On 5/15/20 2:30 AM, Pushkar Deole wrote:
> Hi All,
>
> I am writing a Json customer serializer for our customer objects,
Hi Rapeepat,
1. The parallelism of Kafka Streams does not only depend on the number
of partitions of the input topic. It also depends on the structure of
your topology. Your example topology topicA => transform1 => topicB
=> transform2 => topicC would be subdivided in two subtopologies:
- subtopo
Dear Kafka,
Hi there. I have a question about Kafka Stream parallelism.
I know that Kafka Stream parallelism is based on consumer group.
Like, if you have 3 partitions source topic you can have maximum 3 consumer
instances (or 3 kafka stream instances) at max that will work concurrently.
I have 2
Hi All,
I am writing a Json customer serializer for our customer objects, which
need to implement the Kafka Serializer interface.
I am wondering what the input parameter "topic" is in the
Serializer.serialize method.
I don't think it is the topic on which data is to be published. Please
correct me
thanks.. yes that would help
On Thu, May 14, 2020 at 11:49 PM Matthias J. Sax wrote:
> Yeah, the current API doesn't make it very clear how to do it. You can
> set an in-memory like this:
>
> > builder.globalTable("topic",
> Materialized.as(Stores.inMemoryKeyValueStore("store-name")));
>
>
> We
Check this out: https://rmoff.dev/fix-jdbc-driver-video (specifically here
for Docker instructions https://www.youtube.com/watch?v=vI_L9irU9Pc&t=665s)
--
Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff
On Fri, 15 May 2020 at 08:16, vishnu murali
wrote:
> Hi i am runn
Hi Raffaele,
I would put it differently. Repartition is a concept, but the
repartition topics are an implementation detail. Streams could
repartition data using network shufflling instead of Kafka topics, for
example. Since it is an implementation detail it should not be exposed
publicly, similarl
Hi i am running cp-all-in one docker for the confluent kafka
There i am trying that JDBCSourceConnector.
it is showing this results..
{
"error_code": 400,
"message":
"Connector configuration is invalid and contains the following 2
error(s):\nInvalid value java.sql.SQLException: No suitabl
12 matches
Mail list logo
