RE: Does queue.time still apply for the new Producer?

Kafka documentation. It is mentioned only on one place. Regards, Petr From: Petr Novak [mailto:oss.mli...@gmail.com] Sent: 2. května 2017 15:39 To: users@kafka.apache.org Subject: Does queue.time still apply for the new Producer? The documentation reads as: "As events enter a queue, they are buf

Does queue.time still apply for the new Producer?

The documentation reads as: "As events enter a queue, they are buffered in a queue, until either queue.time or batch.size is reached. A background thread (kafka.producer.async.ProducerSendThread) dequeues the batch of data and lets the kafka.producer.EventHandler serialize and send the data to the

RE: Does Kafka producer waits till previous batch returns responce before sending next one?

> 1 (I think the default is 5) then you can get a commit log with messages out of order wrt the original published order (because retries are done in parallel rather then in series) -hans On Apr 30, 2017, at 3:13 AM, Petr Novak <mailto:oss.mli...@gmail.com> wrote: Hello,

Does Kafka producer waits till previous batch returns responce before sending next one?

Hello, Does Kafka producer waits till previous batch returns response before sending next one? Do I assume correctly that it does not when retries can change ordering? Hence batches delay is introduced only by producer internal send loop time and linger? If a timeout would be localized onl

RE: Pattern to create Task with dependencies (DI)

Hello, Nobody has experience with Kafka Connect tasks with external dependencies? Thanks, Petr From: Petr Novak [mailto:oss.mli...@gmail.com] Sent: 23. února 2017 14:48 To: users@kafka.apache.org Subject: Pattern to create Task with dependencies (DI) Hello, it seems that KConnect

Pattern to create Task with dependencies (DI)

Hello, it seems that KConnect take control over creating task instance and requires no-arg constructor. What is the recommended pattern when I need to create a task which has dependency e.g. on some db client and I want to be able to pass in mock in tests, preferable through constructor? In Java

Re: Does Kafka support adding custom metadata to commits?

nsumer group using the PartitionData constructor > ( > http://apache.osuosl.org/kafka/0.8.2-beta/java-doc/org/apache/kafka/common > /requests/OffsetCommitRequest.PartitionData.html) > -Erik > > On 9/17/15, 1:09 PM, "Petr Novak" wrote: > > >Hi all, > >ca

Does Kafka support adding custom metadata to commits?

Hi all, can I add custom metadata to the new commit topic which I could use to recover my app? This would give me some level of consistency if I could commit my additional data at once instead of writing them to another topic. Many thanks, Petr

Re: How to skip malformed messages in a custom Kafka decoder?

> with them is the application's logic. Consume them, if they're not relevant > pick up the next message. > > On Wed, Aug 19, 2015 at 1:10 PM, Petr Novak wrote: > > > Hi all, > > ... by returning null? > > > > Many thanks, > > Petr > > > > > > -- > -- > Sharninder >

Re: Write-time based synchronization of message consumption from multiple topics

Hi, to my knowledge this is generally solved by partitioner which logs messages with the same key to the same topic. It is not clear why you can't use this approach. With Regards, Petr On Thu, Aug 20, 2015 at 12:35 PM, David KOCH wrote: > Hello, > > Is it possible to consume messages from multi

How to skip malformed messages in a custom Kafka decoder?

Hi all, ... by returning null? Many thanks, Petr