Re: Kafka servers starting and stopping abnormally

It would help if you provide a bit more detail. For example, you mention an OS upgrade - but what OS? from what version to what version? Do you see any errors in the broker logs at the time at which they shut down? -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Mon, 14

Re: SIGSEGV (0xb) on TransactionCoordinator

Hi Wenxing, You can check all the attached core dumps yourself. All of them (even the three from 14/Dec/18) have the following lines: # JRE version: Java(TM) SE Runtime Environment (8.0_92-b14) (build 1.8.0_92-b14) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode linux-amd6

Why do the offsets of the consumer-group (app-id) of my Kafka Streams Application get reset after application restart?

I have a Kafka Streams application for which, whenever I restart it, the offsets for the topic partitions (*KTABLE-SUPPRESS-STATE-STORE*) it is consuming get reset to 0. Hence, for all partitions, the lags increase and the app needs to reprocess all the data. I have ensured the lag is 1 for every

Aw: Re: Re: Doubts in Kafka

Hi, >> ...AND deactivate the key-based log-cleaner on the >> broker so that it does not delete older records >> that have the same key? > How old records are cleaned is independent of what you do with > processed records. You usually retain them for enough time so > you don't loose them before p

Re: SIGSEGV (0xb) on TransactionCoordinator

Thanks to Peter. My fault! On Mon, Jan 14, 2019 at 7:28 PM Peter Levart wrote: > Hi Wenxing, > > You can check all the attached core dumps yourself. All of them (even > the three from 14/Dec/18) have the following lines: > > # JRE version: Java(TM) SE Runtime Environment (8.0_92-b14) (build > 1.

Aw: Re: Re: Doubts in Kafka

One more question: Is there a way to ask Kafka which ProducerRecord.key is mapped to which TopicPartition (for debugging purposes etc.)? Gesendet: Montag, 14. Januar 2019 um 13:49 Uhr Von: "Sven Ludwig" An: users@kafka.apache.org Betreff: Aw: Re: Re: Doubts in Kafka Hi, >> ...AND deactivate

Re: Zookeeper timeout message in logs has value < configured timeout

Thanks. For the record the "A word or two about heartbeats" section at https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting covers how the ZK session timeout is handled. On Thu, 10 Jan 2019 at 09:40 赖剑清 wrote: > Hi, > > I think it's the zk client's problem. > > 1. Where the log

SOCKS5 proxy support for Kafka Producers

​Hi, Is there a way for a kafka producer running inside an enterprise network *without* any direct access to the Internet to connect and send messages to a kafka cluster running in the cloud? Is it possible to configure the producer to use a SOCKS5 proxy? If not, is there any other possible way

Re: Why do the offsets of the consumer-group (app-id) of my Kafka Streams Application get reset after application restart?

Hi Jonathan, With EOS enabled, Kafka Streams does not use checkpoint files for restoring state stores; it will replay the data contained in the changelog topic. But this should not affect where the input source topic(s) after a restart also the changelog topics are only consumed from during a rest

Kafka Broker with -XX:-UseBiasedLocking

Hi, I've been investigating application pauses on my Kafka broker and sometimes see large (high hundreds to one second) safepoint times for revoking bias. Does anyone have any experience of running Kafka brokers with the -XX:-UseBiasedLocking flag to remove these pauses? If so does it impact per

Re: Why do the offsets of the consumer-group (app-id) of my Kafka Streams Application get reset after application restart?

Hello Bill, thanks a lot for the reply, I will implement your recommendation about the *KafkaStreams#setGlobalStateRestoreListener.* About your question: *When you say you have used both "exactly once" and "at least once" for the* *"at least once" case did you run for a while in that mode then re

SOCKS5 proxy support for Kafka Producers

​Hi, Is there a way for a kafka producer running inside an enterprise network *without* any direct access to the Internet to connect and send messages to a kafka cluster running in the cloud? Is it possible to configure the producer to use a SOCKS5 proxy? If not, is there any other possible way

Re: Re: Re: Doubts in Kafka

The mapping of ProducerRecord keys to TopicPartitions is configurable within the producer. As such, you wouldn't be able to ask the Cluster for a mapping since it doesn't know how the mapping was generated. To determine the TopicPartition a given key would map to, you will need to run it through t

Re: KTable.suppress(Suppressed.untilWindowCloses) does not suppress some non-final results when the kafka streams process is restarted

Hi Peter, I see your train of thought, but the actual implementation of the window store is structured differently from your mental model. Unlike Key/Value stores, we know that the records in a window store will "expire" on a regular schedule, and also that every single record will eventually expi