Re: [ANNOUNCE] Apache Kafka 1.1.1 Released

2018-07-19 Thread Ismael Juma
Thank you for managing the release Dong! Ismael On Thu, 19 Jul 2018, 16:54 Dong Lin, wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 1.1.1. > > This is a bug fix release and it includes fixes and improvements from 43 > JIRAs, including a few critical bu

Re: [ANNOUNCE] Apache Kafka 1.1.1 Released

2018-07-19 Thread Dong Lin
On Thu, Jul 19, 2018 at 4:53 PM, Dong Lin wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 1.1.1. > > This is a bug fix release and it includes fixes and improvements from 43 > JIRAs, including a few critical bugs. > > All of the changes in this release ca

[ANNOUNCE] Apache Kafka 1.1.1 Released

2018-07-19 Thread Dong Lin
The Apache Kafka community is pleased to announce the release for Apache Kafka 1.1.1. This is a bug fix release and it includes fixes and improvements from 43 JIRAs, including a few critical bugs. All of the changes in this release can be found in the release notes: https://dist.apache.org/repos

Measure latency from Source to Sink

2018-07-19 Thread antonio saldivar
Hello I am developing an application using kafka and flink, I need to be able to measure the latency from the producer and when it comes out to the sink. I can append the timestamp in Millisecond when I send the trxn from the producer but at the end how to append the timestamp when it comes out t

Measure Latency from Source to Sink

2018-07-19 Thread antonio saldivar
Hello I am developing an application using kafka and flink, I need to be able to measure the latency from the producer and when it comes out to the sink. I can append the timestamp in Millisecond when I send the trxn from the producer but at the end how to append the timestamp when it comes out t

Re: Use Kafka Streams for windowing data and processing each window at once

2018-07-19 Thread Bill Bejeck
Hi Bruno, What you are asking is a common request. There is a KIP in the works, https://cwiki.apache.org/confluence/display/KAFKA/KIP-328%3A+Ability+to+suppress+updates+for+KTables, that should suit the requirements you've outlined. In the meantime, I'll see if I can come up with an alternative

Re: Monitor the state of an event-based choreography

2018-07-19 Thread Christopher Smith
This sounds like a pretty good use case for opentracing.org. There is some nice integration with Kafka now too. --Chris On Thu, Jul 19, 2018, 8:26 AM Dan Rosanova wrote: > Are you looking more for an Actor or orchestration layer and visibility? I > don’t know of one per se, it would also be in

Use Kafka Streams for windowing data and processing each window at once

2018-07-19 Thread Bruno Bottazzini
Hello, We have a doubt about Kafka streams on how it works. Or at least we are having some troubles in making it to work. The purpose we want to achieve is to group by user some messages that we receive from a Kafka topic and window them in order to aggregate the messages we receive in the window

Re: Monitor the state of an event-based choreography

2018-07-19 Thread Dan Rosanova
Are you looking more for an Actor or orchestration layer and visibility? I don’t know of one per se, it would also be interested. Dan From: Jonathan Roy Sent: Thursday, July 19, 2018 3:22 AM To: users@kafka.apache.org Subject: Monitor the state of an event-bas

Re: UNKNOWN_PRODUCER_ID error when running Streams WordCount demo with processing.guarantee set to EXACTLY_ONCE

2018-07-19 Thread Bill Bejeck
Hi Thanks for reporting this. Just off the top of my head, I'm thinking it may have to do with using a console producer, but I'll have to take a deeper look. Thanks, Bill On Thu, Jul 19, 2018 at 9:59 AM lambdaliu(刘少波) wrote: > Hi, > > I test the Kafka Streams WordCount demo follow the steps d

UNKNOWN_PRODUCER_ID error when running Streams WordCount demo with processing.guarantee set to EXACTLY_ONCE

2018-07-19 Thread 刘少波
Hi, I test the Kafka Streams WordCount demo follow the steps described in http://kafka.apache.org/11/documentation/streams/quickstart with change the processing.guarantee property to EXACTLY_ONCE. And seeing the following WARN message in streams demo app logs: [2018-07-18 21:08:03,510] WARN Th

Re: Problem consuming from broker 1.1.0

2018-07-19 Thread SenthilKumar K
Hi Craig Ching, Reg. *We did end up turning on debug logs for the console consumer and found that one broker seemed to be having problems, it would lead to timeouts communicating with it. After restarting that broker, things sorted themselves out.* We had similar problem on prod cluster and i'm

Re: [VOTE] 2.0.0 RC2

2018-07-19 Thread Rajini Sivaram
Hi all, We found a blocker in 2.0.0 which is a bug in the newly added OAuth protocol implementation (https://issues.apache.org/jira/browse/KAFKA-7182). Since the current implementation doesn't conform to the SASL/OAUTHBEARER spec in RFC-7628, we need to fix this before the release to conform to th

org.apache.kafka.common.errors.TimeoutException : xxx passed since batch creation plus linger time

2018-07-19 Thread igor . b
Hi We are using kafka 0.10.2 and recently got following error: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for taz_data_info-0: 270120 ms has passed since batch creation plus linger time The producer configuration is as follows: acks: all auto.complete

Kafka Broker Not Responding

2018-07-19 Thread SenthilKumar K
Hello Kafka Experts, We are currently facing issue on our 3 node Kafka Cluster , one of the broker is not responding to any queries. I've checked logs but founding nothing related to this problem. Kafka Version: 1.1.0 Server.conf: ## Timeout properties to check prod outage replica.fetch.wa

Monitor the state of an event-based choreography

2018-07-19 Thread Jonathan Roy
Hello, We are currently studying event-driven architectures with Kafka, and one thing that scares me a bit is that we have no way to know at any moment what is the state of a business transaction that spans multiple services. Let’s consider the following simple flow: 1. the ‘orders’ service cre