Missing link in online document

2019-12-15 Thread Yu Watanabe
Hello. I was walking through kafka streams document and below link seems to be invalid. It returns below page. > Not Found > The requested URL was not found on this server. https://kafka.apache.org/23/documentation/streams/code-examples.html#streams-code-examples May I ask for correct link for

Re: Missing link in online document

2019-12-15 Thread Guozhang Wang
Hello Yu, Could you point to me which page has the reference to this link? Guozhang On Sun, Dec 15, 2019 at 2:24 AM Yu Watanabe wrote: > Hello. > > I was walking through kafka streams document and below link seems to be > invalid. It returns below page. > > > Not Found > > The requested URL

Re: Static Membership AND Invalid IP addresses forConsumers

2019-12-15 Thread Bradley Peterson
Hello Guozhang, Thank you for this answer. You've made me realize that David and I are conflating two things which are not the same -- the host information in the consumer group, and the host information in the StreamsMetadata. We found that our interactive queries were being forwarded to the w

Re: Missing link in online document

2019-12-15 Thread Yu Watanabe
Hello Guozhang. Thank you for the reply. The link was in this page. https://kafka.apache.org/23/documentation/streams/developer-guide/running-app.html Below ** section has the corresponding link. >For more information about how you can package your application in this way, see the ** Streams co

Re: KafkaProducer - Oversized batches

2019-12-15 Thread Tomoyuki Saito
Hi, Filed up a JIRA. https://issues.apache.org/jira/browse/KAFKA-9301 I'll look into how we can fix the issue, but I might not be able to submit a PR very soon. Thanks, Tomoyuki On Mon, Dec 9, 2019 at 1:52 PM Tomoyuki Saito wrote: > > Hi Guozhang, > > Thank you for your suggestion! > I'll crea

Re: Consumer hangs at poll() and never throw exception on client

2019-12-15 Thread Frank Zhou
Hi Matthias, Seems the issue is related to transaction. Let me expand a bit on our flow of this. Our application will use single consumer to fetch the records, deal with the data and then multiple producers to send data afterwards. There could be multiple applications sending data to a single topi

Re: Consumer hangs at poll() and never throw exception on client

2019-12-15 Thread Frank Zhou
More details, the first application's consumer has been closed and its producers are running already when the second application starts, then second consumer hangs forever if transaction is enabled on producers. On Mon, Dec 16, 2019 at 11:43 AM Frank Zhou wrote: > Hi Matthias, > > Seems the issu

Re: Consumer hangs at poll() and never throw exception on client

2019-12-15 Thread Eric Azama
Consumers are unable to make any progress if the next records to be sent are part of an open transaction. Are your producers properly closing transactions? On Sun, Dec 15, 2019 at 7:49 PM Frank Zhou wrote: > More details, the first application's consumer has been closed and its > producers are r

Re: Consumer hangs at poll() and never throw exception on client

2019-12-15 Thread Frank Zhou
Hi, You mean commitTransaction() or abortTransaction()? If so, then yes, either 1 will be called for each transaction. On Mon, Dec 16, 2019 at 12:46 PM Eric Azama wrote: > Consumers are unable to make any progress if the next records to be sent > are part of an open transaction. Are your produc

Replacing custom value while masking in Kafka connect

2019-12-15 Thread Yu Watanabe
Hello . May I ask will below pull request will be merged in to Kafka connect any time soon or not ? Below function is what I need right now and would like to use it in Kafka Connect. KIP-437: Custom replacement for MaskField SMT https://cwiki.apache.org/confluence/display/KAFKA/KIP-437%3A+Custom+