Jenkins build is back to normal : kafka-trunk-jdk8 #2539

See

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

I am really really sorry for missing the KIP meeting. I am on vacation in Shanghai. I drank too much and over slept yesterday night. Thanks much for the KIP meeting notes! I will read through that carefully. On Thu, Apr 5, 2018 at 10:31 PM, Jeff Chao wrote: > Hi Jun, same for me. jeffc...@me.co

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi folks. I updated KIP 269 to help clarify some of the issues mentioned previously. In particular, I added a new single-method UnderlyingValues interface to make it clear how data is to be provided to SubstitutableValues, and I added information about if/how the underlying values might be re-rea

[jira] [Created] (KAFKA-6769) Upgrade jetty library version

Di Shang created KAFKA-6769: --- Summary: Upgrade jetty library version Key: KAFKA-6769 URL: https://issues.apache.org/jira/browse/KAFKA-6769 Project: Kafka Issue Type: Task Components: core

Build failed in Jenkins: kafka-trunk-jdk7 #3326

See Changes: [github] KAFKA-6768; Transactional producer may hang in close with pending -- [...truncated 411.01 KB...] kafka.admin.ResetConsumerGroupOffsetTest > testRese

Jenkins build is back to normal : kafka-trunk-jdk10 #5

See

Re: [DISCUSS] KIP-279: Fix log divergence between leader and follower after fast leader fail over

Ted and Jason, I see now how the description of unclean leader election made the proposed approach sound more complicated (like there are more roundtrips). I wrote it in such way to show correctness, where theoretically, we could compare the "complete epoch lineage", but in practice we compare only

[jira] [Resolved] (KAFKA-6768) Producer may hang in close with pending transaction

[ https://issues.apache.org/jira/browse/KAFKA-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6768. Resolution: Fixed > Producer may hang in close with pending transaction > --

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Matthias, Thanks for the review. I reply separately in the following sections. --- Vito On Sun, Apr 8, 2018 at 1:30 PM, Matthias J. Sax wrote: > Thanks for updating the KIP and sorry for the long pause... > > Seems you did a very thorough investigation of the code. It's useful to > understand

RE: [DISCUSS] KIP-280: Enhanced log compaction

Hi Matthias, bq. I think adding couple of examples would be good enough. I’ll add them to the KIP then. bq. That's a tricky question... Personally, I think a direct byte[]->long conversion would be straight forward. What is the advantage of the intermediate "String" type? Not that I mind do

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

The following are my notes from today's KIP meeting. The link to the video has been added to https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals . - KIP-253 - partition expansion: We discussed a few things. (1) Is it useful to backfill a compacted topic? The main u

Re: [DISCUSS] KIP-280: Enhanced log compaction

Thanks for the KIP. 1) I'm also in favor of making the `timestamp` a preserved config value along with `offset`, for which we would not go into the headers to look for the matching key, but directly look into the timestamp field of the message. 2) About the `long` conversion from `byte[]` values,

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

+1 (binding) On Mon, Apr 9, 2018 at 11:21 AM, Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 4/9/18 8:57 AM, Bill Bejeck wrote: > > Thanks for the KIP. > > > > +1 > > > > -Bill > > > > On Mon, Apr 9, 2018 at 7:57 AM, zhenya Sun wrote: > > > >> +1 > >> > >> > >> from my iphone！ > >>

[jira] [Created] (KAFKA-6768) Producer may hang in close with pending transaction

Jason Gustafson created KAFKA-6768: -- Summary: Producer may hang in close with pending transaction Key: KAFKA-6768 URL: https://issues.apache.org/jira/browse/KAFKA-6768 Project: Kafka Issue T

Re: [VOTE] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

Thanks to your feedback / vote, the proposal passes with 3 binding and 4 non-binding +1 votes: Binding votes: Jason Gustafson, Matthias J. Sax, Guozhang Wang Non-binding votes: James Cheng, Mickael Maison, Ted Yu, Manikumar The vote thread can be found here: https://www.mail-archive.com/dev@kaf

[jira] [Created] (KAFKA-6767) OffsetCheckpoint write assumes parent directory exists

Steven Schlansker created KAFKA-6767: Summary: OffsetCheckpoint write assumes parent directory exists Key: KAFKA-6767 URL: https://issues.apache.org/jira/browse/KAFKA-6767 Project: Kafka

Re: [DISCUSS] KIP-280: Enhanced log compaction

Thanks for clarification. I understand the config name now. Makes sense. > bq. It might also be good, to elaborate why you suggest "long" for the > compaction value is the KIP itself. > > I would prefer that the definition simply be "long", to keep the code > contract cleaner, and leave the c

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

+1 (binding) -Matthias On 4/9/18 8:57 AM, Bill Bejeck wrote: > Thanks for the KIP. > > +1 > > -Bill > > On Mon, Apr 9, 2018 at 7:57 AM, zhenya Sun wrote: > >> +1 >> >> >> from my iphone！ >> On 04/09/2018 17:27, khaireddine Rezgui wrote: >> Hi guys, >> >> I created this thread to get your agr

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

My notes from today's meeting.  Sorry if I got anyone's name wrong. Plus I missed a few moments with noise at home and/or dropped video. -Ray = KIP-253 Discussion - Currently, adding partitions can cause keys to be read out-of-order.   This KIP is trying to preserve the key ordering when

Re: [DISCUSS] KIP-279: Fix log divergence between leader and follower after fast leader fail over

Hey Anna, Thanks for picking this up! I think the solution looks good to me. Just wanted to check my understanding on one part. When describing the handling of unclean leader elections, you mention comparing the "complete epoch lineage" from both brokers in order to converge on the log. I think th

Java 10 replacing Java 9 in Jenkins for trunk

Hi all, Java 10 was recently released and support for Java 9 has ended since it's not a LTS release. I've added a kafka-trunk Jenkins job for Java 10 and disabled the Java 9 job. I also added a PR Jenkins job for Java 10 and will soon disable the Java 9 PR job. The general idea is to have a separ

Re: [DISCUSS] KIP-281 ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

Hey Alex, Thanks for the KIP. Posting my reply which I accidentally added to another KIP (before this KIP version was corrected): I think the timeout controls the maximum allowed time that the consumer will block for the next record. Maybe the meaning would be clearer with the more concise name `

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

@Ted Yeah, you're right. Sorry about the confusion. Since we're here, I think this KIP is a nice improvement. It's definitely nice to have an easy way to check the version. That said, do we really need to support both `-version` and `--version`? The latter is consistent with our current tools. Al

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

Thanks for the KIP. +1 -Bill On Mon, Apr 9, 2018 at 7:57 AM, zhenya Sun wrote: > +1 > > > from my iphone！ > On 04/09/2018 17:27, khaireddine Rezgui wrote: > Hi guys, > > I created this thread to get your agreement about the new default value of > the retention_ms in kafka stream repartion topi

Build failed in Jenkins: kafka-trunk-jdk10 #4

See -- [...truncated 2.02 MB...] org.apache.kafka.connect.runtime.rest.RestServerTest > testAdvertisedUri STARTED org.apache.kafka.connect.runtime.rest.RestServerTest > testAdvertisedUri

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi folks. Here is a summary of where I think we stand on this KIP and what I believe it means for how we move forward. - There is some desire to use substitution more broadly beyond just JAAS module options. Specifically, cluster/producer/consumer config values such as ssl.keystore.pas

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

Hi Magnus, Thanks for the heads up, added the endianness to the KIP. Here is the current text: "Double A new type needs to be added to transfer quota values. Since the protocol classes in Kafka already uses ByteBuffers it is logical to use their functionality for serializing doubles. The serializ

[jira] [Created] (KAFKA-6766) Kafka offset moved backward

Janmejay Baral created KAFKA-6766: - Summary: Kafka offset moved backward Key: KAFKA-6766 URL: https://issues.apache.org/jira/browse/KAFKA-6766 Project: Kafka Issue Type: Bug Compone

Build failed in Jenkins: kafka-trunk-jdk8 #2538

See Changes: [github] MINOR: Java 10 fixes so that the build passes (#4839) -- [...truncated 3.52 MB...] kafka.utils.json.JsonValueTest > testAsJsonObjectOption STARTED

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

Hi Viktor, since serialization of floats isn't as straight forward as integers, please specify the exact serialization format of DOUBLE in the protocol docs (e.g., IEEE 754), including endianness (big-endian please). This will help the non-java client ecosystem. Thanks, Magnus 2018-04-09 15:16

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

Hi Attila, 1. It uses ByteBuffers, which in turn will use Double.doubleToLongBits to convert the double value to a long and that long will be written in the buffer. I'v updated the KIP with this. 2. Good idea, modified it. 3. During the discussion I remember we didn't really decide which one would

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

+1 from my iphone！ On 04/09/2018 17:27, khaireddine Rezgui wrote: Hi guys, I created this thread to get your agreement about the new default value of the retention_ms in kafka stream repartion topics. https://cwiki.apache.org/confluence/display/KAFKA/KIP- 284%3A+Set+default+retention+ms+for+Str

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

+1 Original message From: khaireddine Rezgui Date: 4/9/18 2:27 AM (GMT-08:00) To: dev@kafka.apache.org Subject: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE Hi guys, I created this thread to get your agreement about the new defau

[jira] [Created] (KAFKA-6765) Intermittent test failure in CustomQuotaCallbackTest

Rajini Sivaram created KAFKA-6765: - Summary: Intermittent test failure in CustomQuotaCallbackTest Key: KAFKA-6765 URL: https://issues.apache.org/jira/browse/KAFKA-6765 Project: Kafka Issue Ty

[VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

Hi guys, I created this thread to get your agreement about the new default value of the retention_ms in kafka stream repartion topics. https://cwiki.apache.org/confluence/display/KAFKA/KIP- 284%3A+Set+default+retention+ms+for+Streams+repartition+ topics+to+Long.MAX_VALUE Thanks,

Re: [DISCUSS] KIP-280: Enhanced log compaction

Hi, bq. About naming: the broker config has `cleaner` in it, while the topic config does not. Is might be more consistent if either both have `cleaner` or none of them? (Personally, I would prefer to strip `cleaner` for the broker config.) Do you mean the "log.cleaner." prefix attached to the

Re: [DISCUSS] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

I think we can. Original message From: khaireddine Rezgui Date: 4/9/18 1:12 AM (GMT-08:00) To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE thank you, can we proceed to a VOTE ? 2018-04-09 4:06

Re: [DISCUSS] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

thank you, can we proceed to a VOTE ? 2018-04-09 4:06 GMT+01:00 Matthias J. Sax : > Thank for the KIP. LGTM. > > -Matthias > > On 4/8/18 1:55 AM, khaireddine Rezgui wrote: > > KIP updated. > > > > 2018-04-08 9:47 GMT+01:00 khaireddine Rezgui : > > > >> ok > >> > >> 2018-04-08 1:04 GMT+01:00 Ted Y