Re: [DISCUSS] KIP-380: Detect outdated control requests and bounced brokers using broker generation

Hi Patrick, Thanks for the KIP! Fixing such correctness issues is always very welcome - they're commonly hard to diagnose and debug when they happen in production. I was wondering if I understood the potential correctness issues correctly. Here is what I got: - If a broker bounces during con

[DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

Hey everybody, I've opened up a very short KIP to make the Max and Min metrics' default values consistent with each other. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-386%3A+Make+Min+metrics%27+default+value+consistent+with+Max+metrics JIRA: https://issues.apache.org/jira/browse/KA

Re: [DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

e. > But its more of a bug rather than improvement proposal. Let's see what > contributors got to say. > > On Mon, Oct 22, 2018 at 7:23 PM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Hey everybody, > > > > I've opened up a ver

Re: [DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

it would still > be sound for the situation you're attempting to address. > > Just to throw it out there, `null` is technically also (maybe moreso) a > sound choice, but I'd be concerned about causing a bunch of null > dereference errors. > > Thanks again, > -John >

Re: [DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

Double.isNaN(metricValue)", > "metricValue != Double.NEGATIVE_INFINITY", and such to skip these values. > > I like John's suggestion for using NaN for the same reason that -INF > and +INF are still technically valid values as they are just the extreme > bounds.

Re: [DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

Hey everybody, Just wanted to say that I plan on starting a voting thread tomorrow if there aren't any further comments. Thanks for the discussion so far! On Wed, Oct 24, 2018 at 11:27 AM Stanislav Kozlovski wrote: > Hi Kevin, > > Thanks for providing context. > > I'

Re: [DISCUSS] KIP-351: Add --under-min-isr option to describe topics command

Hey Kevin, Thanks for the KIP! This sounds like a very useful feature to have and makes excellent use of the new AdminClient introduced in KIP-377 in my opinion. I believe it's perfectly fine to start a vote thread on the fact that KIP-377 was accepted already. On Mon, Nov 5, 2018 at 3:03 PM Kevi

[VOTE] KIP-386: Standardize on Min/Avg/Max metrics' default values

Hey everybody, I'm starting a vote thread on KIP-386: Standardize on Min/Avg/Max metrics' default values In short, after the discussion thread

Re: [VOTE] KIP-386: Standardize on Min/Avg/Max metrics' default values

his clear. > > > > Jun > > > > On Tue, Nov 6, 2018 at 1:00 AM, Stanislav Kozlovski < > > stanis...@confluent.io> > > wrote: > > > > > Hey everybody, > > > > > > I'm starting a vote thread on KIP-386: Standardize on Min/Avg/M

Re: [VOTE] KIP-386: Standardize on Min/Avg/Max metrics' default values

Hey everybody, I've finished with the implementation, here is the PR: https://github.com/apache/kafka/pull/5908 On Mon, Nov 12, 2018 at 6:58 PM Stanislav Kozlovski wrote: > Thank you, everybody, for the votes and discussion. The KIP has passed > with 3 binding votes (Harsha, Jun,

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Hey there everybody, Thanks for the introduction Boyang. I appreciate the effort you are putting into improving consumer behavior in Kafka. @Matt I also believe the default value is high. In my opinion, we should aim to a default cap around 250. This is because in the current model any consumer r

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

n Tue, Nov 20, 2018 at 1:47 PM Stanislav Kozlovski wrote: > Hey there everybody, > > Thanks for the introduction Boyang. I appreciate the effort you are > putting into improving consumer behavior in Kafka. > > @Matt > I also believe the default value is high. In my opinion, we sho

Re: [DISCUSS] KIP-391: Allow Producing with Offsets for Cluster Replication

Hey Edurdo, thanks for the KIP! I have some questions, apologies if they are naive: Is this intended to work for a single producer use case only? How would it work if two producers were producing to the same topic with offsets? How would it work if two producers, one with offsets and one without w

Re: [DISCUSS] KIP-391: Allow Producing with Offsets for Cluster Replication

lag is needed to distinguish a batch with a desired base offset of 0, > from a regular batch for which offsets need to be generated. > I would not restrict a principal to only send-with-offsets (by making that > mandatory via the ACL). > > Thanks > Edo & Mickael > >

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

g forward to discussions around it. > > Best, > Stanislav > > On Tue, Nov 20, 2018 at 1:47 PM Stanislav Kozlovski < > stanis...@confluent.io> wrote: > >> Hey there everybody, >> >> Thanks for the introduction Boyang. I appreciate the effort you are >>

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Subject: Re: [Discuss] KIP-389: Enforce group.max.size to cap member > > metadata growth > > > > Hey everybody, > > > > It's been a week since this KIP and not much discussion has been made. > > I assume that this is a straight forward change and I will open a

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Hi Boyang, The KIP looks very good. One small question I have is now that we have one and a half round-trips needed to join in a rebalance (1 full RT addition), is it worth it to consider increasing the default value of `group.initial.rebalance.delay.ms`? Best, Stanislav On Tue, Nov 27, 2018 at

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

t; > > > > > On Mon, Nov 26, 2018 at 2:44 AM Boyang Chen > wrote: > > > > > > > Thanks Stanislav for the update! One suggestion I have is that it > would > > > be > > > > helpful to put your > > > > > > > > reasoning on de

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

ebalance (1 full RT addition), is it worth it to > >> consider increasing the default value of ` > group.initial.rebalance.delay.ms`? > > I guess we could keep it for now. After KIP-345 and incremental > cooperative rebalancing > > work we should be safe to deprecate `group.initial.r

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

Hey Jason, This is certainly a very exciting KIP. I assume that no changes will be made to the offset commits and they will continue to be sent to the group coordinator? I also wanted to address metrics - have we considered any changes there? I imagine that it would be valuable for users to be ab

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

opportunity to commit > offsets before kicking them from the group? > > 2. Do you think we should make this a dynamic config? > > Thanks, > Jason > > > > > On Wed, Nov 28, 2018 at 2:42 AM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: &

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

;> 3. c1 handles the error and rejoins with assigned member.id; > >>> 4. Broker stores c1 in its group metadata; > >>> 5. Consumer c2 started. Join with UNKNOWN_MEMBER_ID; > >>> 6. Broker rejects while allocating a member.id to c2 in respons

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

ould kill two birds with one > stone in a generic way. What do you think? > > Boyang > > From: Stanislav Kozlovski > Sent: Monday, December 3, 2018 8:35 PM > To: dev@kafka.apache.org > Subject: Re: [Discuss] KIP-389: Enforce group.max.size to

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

rt+and+Policies > > > proposal. > > Let me know if this makes sense. > > Best, > Boyang > > From: Stanislav Kozlovski > Sent: Wednesday, December 5, 2018 5:52 PM > To: dev@kafka.apache.org > Subject: Re: [Discuss] KIP-389: Enforce group.max.size t

Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

This is great work, Boyang. Thank you very much. +1 (non-binding) On Mon, Dec 10, 2018 at 6:09 PM Boyang Chen wrote: > Hey there, could I get more votes on this thread? > > Thanks for the vote from Mayuresh and Mike :) > > Best, > Boyang > > From: Mayuresh Ghara

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

ning after the rebalance > completes. It sounds reasonable if I've understood correctly. > > Thanks, > Jason > > > > On Fri, Dec 7, 2018 at 6:47 AM Boyang Chen wrote: > > > Yep, LGTM on my side. Thanks Stanislav! > > >

Re: [DISCUSS] : KIP-410: Add metric for request handler thread pool utilization by request type

Hey there Mayuresh, Thanks for the KIP! This will prove to be very useful. I am wondering whether we should opt for the name of ` RequestHandlerPoolUsagePercent`. We seem to use the "Percent" suffix to denote fractions of times in other places - e.g `NetworkProcessorAvgIdlePercent` The average f

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

r config feature? Regarding that - I feel it's useful to have and I also think it might not introduce additional complexity. Ås long as we handle the config being changed midway through a rebalance (via using the old value) we should be good to go. On Wed, Dec 12, 2018 at 4:12 PM Stanislav Koz

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

________ > From: Stanislav Kozlovski > Sent: Friday, December 28, 2018 7:59 PM > To: dev@kafka.apache.org > Subject: Re: [Discuss] KIP-389: Enforce group.max.size to cap member > metadata growth > > Hey there everybody, let's work on wrapping this discussion up.

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

lculating the impact of group-size on rebalance > time in order to make good recommendations will take a significant > effort. On the other hand, informing users regarding the memory > footprint of a consumer in a group and using that to make a reasonable > suggestion isn't hard.

[DISCUSSION] KIP-412: Extend Admin API to support dynamic application log levels

Hey there everybody, I'd like to start a discussion about KIP-412. Please take a look at the KIP if you can, I would appreciate any feedback :) KIP: KIP-412 JIRA: KAFKA-7800 <

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

ld be good > to vote IMO. > > Best, > Boyang > ____ > From: Stanislav Kozlovski > Sent: Monday, January 7, 2019 4:21 PM > To: dev@kafka.apache.org > Subject: Re: [Discuss] KIP-389: Enforce group.max.size to cap member > metadata growth > > Hey

Re: [DISCUSSION] KIP-412: Extend Admin API to support dynamic application log levels

To differentiate between the normal Kafka config settings and the > application's log level settings, we will introduce a new resource type - > BROKER_LOGGERS > > Stanislav, can you explain why log level wouldn't be a "normal Kafka config > setting"? > > Ryanne &g

Re: [DISCUSSION] KIP-412: Extend Admin API to support dynamic application log levels

log level wouldn't be a "normal Kafka >> config >> setting"? >> >> Ryanne >> >> On Tue, Jan 8, 2019, 4:26 PM Stanislav Kozlovski > wrote: >> >> > Hey there everybody, >> > >> > I'd like to start a discu

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

arch > (Personally, I wouldn't want what is essentially a linked list that we > iterate to grow beyond 1M). > > Mostly likely, 99% of the users won't need it and those who do will > have the right information to figure things out (or at least, they'll > know ev

[VOTE] KIP-389: Introduce a configurable consumer group size limit

Hey folks, I'd like to initiate a vote thread about KIP-389 . -- Best, Stanislav

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Hey Konstantine, This is a very exciting and fundamental-improving KIP, thanks a lot for working on it! Have you seen KIP-345 ? I was wondering whether Connect would support the static group membership - potentially limiting the need to h

Re: [VOTE] KIP-389: Introduce a configurable consumer group size limit

at 4:16 AM Ismael Juma wrote: > +1 (binding), thanks. > > On Fri, Jan 11, 2019, 1:09 AM Stanislav Kozlovski wrote: > > > Hey folks, > > > > I'd like to initiate a vote thread about KIP-389 > > < > > > https://cwiki.apache.org/confluence/displ

Re: metric for total volume of messages per topic over a period of time

Hey there Amitav, thanks for the proposal. Just to be a bit more concrete on the proposed metrics, you would like to be able to see what the total bytes-in amount was for a given time period (e.g 60 seconds)? Do you mind sharing what the use case of such a metric is? This would require opening up

Re: [DISCUSS] KIP-316: Command-line overrides for ConnectDistributed worker properties

Hey Kevin, I am not yet a committer so my word doesn't hold much weight but to me this is a no-brainer of a decision - this KIP can only improve the cli tool +1 From: Kevin Lafferty Sent: Friday, 15 June 2018 3:10 To: dev@kafka.apache.org Subject: [DISCUSS] KIP-

Contributing: KIP permissions

Hey everybody, I've already implemented some JIRA tickets and would like to further my contributions into this great project by creating KIPs. Can I receive permission to do so? My e-mail: stanis...@confluent.io -- Best, Stanislav

[DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hey everybody, I just created a new KIP about exposing more information in exceptions caused by consumer record deserialization/validation. Please have a look at it, it is a very short page. I am working under the assumption that all invalid record or deserialization exceptions in the consumer pa

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

needed, maybe I'll outright remove the `cause` attribute On Thu, Jul 5, 2018 at 4:55 PM Ismael Juma wrote: > Thanks for the KIP, Stanislav. The following PR looks related: > > https://github.com/apache/kafka/pull/4093/files > > Ismael > > On Thu, Jul 5, 2018 at 8:44 AM Stan

[DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hey group, I just created a new KIP about adding customizable SASL extensions to the OAuthBearer authentication mechanism. More details in the proposal KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-342%3A+Add+support+for+Custom+SASL+extensions+in+OAuthBearer+authentication JIRA: KAFK

Re: KAFKA-6690 Priorities for Source Topics

Hey Nick, This seems like a non-trivial change and more importantly - a change in the public interface. As such, following our contributing guidelines , it warrants a KIP (Kafka I

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

ledging that it is unsupported. > > Thanks for the KIP and the simultaneous PR -- having both at the same time > really helped. > > Ron > > On Tue, Jul 17, 2018 at 6:22 PM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Hey group, >

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

ever is necessary to add > >> the desired SASL extensions to the Subject -- probably in the public > >> credentials -- where a custom sasl.client.callback.handler.class would > be > >> able to find them. The KIP might state this, too. > >> > >

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

ames are composed of only ALPHA [a-zA-Z] characters as per the spec (not > only for that reason, but to also make sure the token available at the > OAUTHBEARER.token negotiated property can't be overwritten). > > Ron > > On Thu, Jul 19, 2018 at 12:43 PM Stanislav Kozlovs

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

com/apache/kafka/pull/5379> Best, Stanislav On Thu, Jul 19, 2018 at 1:58 PM Stanislav Kozlovski wrote: > Hi Ron, > > Agreed. `SaslExtensionsCallback` will be the only public API addition and > new documentation for the extension strings. > A question that came up - should the

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

public and attach an instance of that to the > Subject's public credentials instead. > > Ron > > On Thu, Jul 19, 2018 at 8:15 PM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > I have updated the PR and KIP to address the comments made so far. P

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

I agree with Jason's notion that > implicit use of the empty group.id to commit offsets is more likely to be causing users unexpected problems than actually providing a useful capability. I was initially confused that this is the behavior when investigating a new-ish JIRA issue

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

lowing commit was an initial step in unifying the exception > handling: > > > https://github.com/apache/kafka/commit/96bcfdfc7c9aac075635b2034e65e412a725672e > > But more can be done as you mentioned. > > Ismael > > On 5 Jul 2018 9:36 am, "Stanislav Kozlovski&

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hi Ted, I do plan to start one. When is the appropriate time? My reasoning was that people would like to view the changes first On Fri, Jul 20, 2018, 6:21 PM Ted Yu wrote: > Hi, Stanislav: > Do you plan to start VOTE thread ? > > Cheers > > On Fri, Jul 20, 2018 at 6:11 PM S

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

> { > > >>this(Utils.parseMap(mapStr, keyValueSeparator, > elementSeparator), > > >> saslNameRegexPattern, saslValueRegexPattern); > > >>} > > >> > > >>public SaslExtensions(Map extensionsMap, Pattern > > >>

[DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey group, I created a new KIP about making log compaction more fault-tolerant. Please give it a look here and please share what you think, especially in regards to the points in the "Needs Discussion" paragraph. KIP: KIP-346

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

as the default group id (as > addressed by Jason in his response)? > This default group id would not support offset commits and consumers would > use `auto.offset.reset` config when there is no current offset. > > Thanks. > --Vahid > > > > > From: Stanislav Kozlovski

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

I renamed the KIP and that changed the link. Sorry about that. Here is the new link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-346+-+Improve+LogCleaner+behavior+on+error On Mon, Jul 23, 2018 at 5:11 PM Stanislav Kozlovski wrote: > Hey group, > > I created a new KIP about m

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hi Ted, Yes, absolutely. Thanks for pointing that out! On Mon, Jul 23, 2018 at 6:12 PM Ted Yu wrote: > For `uncleanable-partitions`, should the example include topic name(s) ? > > Cheers > > On Mon, Jul 23, 2018 at 5:46 PM Stanislav Kozlovski < > stanis...@confluent

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

lic API > >> should be sufficient in the vast majority of the cases. Ron, do you > agree? > >> > >> On Mon, Jul 23, 2018 at 11:35 AM, Ron Dagostino > >> wrote: > >> > >> > Hi Stanislav. See https://tools.ietf.org/html/rfc7628#section-3.

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

quest the extensions from its > callback handler, for backwards compatibility it is not necessary for the > callback handler to support SaslExtensionsCallback -- any > UnsupportedCallbackException that is thrown will be ignored and no > extensions will be added. > > Ron > > O

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

his reply, topic-partition name can be long. > It is not necessary to repeat the topic name for each of its partitions. > How about the following format: > > topic-name1-{partition1, partition2, etc} > > That is, topic name only appears once. > > Cheers > > On Mon, Ju

[VOTE] KIP-342 - Add support for custom SASL extensions in OAuthBearer authentication

Hey everbody, I'd like to start a vote thread for KIP-342 Add support for custom SASL extensions in OAuthBearer authentication -- Best, Stanislav

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

for default, unsecured bearer tokens - >`unsecuredLoginExtension_`. > > > 2. Can you add the package for SaslExtensionsCallback class? > > > On Tue, Jul 24, 2018 at 10:03 PM, Stanislav Kozlovski < > stanis...@confluent.io> wrote: > > > Hi Ron, > > > >

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

cases I've seen where a user hit a logic bug in the log cleaner, it was > > > just one partition that had the issue. We also should increase test > > > coverage for the log cleaner. > > > > > > > * About marking disks as offline when exceeding a certain threshold, >

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

isplay/KAFKA/Kafka+Improvement+Proposals#>? > > I tried, but the Wiki won't let me. > > -Ray > > On 7/26/18 2:01 PM, Stanislav Kozlovski wrote: > > Hey guys, > > > > @Colin - good point. I added some sentences mentioning recent > improvements > &

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

ike to view the changes first > > > > On Fri, Jul 20, 2018, 6:21 PM Ted Yu wrote: > > > > > Hi, Stanislav: > > > Do you plan to start VOTE thread ? > > > > > > Cheers > > > > > > On Fri, Jul 20, 2018 at 6:11 PM Stanislav Kozlo

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey group, I am planning on starting a voting thread tomorrow. Please do reply if you feel there is anything left to discuss. Best, Stanislav On Fri, Jul 27, 2018 at 11:05 PM Stanislav Kozlovski wrote: > Hey, Ray > > Thanks for pointing that out, it's fixed now > > Best

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

king" will be? If we're > going with the ".skip" file or equivalent, can we also add how > successful retries will behave? > > -Ray > > On 7/31/18 9:56 AM, Stanislav Kozlovski wrote: > > Hey group, > > > > I am planning on starting a vo

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey group, I just wanted to note that I have an implementation ready for review. Feel free to take a quick look and raise any concerns you might have in due time. I plan on starting the voting thread tomorrow. Best, Stanislav On Wed, Aug 1, 2018 at 10:01 AM Stanislav Kozlovski wrote: >

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Yes, good catch. Thank you, James! Best, Stanislav On Wed, Aug 1, 2018 at 5:05 PM James Cheng wrote: > Can you update the KIP to say what the default is for > max.uncleanable.partitions? > > -James > > Sent from my iPhone > > > On Jul 31, 2018, at 9:56 AM, S

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

s I feel it > > would warrant a bigger discussion > > > Ok, that's fair. I thought I'd suggest it here just to see if there was any > interest in the community. At least with this KIP, users have a viable way > to skip past bad data if they wish. > > -Jason > >

[VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hey everybody, I'd like to start a vote thread for KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation -- Best, Stanislav

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

RecordDeserializationException? > > Thanks, > Jason > > On Thu, Aug 2, 2018 at 5:45 AM, Ted Yu wrote: > > > +1 > > Original message From: Stanislav Kozlovski < > > stanis...@confluent.io> Date: 8/2/18 2:41 AM (GMT-08:00) To: > &

Re: [VOTE] KIP-342 - Add support for custom SASL extensions in OAuthBearer authentication

s the > customized fields in JWT. > > Jun > > On Wed, Jul 25, 2018 at 10:03 AM, Stanislav Kozlovski < > stanis...@confluent.io> wrote: > > > Hey everbody, > > > > I'd like to start a vote thread for KIP-342 Add support for custom SASL > > extensi

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

but in the case that a user has > explicitly > > > > > > > configured > > > > > > > the empty group, then we can just use an old version of the > > > > > OffsetCommit > > > > > > > API which still supports it. In a future r

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

that reflects that serialization is the problem. How about > RecordSerializationException? > > best, > Colin > > > On Thu, Aug 2, 2018, at 15:11, Stanislav Kozlovski wrote: > > Hi Jason and Ted, > > > > @Jason > > I did not oppose the idea as I'

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

r.logdir` ? On Thu, Aug 2, 2018 at 7:32 PM Ray Chiang wrote: > One more thing occurred to me. Should the configuration property be > named "max.uncleanable.partitions.per.disk" instead? > > -Ray > > > On 8/1/18 9:11 AM, Stanislav Kozlovski wrote: >

Re: [VOTE] KIP-342 - Add support for custom SASL extensions in OAuthBearer authentication

t; On Fri, Aug 3, 2018 at 2:13 AM, Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Thank you, Jun. Noted in the KIP. > > > > On Fri, Aug 3, 2018 at 2:28 AM Jun Rao wrote: > > > > > Hi, Stanislav, > > > > > > Thanks for t

[VOTE] KIP-346 - Improve LogCleaner behavior on error

Hey everybody, I'm starting a vote on KIP-346 -- Best, Stanislav

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding) Thanks! On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson wrote: > +1 Thanks Vahid. > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi all, > > > > I'd like to start a vote on KIP-289 to modify the default group id of > > KafkaCon

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

gt; > -Jason > > On Tue, Aug 7, 2018 at 11:17 AM, Colin McCabe wrote: > > > Hi Stanislav, > > > > On Sat, Aug 4, 2018, at 10:44, Stanislav Kozlovski wrote: > > > Hey Colin, > > > > > > It may be a bit vague but keep in mind we also raise the

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

re no objections, I'll update the KIP in due time On Thu, Jul 26, 2018 at 11:14 AM Rajini Sivaram wrote: > Looks good. Thanks, Stanislav. > > On Wed, Jul 25, 2018 at 7:46 PM, Stanislav Kozlovski < > stanis...@confluent.io > > wrote: > > > Hi Rajini, > &

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

case might exist (some kind of trace ID, for > example, to aid in debugging), someone might inadvertently hang themselves > if we give them the rope. The risk vs. reward value proposition here > doesn't feel like a good one at first glance. Thoughts? > > Ron >

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

. > > Ron > > On Wed, Aug 8, 2018 at 11:54 AM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Hey Ron, > > > > I fully agree that token validation is a serious security operation. > > Although, I believe allowing the user to do more va

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

handle > UnsupportedCallbackException for SaslExtensionsValidatorCallback for > backwards compatibility, but that should be ok. > > What do you think? > > > On Wed, Aug 8, 2018 at 5:06 PM, Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Hi Ron, >

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

preference > (assuming I',m not missing anything, which is certainly possible). > Thoughts? > > Ron > > > On Wed, Aug 8, 2018 at 12:39 PM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Hi Rajini, > > > > T

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

ons and then > implement > >> max.uncleanable.partitions.per.logdir in a follow-up change if it seemed > >> to be necessary? What do you think? > >> > >> regards, > >> Colin > >> > >> > >> On Sat, Aug 4, 2018, a

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

the size > of uncleanable disk space instead. What was the reason for rejecting that? > Intuitively, it seems like a better fit for a cleaner failure. It would > provide users some time to react to failures while still protecting them > from exhausting the disk. > > Thanks, &g

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

s all recognized callbacks before > > throwing > > > > UnsupportedCallbackException due to an unrecognized one. > > > > > > > > Note that the above issue does not arise if we simply want the > ability > > to > > > > va

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

tempt to use [auth] will result in an exception" might cause > > confusion and perhaps needs to be clarified to state that the exception > > occurs on the client side before the extensions are sent to the server > > rather than during extension validation on the se

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

. Btw, on error, we shouldn't care about the values in the returned > extensions at all, we should simply fail authentication. > > > On Sat, Aug 11, 2018 at 1:21 PM, Stanislav Kozlovski < > stanis...@confluent.io > > wrote: > > > Hi, > > > > @Ron > > A

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

uncleanable disk space is more useful. > >> I'm not sure if we can set that threshold to be of certain size (e.g > >> 100GB) > >> as log directories might have different sizes. Maybe a percentage would > >> be > >> better then (e.g 30% of whole log dir size), WDYT?

Re: [VOTE] KIP-346 - Improve LogCleaner behavior on error

2018 at 9:48 AM, Dhruvil Shah > wrote: > > > Thanks for the KIP, Stanislav! +1 (non-binding) > > > > - Dhruvil > > > > On Mon, Aug 13, 2018 at 9:39 AM Colin McCabe wrote: > > > > > +1 (non-binding) > > > > > > best, >

Re: [VOTE] KIP-346 - Improve LogCleaner behavior on error

ns-count is per logDir, but uncleanable-bytes is not. > Should we make them consistent? > > Jun > > > On Wed, Aug 22, 2018 at 4:15 AM, Stanislav Kozlovski < > stanis...@confluent.io > > wrote: > > > Hi everybody, > > > > @Jason - I've updated the

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, I really like this KIP, it is very needed. I am still looking through it but unfortunately I am not too familiar with the networking code to evaluate your solution. I'd like to ask what happens if re-authentication consistently fails. Would we retry endlessly? Since the functionality for

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Rajini Thanks for summarizing the discussion so far, Ron! 1) How often do we have such long-lived connection idleness (e.g 5-10 minutes) in practice? 3) I agree that retries for re-authentication are useful. 4) The interleaving of requests sounds like a great feature to have, but the tr

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Thanks for the KIP, Ron! +1 (non-binding) On Fri, Sep 21, 2018 at 5:26 PM Ron Dagostino wrote: > Hi Everyone. This KIP requires 1 more binding up-vote to be considered for > the 2.1.0 release; please vote before the Monday deadline. > > The current vote is 2 binding +1 votes (Rajini and Harsha)

Re: [ANNOUNCE] New committer: Colin McCabe

Congrats Colin! On Tue, Sep 25, 2018 at 9:51 AM Edoardo Comar wrote: > Congratulations Colin ! > -- > > Edoardo Comar > > IBM Event Streams > IBM UK Ltd, Hursley Park, SO21 2JN > > > > > From: Ismael Juma > To: Kafka Users , dev > Date: 2

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

IP as I feel it > >>> would warrant a bigger discussion > >> > >> > >> Ok, that's fair. I thought I'd suggest it here just to see if there was > any > >> interest in the community. At least with this KIP, users have a viable > way >

Re: [VOTE] KIP-346 - Improve LogCleaner behavior on error

Hi, I just wanted to remind everybody that the this KIP has been accepted. The only thing left is for it to pass PR reviews and get merged for 2.1. PR: https://github.com/apache/kafka/pull/5439 On Fri, Aug 24, 2018 at 11:12 AM Stanislav Kozlovski wrote: > Hi Jun, > > Yes, my intenti

  1   2   3   4   >