Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

[Manikumar]

+1 (binding), Thanks for the KIP.

Thanks,

On Sat, Sep 21, 2019 at 2:49 AM Colin McCabe  wrote:

> +1 (binding).  Thanks, Rajini.
>
> best,
> Colin
>
> On Fri, Sep 20, 2019, at 00:43, Rajini Sivaram wrote:
> > Hi all,
> >
> > I would like to start vote on KIP-525 to return configs in CreateTopics
> > response. This is a minor KIP that returns additional data in the
> response
> > without breaking compatibility.
> >
> >-
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
> >
> >
> > Thank you,
> >
> > Rajini
> >
>

Build failed in Jenkins: kafka-trunk-jdk8 #3916

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8609: Add consumer rebalance metrics (#7347)

--
[...truncated 2.63 MB...]
org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestampWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareValueTimestamp 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareValueTimestamp 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWi

RE: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

[Pellerin, Clement]

I may have found the technical reason.

It would be logical if the customizable reconfigurable extension point extends 
the Reconfigurable interface. That's what the first iteration of KIP-383 
proposed. It lost its vote because the voters wanted to preserve the 
reconfiguration checks in SslFactory. The next thing to try is to push those 
checks in the creator which is the channel builder. A Reconfigurable instance 
is mutable by definition. It can either be in its current configuration, or the 
reconfigured configuration but not in between. The checks require to partially 
build the new configuration before accepting it and therefore this approach 
does not work.

We could move the checks in a helper validator class, then we would rely on the 
Reconfigurable instance to call that code. This would reuse the code but the 
checks would not be mandatory.

The conclusion is the checks are mandatory or the interface extends the 
Reconfigurable interface but not both.

Is that reasoning what you were saying?
Do we make the checks mandatory or not?
Do we support all the use cases we want?

-Original Message-
From: Pellerin, Clement 
Sent: Friday, September 20, 2019 5:24 PM
To: dev@kafka.apache.org
Subject: RE: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

The KIP now says: We believe that making SSLEngine creation pluggable is worth 
to allow SSL experts to write their own implementation having the SSL domain 
knowledge and keep them free of knowing much about Kafka's reconfigurability. 

The other custom reconfigurable extension points don't seem to have a problem 
with that. You may have a point though, so I need to look at the 
reconfiguration code you mention.

Is your latest prototype available so I can study it?

-Original Message-
From: Maulin Vasavada [mailto:maulin.vasav...@gmail.com] 
Sent: Friday, September 20, 2019 4:40 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

Thanks Clement for your thoughts. According to my current experience
rewriting the code twice I would say I did what you suggest in the last
point - " We must make an attempt, if only to explain why it fails in the
Rejected Alternatives section of the KIP." In the rejected alternatives I
already mention why not merge SslFactory and SslEngineFactory and make
SslFactory reconfigurable.

@Colin can you please express your thoughts on this discussion so far?
Since you refactored the SslFactory's code to have SslEngineBuilder I feel
you would have more insights into future changes.

Thanks
Maulin


On Fri, Sep 20, 2019 at 7:29 AM Pellerin, Clement 
wrote:

> There will be chaperon code in the base class of the channel builders.
> The arguments you gave me are emotional not technical.
> The SSL extension point is reconfigurable hence it should extend
> Reconfigurable.
> We will encounter issues when we try to prototype it that way.
> We will solve the issues or backtrack to another solution.
> We must make an attempt, if only to explain why it fails in the Rejected
> Alternatives section of the KIP.
>
> -Original Message-
> From: Maulin Vasavada [mailto:maulin.vasav...@gmail.com]
> Sent: Friday, September 20, 2019 2:40 AM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP-519: Make SSL context/engine configuration
> extensible
>
> Overall my thinking is - When somebody wants to customize creation of
> SSLEngine, most likely they are more expert in dealing with SSL domain
> related stuff than "Kafka's reconfigurability" aspect. As a custom
> implementation it makes more sense to me to say - Hey I'll control how I
> initialize my SSL context/engine and btw if Kafka can give me a way to just
> get re-created whenever some set of config keys changes, it is great! This
> is similar to my thinking on KIP-486 which is- as a Kafka operator I am
> just trying to be compliant to my company's security policies to load
> keys/certs in certain way. For that, I should not be penalized by Kafka to
> know all about Java security Providers and how to really create SSLContext
> object etc given Java already provides a way to feed in KeyStore object
> regardless of how I load it.
>
> On Thu, Sep 19, 2019 at 10:57 PM Maulin Vasavada <
> maulin.vasav...@gmail.com>
> wrote:
>
> > Hi Clement
> >
> > There will be good amount of state in the SslEngineFactory's default
> > implementation. Hence I feel we might anyway have a chaperon class to
> > provide reconfigurable functionality and will have one more class to host
> > the state/behavior of actual SSLContext/SSLEngine creation. While doing
> the
> > internal rewrite (so far two times) both of the times I reached to the
> same
> > conclusion.I feel if we leave the reconfigurations to the implementation
> -
> > it will repeat the same pattern of having two classes to manage it -
> since
> > most likely they will also have similar state information. Instead keep
> > that reconfigurations in SslFactory as i