[jira] [Created] (KAFKA-9476) the error "Scanner SubTypeScanner was not configured" caused by invalid jar when starting worker

2020-01-26 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-9476:
-

 Summary: the error "Scanner SubTypeScanner was not configured" 
caused by invalid jar when starting worker
 Key: KAFKA-9476
 URL: https://issues.apache.org/jira/browse/KAFKA-9476
 Project: Kafka
  Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


org.reflections 0.9.12 (this 
[commit|https://github.com/apache/kafka/commit/a024e679c7207aee2242d72aef53dc441c60ca27]
 updates org.reflections from 0.9.11 to 0.9.12) has a 
[issue|https://github.com/ronmamo/reflections/issues/273] which throws an error 
when getting sub classes from invalid plugin (jar file). By contrast, 
org.reflections 0.9.11 does not. It is a breaking change, and Kafka Worker is 
impacted. The unit test in attached patch works with org.reflections 0.9.11 but 
it fails with org.reflections 0.9.12.

I noticed this issue when running IT of my project with latest Kafka. It is not 
a critical breaking but it would be better to keep the behavior compatibility. 
Hence, it seems to me that we can revert the update of org.reflections and then 
wait for the [issue|https://github.com/ronmamo/reflections/issues/273].

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[VOTE] KIP-546: Add quota-specific APIs to the Admin Client, redux

2020-01-26 Thread Brian Byrne
Hello all,

I'd like to start a vote on KIP-456: Add quota-specific APIs to the Admin
Client, redux

The KIP is here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+quota-specific+APIs+to+the+Admin+Client%2C+redux

The discussion thread is here:
https://lists.apache.org/thread.html/802de7a2167677bf647a729df74e828b6acd4185f2eb0a25ddb939f6%40%3Cdev.kafka.apache.org%3E

Thanks,
Brian


Re: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

2020-01-26 Thread Maulin Vasavada
Hi all

I will start the voting thread on this now.

Thanks
Maulin

On Thu, Jan 23, 2020 at 12:51 AM Maulin Vasavada 
wrote:

> Hi all,
>
> I have updated the KIP document with the current state of conclusions.
> Please review it and see if we are ready to move to Voting!
>
> Thanks
> Maulin
>
> On Wed, Jan 22, 2020 at 12:42 AM Maulin Vasavada <
> maulin.vasav...@gmail.com> wrote:
>
>> Hi all,
>>
>> Finally I squeezed time and I've a suggested code changes shown at
>> https://github.com/maulin-vasavada/kafka/pull/4/files for discussing
>> this further. I'll update the KIP document soon. Meanwhile, can you please
>> take a look and continue the discussion?
>>
>> One challenge is at:
>> https://github.com/maulin-vasavada/kafka/pull/4/files#diff-1e3432211fdbb7b2e2b44b5d8838a40bR89
>>
>> Thanks
>> Maulin
>>
>>
>> On Tue, Oct 22, 2019 at 11:13 PM Maulin Vasavada <
>> maulin.vasav...@gmail.com> wrote:
>>
>>> bump! Clement/Rajini? Any responses based on the latest posts?
>>>
>>> On Wed, Oct 16, 2019 at 10:58 PM Maulin Vasavada <
>>> maulin.vasav...@gmail.com> wrote:
>>>
 bump!

 On Sun, Oct 13, 2019 at 11:16 PM Maulin Vasavada <
 maulin.vasav...@gmail.com> wrote:

> Hi Clement
>
> 1) existing validation code will remain in SslFactory
> 2) the createEngine() method in SslEngineBuilder will move to
> SslFactory and the client/server mode setting will go there (I documented
> this in the latest KIP update)
>
> In the current KIP I am proposing (as per the latest updates) to make
> SSLContext loading/configuration/creation pluggable. I am not suggesting 
> we
> do/repeat anything that is already addressed by the existing Providers for
> SSLContext implementation. The createEngine() method (which will move to
> SslFactory) will call SslContextFactory.create() to get references to the
> SSLContext and then call SSLContext#createEngine(peer, host) and set
> client/server mode as it does today. I'll try to put that in a sequence
> diagram and update the KIP to make it clearer.
>
> So to your question about SslFactory returning SSLContext - I am
> saying register SslContextFactory interface to provide the SSLContext
> object instead and keep SslFactory more-or-less as it is today with some
> additional responsibility of createEngine() method.
>
> Thanks
> Maulin
>
> Thanks
> Maulin
>
>
>
>
> On Fri, Oct 11, 2019 at 6:17 AM Pellerin, Clement <
> clement_pelle...@ibi.com> wrote:
>
>> Can you clarify a few points for me?
>>
>> The two stumbling blocks we have are:
>> 1) reuse of the validation code in the existing SslFactory
>> 2) the client/server mode on the SSLEngine
>>
>> How do you deal with those issues in your new proposal?
>>
>> My use case is to register a custom SslFactory that returns an
>> SSLContext previously created elsewhere in the application. Can your new
>> proposal handle this use case?
>>
>> -Original Message-
>> From: Maulin Vasavada [mailto:maulin.vasav...@gmail.com]
>> Sent: Friday, October 11, 2019 2:13 AM
>> To: dev@kafka.apache.org
>> Subject: Re: [DISCUSS] KIP-519: Make SSL context/engine configuration
>> extensible
>>
>> Check this out-
>>
>> https://github.com/apache/httpcomponents-core/blob/master/httpcore5/src/main/java/org/apache/hc/core5/ssl/SSLContextBuilder.java#L349
>>
>> This is exactly what I mean by using existing provider's SSLContext
>> implementation and customizing it with our data points. The similar
>> thing
>> Kafka's SslEngineBuilder is doing right now.
>>
>> On Thu, Oct 10, 2019 at 11:06 PM Maulin Vasavada <
>> maulin.vasav...@gmail.com>
>> wrote:
>>
>> > You meant JSSE not JCE right? We are not talking about cryptographic
>> > providers we are talking about ssl providers hence JSSE.
>> >
>> > I do understand how JSSE Providers work and also the impact of
>> multiple
>> > JSSE providers with same algorithms in same JVM along with
>> sequencing
>> > challenges for the same.
>> >
>> > Like you said- we need to allow customizing the configuration for
>> > SSLContext, so how many ways we have?
>> >
>> > Option-1: Write a custom JSSE Provider with our SSLContext
>> >
>> > Option-2: Use whichever SSLContext impl that you get from existing
>> JSSE
>> > Provider for SSLContext AND customize data for key material, trust
>> material
>> > AND secure random.
>> >
>> > Which one you prefer for this context?
>> >
>> > I feel we are making it complicated for no reason. It is very
>> simple -
>> > When we need to have SSL we need data points like - 1) Keys, 2)
>> Trust certs
>> > and 3) Secure Random which is feed to SSLContext and we are done.
>> So we can
>> > keep existing Kafka implementa

[VOTE] KIP-519: Make SSL context/engine configuration extensible

2020-01-26 Thread Maulin Vasavada
Hi all

After a good discussion on the KIP at
https://www.mail-archive.com/dev@kafka.apache.org/msg101011.html I think we
are ready to start voting.

KIP:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=128650952

The KIP proposes - Making SSLEngine creation pluggable to support
customization of various security related aspects.

Thanks
Maulin