Jenkins build is back to normal : kafka-trunk-jdk8 #3877

See

Build failed in Jenkins: kafka-2.3-jdk8 #98

See Changes: [wangguoz] HOTFIX: AssignedStreamsTasksTest lacks one parameter -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

I thought about it more. I feel that no matter how we refactor the code (with or without KIP-383 integrated), ultimately the need of customizing loading for keys and certs will still remain. Whenever that need arises we might end up thinking about the solution suggested by our KIP-486. Hence regard

Re: KIP-382: MirrorMaker 2.0 progress to delivery?

Andrew, thanks for your continued interest in MM2 :) My plan is to bug the committers after this US holiday weekend. Ryanne On Wed, Aug 28, 2019 at 10:57 AM Andrew Schofield wrote: > Hi, > KIP-382 (MirrorMaker 2.0) has been approved for a while now but the code > hasn’t yet made it into Kafk

Re: [VOTE] KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting

Thanks for the votes all! With two binding votes we’re in need of one more for the KIP to be accepted. With the 2.4 release coming in September, it would be great to get another committer to take a look soon so I could set aside some time to get implementation/documentation done to make it into

[jira] [Resolved] (KAFKA-8828) [BC Break] Global store returns a TimestampedKeyValueStore in 2.3

[ https://issues.apache.org/jira/browse/KAFKA-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcos Passos resolved KAFKA-8828. -- Resolution: Invalid > [BC Break] Global store returns a TimestampedKeyValueStore in 2.3 > -

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

On that, I actually looked at KIP-383 before briefly. However, that sounded like lot of changes suggested. One "key" thing we have to keep in mind is - IF we need lot of customization Kafka already allows you to use your SslProvider via ssl.providers or the changes done by KIP-492 and SSLContext.

RE: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

KIP-383 in its present form was vetoed because it was not possible to add validation of custom properties in a future KIP. The solution to that is the first proposal I wrote for KIP-383 which made the whole SslFactory pluggable. That first solution was also vetoed hence the deadlock. Replacing

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Maulin, In SSL scenarios, I imagine security providers introduced by KIP-492 are likely to be most useful when you want to use third party providers. The biggest advantage of the config from that KIP is that you don't need to write much code to integrate existing security providers into Kafka b

[DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

Hi, this is my first KIP for a change in Apache Kafka, so I'm really need to the process. Looking forward to hearing from you and learn the best ropes here. I would like to propose this KIP-515 to enable the ZookeeperClients to take full advantage of the TLS communication in the new Zookeeper 3.5

Re: [ DISCUSS ] KIP-512:Adding headers to RecordMetaData

Hi Colin, yes we agree but RecordMetadata in Interceptors and callbacks will not have headers which gives context on for which record this MetaData belongs to. To fill that Gap, we are proposing these changes. Thanks Renuka M On Thu, Aug 29, 2019 at 10:20 AM Colin McCabe wrote: > As Gwen com

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Harsha Thank you. Appreciate your time and support on this. Let me go back do some more research and get back to you on the KeyStore interface part. Basically, if we return certs and keys in the interface then Kafka code will have to build KeyStore object - which is also reasonable. Thanks Mau

Re: Kafka Connect Schema from JSON

In case this is helpful, I wrote a (WIP) version that does what you say, but using JSONSchema instead of avro schema. https://github.com/ottomata/kafka-connect-jsonschema On Thu, Aug 29, 2019 at 12:48 PM Josef Hak wrote: > Hi, > please is it possible to create kafka connect Schema using json, e

Re: [ DISCUSS ] KIP-512:Adding headers to RecordMetaData

As Gwen commented earlier, the client already has the record that it sent, including all the headers. > > Future future = producer.send(myRecord, null); > future.get(); > System.out.println("I sent myRecord with headers " + myRecord.headers()); > best, Colin On Tue, Aug 27, 2019, at 17:06, Ren

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

On Mon, Aug 26, 2019, at 14:03, Jason Gustafson wrote: > Hi Arjun, > > From a high level, I feel like we are making light of the JMX api because > it's convenient and the broker already has it. Personally I would take the > broker out of the picture. The JMX endpoint is not something we were happy

[jira] [Created] (KAFKA-8848) Update system test to use new authorizer

Rajini Sivaram created KAFKA-8848: - Summary: Update system test to use new authorizer Key: KAFKA-8848 URL: https://issues.apache.org/jira/browse/KAFKA-8848 Project: Kafka Issue Type: Sub-task

Re: Permissions to create a KIP for KAFKA-8843

Working on it, looking forward to do my best with my first KIP, I will send details for review as soon as I have them. -- Pere Missatge de Colin McCabe del dia dj., 29 d’ag. 2019 a les 19:06: > On Thu, Aug 29, 2019, at 09:27, Pere Urbón Bayes wrote: > > Thanks, > > yes I know the KIP-500,

[jira] [Created] (KAFKA-8847) Deprecate and remove usage of supporting classes in kafka.security.auth

Rajini Sivaram created KAFKA-8847: - Summary: Deprecate and remove usage of supporting classes in kafka.security.auth Key: KAFKA-8847 URL: https://issues.apache.org/jira/browse/KAFKA-8847 Project: Kafk

Re: Permissions to create a KIP for KAFKA-8843

On Thu, Aug 29, 2019, at 09:27, Pere Urbón Bayes wrote: > Thanks, > yes I know the KIP-500, how realistic is to have it landing for 2.4? if > not really, end of the day we will need something like KAFKA-8843 for this > version. > > Looking forward to help out. > Hi Pere, Thanks for contributi

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Maulin, Use cases are clear now. I am +1 for moving forward with the discussions on having such configurable option for users. But the interfaces is proposed doesn't look right to me. We are still talking about keystore interfaces. Given keystore's are used as filebased way

[VOTE] KIP-486: Support custom way to load KeyStore and TrustStore

Hi all After a good discussion on the KIP at https://www.mail-archive.com/dev@kafka.apache.org/msg99126.html I think we are ready to start voting. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-486%3A+Support+custom+way+to+load+KeyStore+and+TrustStore The KIP proposes - supporting cu

Re: [DISCUSS] KIP-511: Collect and Expose Client's Name and Version in the Brokers

On Fri, Aug 23, 2019, at 00:07, Magnus Edenhill wrote: > Great proposal, this feature is well overdue! > > 1) > From an operator's perspective I don't think the kafka client > implementation name and version are sufficient, > I also believe the application name and version are of interest. > You c

Kafka Connect Schema from JSON

Hi, please is it possible to create kafka connect Schema using json, e.g. avro definition json? I need it for writing custom Single Message Transformation and can not find how to do that. I miss something like method in Schema loadFromJson(String avroDefJson). Without it probably I will have to wr

Re: Permissions to create a KIP for KAFKA-8843

Thanks, yes I know the KIP-500, how realistic is to have it landing for 2.4? if not really, end of the day we will need something like KAFKA-8843 for this version. Looking forward to help out. -- Pere Missatge de Guozhang Wang del dia dj., 29 d’ag. 2019 a les 17:35: > Hello Pere, > > Thanks

Re: Permissions to create a KIP for KAFKA-8843

Hello Pere, Thanks for you interest in contributing to Kafka, I've added you to the contributors list and you should be able to create wiki pages now. BTW there's an on-going KIP-500 which aims at removing ZK dependency of Kafka recently; depending when it would be voted and be adopted, I think i

Permissions to create a KIP for KAFKA-8843

Hi, I would like to create, and start the process to fix KAFKA-8843, so as I understand it I should first create a KIP. I registered to the wiki: email: pere.ur...@gmail.com user: pere.urbon would you be so nice to provide permissions to create the KIP? thanks a lot, -- Pere Urbon-Bayes

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Rajan Your email format doesn't show up correctly. Can you repost to make it more readable? Thanks Maulin On Wed, Aug 28, 2019 at 2:32 PM Rajan Dhabalia wrote: > *Hi Harsha/Maulin,I am following KIP-486 and KIP-492 and it seems > https://github.com/apache/kafka/pull/7090 >

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

Hey Yishun! Glad to see this is in the works :) Within the past month or so, needing state stores for unit tests has been brought up multiple times. Unfortunately, before now some people had to rely on internal APIs to get a store for their tests, which is unsafe as they can (and in this case