[DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi, I would like to start a discussion for KIP-297 to externalize secrets from Kafka Connect configurations. Any feedback is appreciated. < https://cwiki.apache.org/confluence/display/KAFKA/KIP-297%3A+Externalizing+Secrets+for+Connect+Configurations > JIRA:

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

igured > to connect to it? I assume that's out of scope for this proposal but wanted > to clarify it. > > Thanks > Magesh > > On Tue, May 8, 2018 at 1:52 PM, Robert Yokota wrote: > > > Hi, > > > > I would like to start a discussion for KIP-297 to exte

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

If so, trying to understand > > the intent of the config() in SourceTaskContext and the SinkTaskContext > > 3. What if the provider itself needs some kind of secrets to be > configured > > to connect to it? I assume that's out of scope for this proposal but > wanted >

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

/4990/files. I also added an example of a FileConfigProvider to the KIP. Thanks, Robert On Wed, May 9, 2018 at 10:04 AM, Robert Yokota wrote: > Hi Magesh, > > Thanks for the feedback! > > I will put together a PR to demonstrate what the implementation might look > like, as w

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

d to ConfigTransformer: do we need to include all this code in > the KIP? Seems like an implementation detail. > > > Other connectors such as the S3 connector are tightly coupled with a > particular secret manager, and may > > wish to handle rotation on their own. > >

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

remote systems anyway), but regardless, that class > should be useful at least in some minimal sense if not more than that. See > https://github.com/apache/kafka/pull/4994. > > Ron > > Ron > > On Tue, May 15, 2018 at 8:01 PM, Robert Yokota wrote: > > > Hi Colin, > &g

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

just returns a map of >key-value pairs. Providers that return metadata could extend it to > provide >metadata in a meaningful format instead of Map. >3. For ZK, we would use ConfigProvider.get() without `keys` to get all >keys in the path. Do we have two get() methods si

[VOTE] KIP-297: Externalizing Secrets for Connect Configurations

Hello everyone, After a good round of discussions with excellent feedback and no major objections, I would like to start a vote on KIP-297 to externalize secrets from Kafka Connect configurations. My thanks in advance! KIP: < https://cwiki.apache.org/confluence/display/KAFKA/KIP-297%3A+Externali

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

ositive delayMs > indicates > >// that a future change is anticipated (such as a lease duration) > >void onChange(String path, Map values, int delayMs); > > Do we really need delayMs? It seems like if you get a callback with > delayMs set, you don't know what

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

start the > Connector when the keys are actually changed? Or is the concern that > this would lengthen the effective key rotation time? Can’t the user > just configure a slightly shorter key rotation time to counteract > this concern? > Regards, > Colin > > On Wed, May 16, 20

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

t;lease duration" mean for a configuration key? > Does that mean Vault will reject changes to the configuration key if I try > to make them within the lease duration? Or is this like a period after > which a password is automatically rotated? > > On Wed, May 16, 2018, at

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

Hi Colin, I've changed the KIP to have a composite object returned from get(). It's probably the most straightforward option. Please let me know if you have any other concerns. Thanks, Robert On Thu, May 17, 2018 at 11:44 AM, Robert Yokota wrote: > > > Hi Colin, > >

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

ct: Re: [VOTE] KIP-297: Externalizing Secrets > > for Connect Configurations > > Thanks Robert, this looks great > > > > +1 (non-binding) > > > > On Thu, May 17, 2018 at 5:35 PM, Colin McCabe > wrote: > > > > > Thanks, Robert! > > > >

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

t 6:19 PM Ted Yu wrote: > > > > > +1 > > > Original message From: Magesh Nandakumar < > > > mage...@confluent.io> Date: 5/17/18 6:05 PM (GMT-08:00) To: > > > dev@kafka.apache.org Subject: Re: [VOTE] KIP-297: Externalizing > Secrets &g

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

KIP description. > > - Konstantine > > > > On Wed, May 16, 2018 at 10:33 AM, Rajini Sivaram > wrote: > > > Thanks for the update, Robert. Looks good to me. > > > > Regards, > > > > Rajini > > > > On Wed, May 16, 2018 at 4:43

Please add me to the contributor list

Hi, Can someone please add me to the contributor list? I want to assign KAFKA-2925 to myself so I can close it out. Thanks, Robert Yokota

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

varam wrote: > > > Hi Konstantine, > > > > > > Sounds reasonable to me too. > > > > > > Regards, > > > > > > Rajini > > > > > > On Mon, Jun 11, 2018 at 7:55 PM, Robert Yokota > > wrote: > > >

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

ding this > behavior to FileConfigProvider seems like it might give someone who > accidentally configures a directory rather than a file a nasty surprise. > How about adding a DirectoryConfigProvider which adds this behavior? > > best, > Colin > > > On Tue, Sep 4, 2018, a

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Thanks for the great KIP Konstantine! +1 (non-binding) Robert On Thu, Mar 7, 2019 at 2:56 PM Guozhang Wang wrote: > Thanks Konstantine, I've read the updated section on > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect > and i

Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

al we could fallback to backslash to escape > back-ticks? > > What do you think? > > Many thanks, > Jorge. > > > On Wed, 20 Jul 2022 at 23:03, Robert Yokota wrote: > > > Hi, > > > > I'm late to this thread, but would like to comment on the double

Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

Hi, I'm late to this thread, but would like to comment on the double dot/double asterisk syntax. Unfortunately double dot is often used in JSON Path as a descendant selector, see https://www.ietf.org/id/draft-ietf-jsonpath-base-05.html I think a better notation would be to use backticks to quote

[jira] [Resolved] (KAFKA-7512) java.lang.ClassCastException: java.util.Date cannot be cast to java.lang.Number

[ https://issues.apache.org/jira/browse/KAFKA-7512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Yokota resolved KAFKA-7512. -- Resolution: Duplicate > java.lang.ClassCastException: java.util.Date cannot be cast

[jira] [Created] (KAFKA-6886) Externalize Secrets for Kafka Connect Configurations

Robert Yokota created KAFKA-6886: Summary: Externalize Secrets for Kafka Connect Configurations Key: KAFKA-6886 URL: https://issues.apache.org/jira/browse/KAFKA-6886 Project: Kafka Issue

[jira] [Created] (KAFKA-6894) Cannot access GlobalKTable store from KStream.transform()

Robert Yokota created KAFKA-6894: Summary: Cannot access GlobalKTable store from KStream.transform() Key: KAFKA-6894 URL: https://issues.apache.org/jira/browse/KAFKA-6894 Project: Kafka

[jira] [Resolved] (KAFKA-2925) NullPointerException if FileStreamSinkTask is stopped before initialization finishes

[ https://issues.apache.org/jira/browse/KAFKA-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Yokota resolved KAFKA-2925. -- Resolution: Cannot Reproduce I wasn't able to reproduce the NPE and by reviewing the co

[jira] [Resolved] (KAFKA-6407) Sink task metrics are the same for all connectors

[ https://issues.apache.org/jira/browse/KAFKA-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Yokota resolved KAFKA-6407. -- Resolution: Duplicate > Sink task metrics are the same for all connect

[jira] [Created] (KAFKA-7370) Enhance FileConfigProvider to read a directory

Robert Yokota created KAFKA-7370: Summary: Enhance FileConfigProvider to read a directory Key: KAFKA-7370 URL: https://issues.apache.org/jira/browse/KAFKA-7370 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-7370) Enhance FileConfigProvider to read a directory

[ https://issues.apache.org/jira/browse/KAFKA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Yokota resolved KAFKA-7370. -- Resolution: Won't Do > Enhance FileConfigProvider to read a d

[jira] [Created] (KAFKA-7476) SchemaProjector is not properly handling Date-based logical types

Robert Yokota created KAFKA-7476: Summary: SchemaProjector is not properly handling Date-based logical types Key: KAFKA-7476 URL: https://issues.apache.org/jira/browse/KAFKA-7476 Project: Kafka

[jira] [Created] (KAFKA-8404) Authorization header is not passed in Connect when forwarding REST requests

Robert Yokota created KAFKA-8404: Summary: Authorization header is not passed in Connect when forwarding REST requests Key: KAFKA-8404 URL: https://issues.apache.org/jira/browse/KAFKA-8404 Project