Jenkins build is back to normal : kafka-2.1-jdk8 #179

See

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

Hi Andrew, Thanks for the KIP. I'm not too familiar with the internals of KC so I hope you can clarify a couple of things: - It seems the KIP is proposing a new interface because the existing "stop()" interface doesn't fully perform what it should ideally be doing. Is that a fair stateme

Re: [DISCUSS] 2.2.1 Bug Fix Release

If there are no objections on the proposed plan, I'll start preparing the first release candidate. Thanks, --Vahid On Thu, Apr 25, 2019 at 6:27 AM Ismael Juma wrote: > Thanks Vahid! > > On Wed, Apr 24, 2019 at 10:44 PM Vahid Hashemian < > vahid.hashem...@gmail.com> > wrote: > > > Hi all, > > >

Re: [VOTE] KIP-454: Expansion of the ConnectClusterState interface

Thanks a lot for the work on this KIP Chris. +1(non-binding) On Thu, May 2, 2019, 4:56 PM Chris Egerton wrote: > Hi all, > > I'd like to start a vote for KIP-454: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterState+interface > > The discussion t

Jenkins build is back to normal : kafka-trunk-jdk8 #3596

See

Build failed in Jenkins: kafka-trunk-jdk11 #481

See Changes: [wangguoz] KAFKA-8289: Fix Session Expiration and Suppression (#6654) [github] KAFKA-7601; Clear leader epoch cache on downgraded format in append -- Started

Jenkins build is back to normal : kafka-2.2-jdk8 #99

See

[jira] [Resolved] (KAFKA-7601) Handle message format downgrades during upgrade of message format version

[ https://issues.apache.org/jira/browse/KAFKA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7601. Resolution: Fixed Fix Version/s: 2.2.1 2.1.2 The patch fixes the

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Ivan, I’ll definitely forfeit my point on the clumsiness of the branch(predicate, consumer) solution, I don’t see any real drawbacks for the dynamic case. IMO the one trade off to consider at this point is the scope question. I don’t know if I totally agree that “we rarely need them in the sam

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

Paul, You might want to make a note on the KIP regarding the impact on quotas. Thanks, On Thu, May 2, 2019 at 9:48 AM Paul Davidson wrote: > Thanks for the votes everyone! KIP-411 is now accepted with: > > +3 binding votes (Randall, Jason, Gwen) , and > +3 non-binding votes (Ryanne, Arjun, Mag

Build failed in Jenkins: kafka-trunk-jdk11 #480

See Changes: [wangguoz] KAFKA-8285: enable localized thread IDs in Kafka Streams (#6632) -- Started by an SCM change [EnvInject] - Loading node environment variables. Buil

[jira] [Resolved] (KAFKA-8285) Handle thread-id random switch on JVM for KStream

[ https://issues.apache.org/jira/browse/KAFKA-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8285. -- Resolution: Fixed Fix Version/s: 2.3.0 > Handle thread-id random switch on JVM for KStre

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Hello everyone, thank you all for joining the discussion! Well, I don't think the idea of named branches, be it a LinkedHashMap (no other Map will do, because order of definition matters) or `branch` method  taking name and Consumer has more advantages than drawbacks. In my opinion, the only

[VOTE] KIP-454: Expansion of the ConnectClusterState interface

Hi all, I'd like to start a vote for KIP-454: https://cwiki.apache.org/confluence/display/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterState+interface The discussion thread can be found at https://www.mail-archive.com/dev@kafka.apache.org/msg96911.html Thanks to Konstantine Karantasis and Mag

RE: [DISCUSS] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

Good idea, Dan. One thing I might suggest is to have the query parameters reflect the fact that there are multiple resources under each connector. There is `connectors//`, `connectors//config`, and `connectors//status`. Each of them returns a slightly different set of information, so it would be us

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

I'd like to move this KIP along. To summarize where we've landed so far: - Should suppress() flush all records (would be _before_ the time elapsed)? - Or should it preserve buffered records and reload on restart? For this case, should the record be flushed on reload (elapsed time is unknown) or sh

Build failed in Jenkins: kafka-trunk-jdk11 #479

See Changes: [harsha] KAFKA-8191: Add pluggability of KeyManager to generate the broker -- Started by an SCM change [EnvInject] - Loading node environment variables. Build

Re: [VOTE] KIP-460: Admin Leader Election RPC

+1 (binding) thanks, Jose. best, Colin On Wed, May 1, 2019, at 14:44, Jose Armando Garcia Sancio wrote: > Hi all, > > I would like to start the voting for KIP-460: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-460%3A+Admin+Leader+Election+RPC > > The thread discussion is here: > http

Build failed in Jenkins: kafka-trunk-jdk8 #3595

See Changes: [github] KAFKA-8248; Ensure time updated before sending transactional request -- [...truncated 2.40 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Build failed in Jenkins: kafka-trunk-jdk11 #478

See Changes: [github] KAFKA-8248; Ensure time updated before sending transactional request -- Started by an SCM change [EnvInject] - Loading node environment variables. Bu

[jira] [Created] (KAFKA-8317) ClassCastException using KTable.suppress()

Andrew created KAFKA-8317: - Summary: ClassCastException using KTable.suppress() Key: KAFKA-8317 URL: https://issues.apache.org/jira/browse/KAFKA-8317 Project: Kafka Issue Type: Bug Report

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

Thanks for the votes everyone! KIP-411 is now accepted with: +3 binding votes (Randall, Jason, Gwen) , and +3 non-binding votes (Ryanne, Arjun, Magesh) Regards, Paul On Wed, May 1, 2019 at 10:07 PM Arjun Satish wrote: > Good point, Gwen. We always set a non empty value for client id: > > http

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

Thanks for the input Randall. I'm happy adding it natively to NewTopic instead of introducing more verbosity - updating the KIP to reflect this now. On Thu, May 2, 2019 at 9:28 AM Randall Hauch wrote: > I wrote the `NewTopicBuilder` in Connect, and it was simply a convenience > to more easily se

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

Hey Colin & George, Thinking on George's points I was wondering if it's feasible to submit a big reassignment to the controller and thus Zookeeper as frequent writes are slow as the quorum has to synchronize. Perhaps it should be the responsibility of KIP-435

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

I wrote the `NewTopicBuilder` in Connect, and it was simply a convenience to more easily set some of the frequently-used properties and the # of partitions and replicas for the new topic in the same way. An example is: NewTopic topicDescription = TopicAdmin.defineTopic(topic).

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

Sure thing, added more detail to the KIP! To clarify, the plan is to move an existing API from one package to another (NewTopicBuilder exists in the connect.runtime package) leaving the old in place for compatibility and deprecating it. I'm happy to hear thoughts on whether we should (a) move it t

[jira] [Resolved] (KAFKA-4304) Extend Interactive Queries for return latest update timestamp per key

[ https://issues.apache.org/jira/browse/KAFKA-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-4304. Resolution: Duplicate > Extend Interactive Queries for return latest update timestamp per k

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

If you are adding new API, you need to specify it all in the KIP. Ismael On Thu, May 2, 2019, 8:04 AM Almog Gavra wrote: > I think that sounds reasonable - I updated the KIP and I will remove the > constructor that takes in only partitions. > > On Thu, May 2, 2019 at 4:44 AM Andy Coates wrote:

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

I think that sounds reasonable - I updated the KIP and I will remove the constructor that takes in only partitions. On Thu, May 2, 2019 at 4:44 AM Andy Coates wrote: > Rather than adding overloaded constructors, which can lead to API bloat, > how about using a builder pattern? > > I see it’s alr

[jira] [Created] (KAFKA-8316) Remove deprecated usage of Slf4jRequestLog, SslContextFactory

Ismael Juma created KAFKA-8316: -- Summary: Remove deprecated usage of Slf4jRequestLog, SslContextFactory Key: KAFKA-8316 URL: https://issues.apache.org/jira/browse/KAFKA-8316 Project: Kafka Issu

Re: [DISCUSS] KIP-401 TransformerSupplier/ProcessorSupplier enhancements

Hi Paul, I will try to express myself a bit clearer. Ad 1) My assumption is that if `StateStoreConnector#stateStores()` returns `null` Kafka Streams will throw an NPE because on purpose no null check is performed before the loop that calls `StreamsBuilder#addStateStore()`. When the user finally u

[jira] [Created] (KAFKA-8315) Cannot pass Materialized into a join operation

Andrew created KAFKA-8315: - Summary: Cannot pass Materialized into a join operation Key: KAFKA-8315 URL: https://issues.apache.org/jira/browse/KAFKA-8315 Project: Kafka Issue Type: Bug Re

Re: [DISCUSS] KIP-464 Default Replication Factor for AdminClient#createTopic

Rather than adding overloaded constructors, which can lead to API bloat, how about using a builder pattern? I see it’s already got some constructor overloading, but we could add a single new constructor that takes just the name, and support everything else being set via builder methods. This w

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

I also agree with Michael's observation about the core problem of current `branch()` implementation. However, I also don't like to pass in a clumsy Map object. My thinking was more aligned with Paul's proposal to just add a name to each `branch()` statement and return a `Map`. It makes the code e

[jira] [Created] (KAFKA-8314) Managing the doc field in case of schema projection - kafka connect

kaushik srinivas created KAFKA-8314: --- Summary: Managing the doc field in case of schema projection - kafka connect Key: KAFKA-8314 URL: https://issues.apache.org/jira/browse/KAFKA-8314 Project: Kafk

Kafka Consumer - Random access fetching of partition range

Hi Kafka developers, *The question:* How can I randomly fetch an old chunk of messages with a given range definition of [partition, start offset, end offset]. Hopefully ranges from multiple partitions at once (one range for each partition). This needs to be supported in a concurrent environment to

Re: [VOTE] KIP-464: Defaults for AdminClient#createTopic

I was planning to write a KIP for the exact same feature! +1 (non binding) Thanks for the KIP On Wed, May 1, 2019 at 7:24 PM Almog Gavra wrote: > > Hello Everyone! > > Kicking off the voting for > https://cwiki.apache.org/confluence/display/KAFKA/KIP-464%3A+Defaults+for+AdminClient%23createTopic