[jira] [Resolved] (KAFKA-6716) discardChannel should be released in MockSelector#completeSend

[ https://issues.apache.org/jira/browse/KAFKA-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6716. -- Resolution: Fixed Fix Version/s: 1.2.0 > discardChannel should be released in MockSelector#comple

[jira] [Resolved] (KAFKA-6630) Speed up the processing of TopicDeletionStopReplicaResponseReceived events on the controller

[ https://issues.apache.org/jira/browse/KAFKA-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6630. -- Resolution: Fixed Fix Version/s: 1.2.0 > Speed up the processing of TopicDeletionStopReplicaRespo

[jira] [Resolved] (KAFKA-6702) Wrong className in LoggerFactory.getLogger method

[ https://issues.apache.org/jira/browse/KAFKA-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6702. -- Resolution: Fixed > Wrong className in LoggerFactory.getLogger method >

[jira] [Resolved] (KAFKA-6707) The default value for config of Type.LONG should be *L

[ https://issues.apache.org/jira/browse/KAFKA-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6707. -- Resolution: Fixed > The default value for config of Type.LONG should be *L > ---

[jira] [Created] (KAFKA-6740) Plugins class' newConverter and newHeaderConverter methods are unclear

Randall Hauch created KAFKA-6740: Summary: Plugins class' newConverter and newHeaderConverter methods are unclear Key: KAFKA-6740 URL: https://issues.apache.org/jira/browse/KAFKA-6740 Project: Kafka

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Thanks for sharing your thoughts. As I mentioned originally, I am not sure about the right log level either. Your arguments are convincing -- thus, I am fine with keeping WARN level. The task vs thread level argument is an interesting one. However, I am wondering if we should add this information

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

Hey friends, both KIP and pull request are updated. Feel free to take another look. Thanks for your valuable feedback! Best, Boyang

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

Thanks Matthias, Ted and Guozhang for the inputs. I shall address them in next round. From: Matthias J. Sax Sent: Tuesday, April 3, 2018 4:43 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers Yes, your

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Regarding logging: I'm inclined to keep logging at WARN level since skipped records are not expected in normal execution (for all reasons that we are aware of), and hence when error happens users should be alerted from metrics and looked into the log files, so to me if it is really spamming the log

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Hi Matthias, No worries! Thanks for the reply. 1) There isn't a connection. I tried using the TopologyTestDriver to write a quick test exercising the current behavior and discovered that the metrics weren't available. It seemed like they should be, so I tacked it on to this KIP. If you feel it's

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Hey guys, just sharing my two cents here (I promise it will be shorter than John's article :). 0. Just to quickly recap, the main discussion point now is how to support "key partitioning preservation" (John's #4 in topic characteristics above) beyond the "single-key ordering preservation" that KIP

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Hi, John, Thanks for the reply. It summarizes the tradeoffs well. At the high level, there are two general approaches for KStreams. (a) KStreams completely decouples from the input partitions by over-partitioning. Inside KStreams, partition expansion is never needed. (b) Over-partitioning is not e

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

John, sorry for my late reply and thanks for updating the KIP. I like your approach about "metrics are for monitoring, logs are for debugging" -- however: 1) I don't see a connection between this and the task-level metrics that you propose to get the metrics in `TopologyTestDriver`. I don't thin

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

Yes, your examples make sense to me. That was the idea behind the proposal. -Matthias On 4/2/18 11:25 AM, Guozhang Wang wrote: > @Matthias > > That's a good question: I think adding another config for the main consumer > makes good tradeoffs between compatibility and new user convenience. Just

Build failed in Jenkins: kafka-trunk-jdk8 #2520

See Changes: [github] MINOR: Fix partition loading checks in GroupCoordinator (#4788) [jason] KAFKA-5674; Reduce max.connections.per.ip minimum to 0 (#3610) -- [...trunca

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

+1 On Mon, Apr 2, 2018 at 8:42 AM, Guozhang Wang wrote: > +1 (binding). > > On Mon, Apr 2, 2018 at 7:22 AM, Ted Yu wrote: > > > +1 > > > > On Mon, Apr 2, 2018 at 7:11 AM, Bill Bejeck wrote: > > > > > Thanks for the KIP. > > > > > > +1 > > > > > > -Bill > > > > > > On Mon, Apr 2, 2018 at 10:09

[jira] [Created] (KAFKA-6739) Broker receives error when handling request with java.lang.IllegalArgumentException: Magic v0 does not support record headers

Koelli Mungee created KAFKA-6739: Summary: Broker receives error when handling request with java.lang.IllegalArgumentException: Magic v0 does not support record headers Key: KAFKA-6739 URL: https://issues.apache.o

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

@Matthias That's a good question: I think adding another config for the main consumer makes good tradeoffs between compatibility and new user convenience. Just to clarify, from user's pov on upgrade: 1) I'm already overriding some consumer configs, and now I want to override these values differen

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

bq. to introduce one more prefix `main.consumer.` Makes sense. On Mon, Apr 2, 2018 at 11:06 AM, Matthias J. Sax wrote: > Boyang, > > thanks a lot for the KIP. > > Couple of questions: > > > (MODIFIED) public Map getRestoreConsumerConfigs(final > String clientId); > > You mean that the implement

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

Boyang, thanks a lot for the KIP. Couple of questions: > (MODIFIED) public Map getRestoreConsumerConfigs(final String > clientId); You mean that the implementation/semantics of this method changes, but not the API itself? Might be good to add this as "comment style" instead of "(MODIFIED)" pre

Re: Seeking Feedback on Kafka Connect Issues

Howdy, I'll keep an eye on that ticket. KIP-275 came out of some work we've done internally on our private forks of Kafka and the Confluent Cloud Storage connector. Essentially, with that extra API we've tweaked the S3 connector to check the value of isClosing in preCommit and immediately attempt

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

Thanks everybody for voting. With 3 binding and 4 non-binding votes, I will mark the KIP as accepted. Thanks, Allen On Fri, Mar 30, 2018 at 7:11 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > The incompatibility is unfortunate, but seems unlikely to cause a problem > in practice. Let's ju

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

+1 (binding). On Mon, Apr 2, 2018 at 7:22 AM, Ted Yu wrote: > +1 > > On Mon, Apr 2, 2018 at 7:11 AM, Bill Bejeck wrote: > > > Thanks for the KIP. > > > > +1 > > > > -Bill > > > > On Mon, Apr 2, 2018 at 10:09 AM, John Roesler wrote: > > > > > Dear Kafka community, > > > > > > I am proposing KIP

Re: Seeking Feedback on Kafka Connect Issues

Yes, Confluent would be interested in improvements to the S3 connector. Feel free to create an issue/PR in https://github.com/confluentinc/kafka-connect-storage-cloud/. I just created https://issues.apache.org/jira/browse/KAFKA-6738 to deal with the bad data handling issue, and we can use that to

[jira] [Created] (KAFKA-6738) Kafka Connect handling of bad data

Randall Hauch created KAFKA-6738: Summary: Kafka Connect handling of bad data Key: KAFKA-6738 URL: https://issues.apache.org/jira/browse/KAFKA-6738 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-275 - Indicate "isClosing" in the SinkTaskContext

Ugh * I can update I need more caffeine... On Mon, Apr 2, 2018 at 11:01 AM, Matt Farmer wrote: > I'm can update the rejected alternatives section as you describe. > > Also, adding a paragraph to the preCommit javadoc also seems like a > Very Very Good Idea™ so I'll make that update to the KIP

Re: [DISCUSS] KIP-275 - Indicate "isClosing" in the SinkTaskContext

I'm can update the rejected alternatives section as you describe. Also, adding a paragraph to the preCommit javadoc also seems like a Very Very Good Idea™ so I'll make that update to the KIP as well. On Mon, Apr 2, 2018 at 10:48 AM, Randall Hauch wrote: > Thanks for the KIP proposal, Matt. > >

Re: [DISCUSS] KIP-275 - Indicate "isClosing" in the SinkTaskContext

Thanks for the KIP proposal, Matt. You mention in the "Rejected Alternatives" section that you considered changing the signature of the `preCommit` method but rejected it because it would break backward compatibility. Strictly speaking, it is possible to do this without breaking compatibility by i

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

+1 On Mon, Apr 2, 2018 at 7:11 AM, Bill Bejeck wrote: > Thanks for the KIP. > > +1 > > -Bill > > On Mon, Apr 2, 2018 at 10:09 AM, John Roesler wrote: > > > Dear Kafka community, > > > > I am proposing KIP-274 to improve visibility when Streams skips invalid > > records. > > > > The proposal is

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

Thanks for the KIP. +1 -Bill On Mon, Apr 2, 2018 at 10:09 AM, John Roesler wrote: > Dear Kafka community, > > I am proposing KIP-274 to improve visibility when Streams skips invalid > records. > > The proposal is to simplify the metrics and add warning logs. > > Please find details in the wiki

[VOTE] KIP-274: Kafka Streams Skipped Records Metrics

Dear Kafka community, I am proposing KIP-274 to improve visibility when Streams skips invalid records. The proposal is to simplify the metrics and add warning logs. Please find details in the wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-274%3A+Kafka+Streams+Skipped+Records+Metrics

[jira] [Resolved] (KAFKA-6645) Host Affinity to facilitate faster restarts of kafka streams applications

[ https://issues.apache.org/jira/browse/KAFKA-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridhar Addepalli resolved KAFKA-6645. --- Resolution: Information Provided > Host Affinity to facilitate faster restarts of kafk

Jenkins build is back to normal : kafka-trunk-jdk8 #2519

See