[jira] [Created] (KAFKA-5630) Consumer poll loop over the same record after a CorruptRecordException

2017-07-24 Thread Vincent Maurin (JIRA)
Vincent Maurin created KAFKA-5630: - Summary: Consumer poll loop over the same record after a CorruptRecordException Key: KAFKA-5630 URL: https://issues.apache.org/jira/browse/KAFKA-5630 Project: Kafka

[jira] [Created] (KAFKA-5631) Use Jackson for serialising to JSON

2017-07-24 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-5631: -- Summary: Use Jackson for serialising to JSON Key: KAFKA-5631 URL: https://issues.apache.org/jira/browse/KAFKA-5631 Project: Kafka Issue Type: Improvement

[GitHub] kafka pull request #3566: KAFKA-5629: Added a warn message to the output of ...

2017-07-24 Thread soenkeliebau
GitHub user soenkeliebau opened a pull request: https://github.com/apache/kafka/pull/3566 KAFKA-5629: Added a warn message to the output of ConsoleConsumer when "auto.offset.reset" property is specified on the command line but overridden by the code during startup. Currently the Co

[jira] [Created] (KAFKA-5632) Message headers not supported by Kafka Streams

2017-07-24 Thread CJ Woolard (JIRA)
CJ Woolard created KAFKA-5632: - Summary: Message headers not supported by Kafka Streams Key: KAFKA-5632 URL: https://issues.apache.org/jira/browse/KAFKA-5632 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-07-24 Thread Ismael Juma
Hi Tom, I think it makes sense to keep progress reporting simple in the initial version. As you say, time to completion is tricky to compute and it seems like it should be tackled as its own KIP. Regarding waiting for reassignment completion, it's easy enough for people to do that via a script, s

Re: Consumer offsets partitions size much bigger than others

2017-07-24 Thread Luciano Afranllie
Thanks James The issue was that the log cleaner was disabled in that cluster due to an old configuration that we have since 0.9.0.0. Regards Luciano On Tue, Jul 18, 2017 at 7:04 PM, James Cheng wrote: > It's possible that the log-cleaning thread has crashed. That is the thread > that implement

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

2017-07-24 Thread Ewen Cheslack-Postava
Vahid, Thanks for the KIP. I think we're mostly in violent agreement that the lack of any Write permissions on consumer groups is confusing. Unfortunately it's a pretty annoying issue to fix since it would require an increase in permissions. More generally, I think it's unfortunate because by sque

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

2017-07-24 Thread Colin McCabe
Hi all, I think maybe it makes sense to rename the "PreparingRebalance" consumer group state to "Rebalancing." To me, "Preparing" implies that there will be a later "rebalance" state that follows-- but there is not. Since we're now exposing this state name publicly in these metrics, perhaps it m

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

2017-07-24 Thread Ismael Juma
Sounds good to me. Ismael On Mon, Jul 24, 2017 at 6:46 PM, Colin McCabe wrote: > Hi all, > > I think maybe it makes sense to rename the "PreparingRebalance" consumer > group state to "Rebalancing." To me, "Preparing" implies that there > will be a later "rebalance" state that follows-- but the

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

2017-07-24 Thread Guozhang Wang
Actually Rebalancing includes two steps, and we name them PrepareRebalance and WaitSync (arguably they may not be the best names). But these two steps together should be treated as the complete rebalance cycle. Guozhang On Mon, Jul 24, 2017 at 10:46 AM, Colin McCabe wrote: > Hi all, > > I thin

[GitHub] kafka pull request #3536: KAFKA-3856: Refactoring for KIP-120

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3536 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Reopened] (KAFKA-3856) Cleanup Kafka Streams builder API

2017-07-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-3856: -- Leave the JIRA as open since I think there is another PR coming for it. BTW moving forward it's bet

Re: [DISCUSS] 2017 October release planning and release version

2017-07-24 Thread Becket Qin
+1 on 1.0. I think it makes a lot of sense given the point Kafka is now. To me Kafka has absolutely reached 1.0 in terms of features/functions. That said, I do share the same opinion with others that the usablity and policies might still need some improvement to meet the 1.0 standard. On Sat, Jul

[GitHub] kafka pull request #3567: KAFKA-4711: fix docs onunclean.leader.election.ena...

2017-07-24 Thread bobrik
GitHub user bobrik opened a pull request: https://github.com/apache/kafka/pull/3567 KAFKA-4711: fix docs onunclean.leader.election.enabled default in design section You can merge this pull request into a Git repository by running: $ git pull https://github.com/bobrik/kafka un

[jira] [Created] (KAFKA-5633) Clarify another scenario of unclean leader election

2017-07-24 Thread Ivan Babrou (JIRA)
Ivan Babrou created KAFKA-5633: -- Summary: Clarify another scenario of unclean leader election Key: KAFKA-5633 URL: https://issues.apache.org/jira/browse/KAFKA-5633 Project: Kafka Issue Type: Bug

Re: [DISCUSS] 2017 October release planning and release version

2017-07-24 Thread Guozhang Wang
Thanks for everyone who has shared their thoughts and feedbacks to this discussion. Here is my conclusion: 1. So far everyone has agreed on naming the next major release version to 1.0; if I did not hear any other opinions by the end of today PDT time I will name the next release as 1.0.0 and upda

Re: [DISCUSS] 2017 October release planning and release version

2017-07-24 Thread Sönke Liebau
I am happy to volunteer for working on documenting the release logistics and related topics if someone is needed for that part. Best regards, Sönke Am 24.07.2017 9:28 nachm. schrieb "Guozhang Wang" : > Thanks for everyone who has shared their thoughts and feedbacks to this > discussion. Here is

[GitHub] kafka pull request #2950: enhance JmxTool usage and output format

2017-07-24 Thread scottf
Github user scottf closed the pull request at: https://github.com/apache/kafka/pull/2950 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2950: enhance JmxTool usage and output format

2017-07-24 Thread scottf
GitHub user scottf reopened a pull request: https://github.com/apache/kafka/pull/2950 enhance JmxTool usage and output format one-time flag to only run the pull once instead of looping forever reporting-interval -1 equivalent to setting one-time reportFormatOpt supports 'orig

Kafka Connect suggestion before creating new KIP

2017-07-24 Thread Florian Hussonnois
Hi all, Here is two suggestions for the Kafka Connect API that I prefer to present you first before creating KIPs. 1/ Currently, in Kafka Connect we distinguish two configuration types: the worgker config and the connector config. The connector config is accessible from the Connector class throug

[GitHub] kafka pull request #3568: updated configs to use one try/catch for serdes

2017-07-24 Thread bbejeck
GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3568 updated configs to use one try/catch for serdes removed `try/catch` from `keySerde` and `valueSerde` methods so only the `try\catch` blocks in `defaultKeySerde` and `defaultValueSerde` perform error

[GitHub] kafka pull request #3569: MINOR: enforce setting listeners in CREATE state.

2017-07-24 Thread bbejeck
GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3569 MINOR: enforce setting listeners in CREATE state. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbejeck/kafka MINOR_enforce_adding_listeners_

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

2017-07-24 Thread Vahid S Hashemian
Hi Ewen, Thanks for reviewing the KIP. Your comment about the "food for thought" section makes sense. It seems like a bug to me, not sure how you and others feel about it. I'll remove it for now, and open a separate JIRA for it, so we have a record of it. The read vs. write discussion and fixin

[jira] [Resolved] (KAFKA-5616) unable perform a rolling upgrade from a non-secure to a secure Kafka cluster

2017-07-24 Thread zhu fangbo (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu fangbo resolved KAFKA-5616. --- Resolution: Not A Problem This is due to improper ACL configuration,I did not set the operation for ea

[jira] [Created] (KAFKA-5634) Replica fetcher thread crashes due to OffsetOutOfRangeException

2017-07-24 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-5634: -- Summary: Replica fetcher thread crashes due to OffsetOutOfRangeException Key: KAFKA-5634 URL: https://issues.apache.org/jira/browse/KAFKA-5634 Project: Kafka

Re: [DISCUSS] 2017 October release planning and release version

2017-07-24 Thread Guozhang Wang
Hello, Since I have not heard any further comments till now. I will close this discussion and formally name our next major release (this coming Oct.) 1.0.0; will update the wiki pages accordingly. Sönke, Thanks for volunteering in documenting the release logistics. I'll leave it to Ismael to to

[GitHub] kafka pull request #3570: KAFKA-4218: KIP-149, Enabling withKey interfaces i...

2017-07-24 Thread jeyhunkarimov
GitHub user jeyhunkarimov opened a pull request: https://github.com/apache/kafka/pull/3570 KAFKA-4218: KIP-149, Enabling withKey interfaces in streams This PR aims to provide key access to ValueMapper, ValueJoiner, ValueTransformer, Initializer and Reducer interfaces. More details c

[GitHub] kafka pull request #2946: KAFKA-4218: Enable access to key in ValueTransform...

2017-07-24 Thread jeyhunkarimov
Github user jeyhunkarimov closed the pull request at: https://github.com/apache/kafka/pull/2946 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

1.0.0 First Update

2017-07-24 Thread Guozhang Wang
Hello all, I'm still working on the tentative release dates for the next 1.0.0 major release, but just want to give you a heads up now: 1. I have updated the release versions in KIP wiki and the release plan. Details of the coming major release can be found in this page: https://cwiki.apache.org

[GitHub] kafka pull request #3571: KAFKA-5611; AbstractCoordinator should handle wake...

2017-07-24 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3571 KAFKA-5611; AbstractCoordinator should handle wakeup raised from onJoinComplete You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kaf