Build failed in Jenkins: kafka-trunk-jdk8 #1835

See Changes: [me] MINOR: Make Kafka Connect step in quickstart more Windows friendly -- [...truncated 972.70 KB...] kafka.integration.TopicMetadataTest > testAutoCreateTo

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

Hi, The Vote for this KIP is now closed. KIP-173 has been accepted with 3 binding (Guozhang, Damian, Ismael) 2 non-binding (Eno, Gouzhang) Thanks, Damian On Thu, 20 Jul 2017 at 14:29 Ismael Juma wrote: > Yes, I know it is a separate thread in the archives, it's just that Gmail's > algorithm dec

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

Sorry, i mentioned Gouzhang twice in the vote. Actual votes: 3 binding (Guozhang, Damian, Ismael) 2 non-binding (Eno, Matthias) Thanks, Damian On Fri, 21 Jul 2017 at 10:00 Damian Guy wrote: > Hi, > The Vote for this KIP is now closed. > KIP-173 has been accepted with > 3 binding (Guozhang, Dam

[DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Aside: I've started this new DISCUSS thread for KIP-179 since the original one had the incorrect KIP number 178. The original thread can be found here: http://mail-archives.apache.org/mod_mbox/kafka-dev/201707.mbox/%3cCAMd5YszudP+-8z5KTbFh6JscT2p4xFi1=vzwwx+5dccpxry...@mail.gmail.com%3e I've just

Re: [DISCUSS] KIP-178: Change ReassignPartitionsCommand to use AdminClient

Due to the incorrect initial numbering of this KIP, discussion has moved to this thread: http://mail-archives.apache.org/mod_mbox/kafka-dev/201707.mbox/%3CCAMd5Ysy3bY7Fq2xA3sk6BWW6%3D9TjT4%2Bya7mufRf6Wgre-S-UPg%40mail.gmail.com%3E On 20 July 2017 at 16:47, Ismael Juma wrote: > Hi Tom, > > Yes, a

[jira] [Resolved] (KAFKA-3741) Allow setting of default topic configs via StreamsConfig

[ https://issues.apache.org/jira/browse/KAFKA-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-3741. --- Resolution: Fixed Fix Version/s: 0.11.1.0 Issue resolved by pull request 3459 [https://github.c

[GitHub] kafka pull request #3459: KAFKA-3741: allow users to specify default topic c...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3459 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Hi Jeyhun, Feel free to close the vote. It has been accepted. Thanks, Damian On Mon, 17 Jul 2017 at 06:18 Guozhang Wang wrote: > +1. Thanks! > > On Sat, Jul 8, 2017 at 1:35 AM, Damian Guy wrote: > > > +1 > > On Fri, 7 Jul 2017 at 16:08, Eno Thereska > wrote: > > > > > +1 (non-binding) Thanks

[GitHub] kafka pull request #3560: MINOR: Improve log warning to include the log name

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3560 MINOR: Improve log warning to include the log name You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka tweak-log-and-upgrade-note Alterna

[GitHub] kafka pull request #3561: MINOR: Give correct instructions for retaining pre...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3561 MINOR: Give correct instructions for retaining previous unclear leader election behaviour You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/k

[GitHub] kafka pull request #3560: MINOR: Improve log warning to include the log name

Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/3560 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] kafka pull request #3562: MINOR: Improve log warning to include the log name

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3562 MINOR: Improve log warning to include the log name You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka tweak-log-warning Alternatively yo

[GitHub] kafka pull request #3553: KAFKA-5608: Follow-up to fix potential NPE and cla...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3553 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Thanks Damian. This vote is now closed and KIP-149 is accepted with +3 binding +2 non-binding votes. Cheers, Jeyhun On Fri, Jul 21, 2017 at 3:52 PM Damian Guy wrote: > Hi Jeyhun, > > Feel free to close the vote. It has been accepted. > > Thanks, > Damian > > On Mon, 17 Jul 2017 at 06:18 Guozha

Re: Kafka doc : official repo or web site repo ?

Hi Ewen, thank you very much for your answer. I know that you guys are really busy and I pushed my PRs during the 0.11.0 release cycle so ... I can understand, no worries ;) Regarding the documentation happy to know that I pushed in the right repo but ... ... regarding the Kafka Streams new

Jenkins build is back to normal : kafka-trunk-jdk8 #1836

See

Re: Consumer throughput drop

After some tuning, I got better results. What I changed, as suggested: dirty_ratio = 10 (previously 20) dirty_background_ratio=3 (previously 10) It results that disk read I/O is almost completely 0 (I have enough cache, the consumer is keeping with the producer). - producer throughput remains

Re: Consumer throughput drop

Thanks for reporting the results. Maybe you could submit a PR that updates the ops section? https://github.com/apache/kafka/blob/trunk/docs/ops.html Ismael On Fri, Jul 21, 2017 at 2:49 PM, Ovidiu-Cristian MARCU < ovidiu-cristian.ma...@inria.fr> wrote: > After some tuning, I got better results.

Build failed in Jenkins: kafka-0.11.0-jdk7 #240

See Changes: [ismael] KAFKA-5608; Follow-up to fix potential NPE and clarify method name -- [...truncated 2.42 MB...] org.apache.kafka.streams.state.internals.ChangeLoggin

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

Thanks for the KIP, Colin. This will definitely be useful. One question: would it be useful to have a metric for for the number of groups in each possible state? The KIP suggests "PreparingRebalance" and "AwaitingSync". That leaves "Stable", "Dead" and "Empty". Are those not useful? Ismael On Thu

Re: [DISCUSS] 2017 October release planning and release version

+1 Why not just drop the leading 0, and call next version 12.0.0 instead of 1.0.0, I think in my head I’ve always just dropped the leading 0 anyhow. On 21/07/2017, 04:15, "Neha Narkhede" wrote: +1 on 1.0. It's about time :) On Thu, Jul 20, 2017 at 5:11 PM Ewen Cheslack-Postava wro

Re: [DISCUSS] 2017 October release planning and release version

+1 for 1.0 (not for 12.0) On 21/07/17 15:59, Michael Pearce wrote: +1 Why not just drop the leading 0, and call next version 12.0.0 instead of 1.0.0, I think in my head I’ve always just dropped the leading 0 anyhow. On 21/07/2017, 04:15, "Neha Narkhede" wrote: +1 on 1.0. It's about ti

[jira] [Resolved] (KAFKA-1421) Error: Could not find or load main class kafka.perf.SimpleConsumerPerformance

[ https://issues.apache.org/jira/browse/KAFKA-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryl Erwin resolved KAFKA-1421. Resolution: Done > Error: Could not find or load main class kafka.perf.SimpleConsumerPerformance > -

Re: [DISCUSS] 2017 October release planning and release version

On the topic of documentation, we should also document which releases are still supported and which are not. There a few factors to consider: 1. Which branches receive bug fixes. We typically backport fixes to the two most recent stable branches (the most recent stable branch typically gets more b

[GitHub] kafka pull request #3442: KAFKA-5512: Awake the heartbeat thread when timeto...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3442 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk7 #2552

See Changes: [jason] KAFKA-5512; Awake the heartbeat thread when timetoNextHeartbeat is equal -- [...truncated 973.96 KB...] kafka.log.BrokerCompressionTest > testBrokerSi

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

My two cents: "Dead" and "Empty" states are transient: groups usually only leaves in this state for a short while and then being deleted or transited to other states. Since we have the existing "*NumGroups*" metric, `*NumGroups - **NumGroupsRebalancing - **NumGroupsAwaitingSync`* should cover the

[jira] [Created] (KAFKA-5624) Unsafe use of expired sensors

Jason Gustafson created KAFKA-5624: -- Summary: Unsafe use of expired sensors Key: KAFKA-5624 URL: https://issues.apache.org/jira/browse/KAFKA-5624 Project: Kafka Issue Type: Bug R

Jenkins build is back to normal : kafka-0.11.0-jdk7 #241

See

Jenkins build is back to normal : kafka-trunk-jdk7 #2553

See

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

> > Regarding your comment about the current limitation on the information > returned for a consumer group, do you think it's worth expanding the API > to return some additional info (e.g. generation id, group leader, ...)? Seems outside the scope of this KIP. Up to you, but I'd probably leave it

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

> > "Dead" and "Empty" states are transient: groups usually only leaves in this > state for a short while and then being deleted or transited to other > states. This is not strictly true for the "Empty" state which we also use to represent simple groups which only use the coordinator to store off

Re: [DISCUSS] 2017 October release planning and release version

Thanks Ismael. I agree with you on all these points, and for some of these points like 3) we never have a written-down policy though in practice we tend to follow some patterns. To me deciding what's the version number of the next major release does not necessarily mean we need now to change any o

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

Ah, that's right Jason. With that I can be convinced to add one metric per each state. Guozhang On Fri, Jul 21, 2017 at 11:44 AM, Jason Gustafson wrote: > > > > "Dead" and "Empty" states are transient: groups usually only leaves in > this > > state for a short while and then being deleted or t

Re: [VOTE] KIP-167 (Addendum): Add interface for the state store restoration process

+1 On Thu, Jul 20, 2017 at 11:00 PM, Matthias J. Sax wrote: > +1 > > On 7/20/17 4:22 AM, Bill Bejeck wrote: > > Hi, > > > > After working on the PR for this KIP I discovered that we need to add and > > additional parameter (TopicPartition) to the StateRestoreListener > interface > > methods. > >

Re: [DISCUSS] 2017 October release planning and release version

Yes, I agree that the choice of version number can be done separately. That's why I said I'd file a separate JIRA for the documentation improvements. Having said that, there are some expectations that people have for projects that have reached 1.0.0 and we should try to allocate time for the import

Re: [DISCUSS] 2017 October release planning and release version

1.0! Let's do it! -Jay On Tue, Jul 18, 2017 at 3:36 PM, Guozhang Wang wrote: > Hi all, > > With 0.11.0.0 out of the way, I would like to volunteer to be the > release manager > for our next time-based feature release. See https://cwiki.apache.org/ > confluence/display/KAFKA/Time+Based+Release+P

Re: [DISCUSS] 2017 October release planning and release version

That's fair enough too. Guozhang On Fri, Jul 21, 2017 at 12:13 PM, Ismael Juma wrote: > Yes, I agree that the choice of version number can be done separately. > That's why I said I'd file a separate JIRA for the documentation > improvements. Having said that, there are some expectations that pe

[jira] [Created] (KAFKA-5625) Invalid subscription data may cause streams app to throw BufferUnderflowException

Xavier Léauté created KAFKA-5625: Summary: Invalid subscription data may cause streams app to throw BufferUnderflowException Key: KAFKA-5625 URL: https://issues.apache.org/jira/browse/KAFKA-5625 Proje

Re: [DISCUSS] KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress

That's a good point. I revised the KIP to add metrics for all the group states. best, Colin On Fri, Jul 21, 2017, at 12:08, Guozhang Wang wrote: > Ah, that's right Jason. > > With that I can be convinced to add one metric per each state. > > Guozhang > > On Fri, Jul 21, 2017 at 11:44 AM, Jas

Re: Kafka doc : official repo or web site repo ?

Paolo, Good question and good catch! I think those updates caused things to diverge because the author needed to make updates to both the regular docs and some of the styling assets that are only in the kafka-site repo. I've pinged the folks that worked on it to make sure the changes make it prope

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Hi Jason, Yes, I meant as a separate KIP. I can start a KIP for that sometime soon. Thanks. --Vahid From: Jason Gustafson To: dev@kafka.apache.org Cc: Kafka Users Date: 07/21/2017 11:37 AM Subject:Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCom

[GitHub] kafka pull request #3563: Added safe deserialization impl

GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/3563 Added safe deserialization impl You can merge this pull request into a Git repository by running: $ git pull https://github.com/rhauch/kafka deserialization-validation Alternatively you can re

Re: Kafka doc : official repo or web site repo ?

Ewen, Paolo: There is already a PR from Derrick who did the kafka-site PR for kafka repo as well: https://github.com/apache/kafka/pull/3540 We usually only first push to kafka, and then piggy-back to kafka-site upon releases; for some changes that we'd like to push in the middle of a release cycl

[jira] [Created] (KAFKA-5626) Producer should be able to negotiate ProduceRequest version with broker

Dong Lin created KAFKA-5626: --- Summary: Producer should be able to negotiate ProduceRequest version with broker Key: KAFKA-5626 URL: https://issues.apache.org/jira/browse/KAFKA-5626 Project: Kafka

[GitHub] kafka pull request #3564: MINOR: Don't imply that ConsumerGroupCommand won't...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3564 MINOR: Don't imply that ConsumerGroupCommand won't work with non-Java clients You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka mino

[GitHub] kafka pull request #3528: Kafka-4763 (Used for triggering test only)

Github user lindong28 closed the pull request at: https://github.com/apache/kafka/pull/3528 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] kafka pull request #3563: Added safe deserialization impl

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3563 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] 2017 October release planning and release version

+1! On Fri, Jul 21, 2017 at 12:35 PM, Jay Kreps wrote: > 1.0! Let's do it! > > -Jay > > On Tue, Jul 18, 2017 at 3:36 PM, Guozhang Wang wrote: > > > Hi all, > > > > With 0.11.0.0 out of the way, I would like to volunteer to be the > > release manager > > for our next time-based feature release.

Build failed in Jenkins: kafka-trunk-jdk7 #2554

See Changes: [me] MINOR: Added safe deserialization implementation -- [...truncated 976.96 KB...] kafka.server.epoch.LeaderEpochFileCacheTest > shouldDropEntriesOnEpochBo

[GitHub] kafka pull request #3383: KAFKA-5481: ListOffsetResponse isn't logged in the...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3383 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk7 #2555

See Changes: [me] KAFKA-5481: ListOffsetResponse isn't logged in the right way with trace -- [...truncated 969.49 KB...] kafka.server.epoch.LeaderEpochFileCacheTest > sh

Jenkins build is back to normal : kafka-0.10.1-jdk7 #126

See

Jenkins build is back to normal : kafka-0.10.2-jdk7 #187

See

[GitHub] kafka pull request #3540: Streams landing page

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3540 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk8 #1841

See Changes: [me] KAFKA-5481: ListOffsetResponse isn't logged in the right way with trace -- [...truncated 2.64 MB...] org.apache.kafka.connect.runtime.distributed.Distrib

[GitHub] kafka pull request #3564: MINOR: Don't imply that ConsumerGroupCommand won't...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3564 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Jenkins build is back to normal : kafka-trunk-jdk7 #2556

See

[GitHub] kafka-site pull request #68: Docs TOC adjustments

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/68 Docs TOC adjustments Docs TOC updates for Streams You can merge this pull request into a Git repository by running: $ git pull https://github.com/derrickdoo/kafka-site docs-toc-adjustment

Jenkins build is back to normal : kafka-trunk-jdk8 #1842

See

Re: Using JMXMP to access Kafka metrics

We use jmxtrans and works pretty good. [image: Turn] *Fernando Vega* Sr. Operations Engineer *cell* (415) 810-0242 901 Marshall St, Suite 200, Redwood City, CA 94063 turn.com | @TurnPlatform This message is T

[GitHub] kafka-site pull request #68: Docs TOC adjustments

Github user asfgit closed the pull request at: https://github.com/apache/kafka-site/pull/68 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] kafka-site issue #68: Docs TOC adjustments

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/68 LGTM. Merged to asf-site. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an