Re: Brokers are crash due to __consumer_offsets folder are deleted

Dear 黄杰斌: I am guessing your operating system is configured to delete your /tmp directory when you restart the server. You will need to change the "log.dir" property in your broker's server.properties file to someplace permanent. Unfortunately, your data is lost unless you had a backup or had

Build failed in Jenkins: kafka-trunk-jdk8 #731

See Changes: [ismael] MINOR: Typo fix in docs ops [ismael] MINOR: Warning instead of error if TGT cannot be renewed beyond the next -- [...truncated 1195 lines...] kafka.server.KafkaConfigTest

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359943#comment-15359943 ] ASF GitHub Bot commented on KAFKA-3854: --- GitHub user vahidhashemian reopened a pull

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359942#comment-15359942 ] ASF GitHub Bot commented on KAFKA-3854: --- Github user vahidhashemian closed the pull

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/1572 KAFKA-3854: Fix issues with new consumer's subsequent regex (pattern) subscriptions This patch fixes two issues: 1. Subsequent regex subscriptions fail with the new consumer. 2. Sub

Build failed in Jenkins: kafka-trunk-jdk8 #730

See Changes: [ismael] MINOR: remove "auto.commit.interval.ms" from "Manual Offset Control" [ismael] MINOR: Fix consumer constructor doc string -- [...truncated 11396 lines...] org.apache.kafka.s

Build failed in Jenkins: kafka-trunk-jdk7 #1400

See Changes: [ismael] MINOR: remove "auto.commit.interval.ms" from "Manual Offset Control" [ismael] MINOR: Fix consumer constructor doc string [ismael] MINOR: Typo fix in docs ops [ismael] MINOR: Warning instead of error if TGT cann

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359911#comment-15359911 ] ASF GitHub Bot commented on KAFKA-3854: --- GitHub user vahidhashemian reopened a pull

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359909#comment-15359909 ] ASF GitHub Bot commented on KAFKA-3854: --- Github user vahidhashemian closed the pull

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/1572 KAFKA-3854: Fix issues with new consumer's subsequent regex (pattern) subscriptions This patch fixes two issues: 1. Subsequent regex subscriptions fail with the new consumer. 2. Sub

[GitHub] kafka pull request #1510: Minor: warn if the TGT cannot be renewed.

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1510 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk7 #1399

See Changes: [wangguoz] KAFKA-3902: Optimize KTable.filter in Streams DSL to avoid forwarding if -- [...truncated 3330 lines...] kafka.producer.AsyncProducerTest > testJavaProducer PASSED kaf

[GitHub] kafka pull request #1568: MINOR: Fix consumer constructor doc string

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1568 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #1578: MINOR: remove "auto.commit.interval.ms" from "Manu...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1578 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk8 #729

See Changes: [wangguoz] KAFKA-3902: Optimize KTable.filter in Streams DSL to avoid forwarding if -- [...truncated 1113 lines...] kafka.server.KafkaConfigTest > testDuplicateListeners PASSED kaf

[jira] [Commented] (KAFKA-3902) Optimize KTable.filter() to reduce unnecessary traffic

[ https://issues.apache.org/jira/browse/KAFKA-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359843#comment-15359843 ] Guozhang Wang commented on KAFKA-3902: -- [~phderome] Thanks for your patch! It has bee

[jira] [Resolved] (KAFKA-3902) Optimize KTable.filter() to reduce unnecessary traffic

[ https://issues.apache.org/jira/browse/KAFKA-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3902. -- Resolution: Fixed Fix Version/s: 0.10.0.1 Issue resolved by pull request 1556 [https://gi

[jira] [Commented] (KAFKA-3902) Optimize KTable.filter() to reduce unnecessary traffic

[ https://issues.apache.org/jira/browse/KAFKA-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359841#comment-15359841 ] ASF GitHub Bot commented on KAFKA-3902: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1556: KAFKA-3902

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1556 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-3902) Optimize KTable.filter() to reduce unnecessary traffic

[ https://issues.apache.org/jira/browse/KAFKA-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3902: - Assignee: Phil Derome > Optimize KTable.filter() to reduce unnecessary traffic > -

[jira] [Commented] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359835#comment-15359835 ] Guozhang Wang commented on KAFKA-3705: -- Yes, this does finally clarify your scenario,

[jira] [Created] (KAFKA-3924) Data loss due to halting when LEO is larger than leader's LEO

Maysam Yabandeh created KAFKA-3924: -- Summary: Data loss due to halting when LEO is larger than leader's LEO Key: KAFKA-3924 URL: https://issues.apache.org/jira/browse/KAFKA-3924 Project: Kafka

Re: Kafka Connect Transformers

Added wiki access. Enjoy :) On Fri, Jul 1, 2016 at 11:24 AM, Nisarg Shah wrote: > Need to submit a KIP for https://issues.apache.org/jira/browse/KAFKA-3209. > Please provide wiki write access to ‘snisarg’. > > Thanks, > Nisarg Shah. > >> On Jun 28, 2016, at 6:27 PM, Nisarg Shah wrote: >> >> Nee

Build failed in Jenkins: kafka-trunk-jdk7 #1398

See Changes: [me] MINOR: fix Bash shebang on vagrant/ scripts -- [...truncated 3368 lines...] kafka.producer.AsyncProducerTest > testJavaProducer PASSED kafka.producer.AsyncProducerTest > test

[jira] [Commented] (KAFKA-3922) Add a copy-constructor to AbstractStream

[ https://issues.apache.org/jira/browse/KAFKA-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359577#comment-15359577 ] Guozhang Wang commented on KAFKA-3922: -- [~fhussonnois] This sounds a good idea. I hav

[jira] [Updated] (KAFKA-3922) Add a copy-constructor to AbstractStream

[ https://issues.apache.org/jira/browse/KAFKA-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3922: - Assignee: Florian Hussonnois (was: Guozhang Wang) > Add a copy-constructor to AbstractStream > --

[jira] [Updated] (KAFKA-3922) Add a copy-constructor to AbstractStream

[ https://issues.apache.org/jira/browse/KAFKA-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3922: - Labels: api (was: ) > Add a copy-constructor to AbstractStream >

Build failed in Jenkins: kafka-trunk-jdk8 #728

See Changes: [me] MINOR: fix Bash shebang on vagrant/ scripts -- [...truncated 3399 lines...] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.u

[jira] [Work stopped] (KAFKA-3429) Remove Serdes needed for repartitioning in KTable stateful operations

[ https://issues.apache.org/jira/browse/KAFKA-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3429 stopped by Matthias J. Sax. -- > Remove Serdes needed for repartitioning in KTable stateful operations > ---

[GitHub] kafka pull request #1580: Hotfix: auto-repartitioing for merge and code simp...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/1580 Hotfix: auto-repartitioing for merge and code simplication follow-up to auto-through feature: - add sourceNode to transform() - enable auto-repartitioning in merge() - null check not req

[GitHub] kafka pull request #1567: MINOR: fix Bash shebang on vagrant/ scripts

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1567 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: Kafka Connect Transformers

Need to submit a KIP for https://issues.apache.org/jira/browse/KAFKA-3209. Please provide wiki write access to ‘snisarg’. Thanks, Nisarg Shah. > On Jun 28, 2016, at 6:27 PM, Nisarg Shah wrote: > > Need permissions to edit the wiki. Username is ‘snisarg’. > > Thanks, > Nisarg. > >> On Jun 2

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359383#comment-15359383 ] ASF GitHub Bot commented on KAFKA-3854: --- Github user vahidhashemian closed the pull

[jira] [Commented] (KAFKA-3854) Subsequent regex subscription calls fail

[ https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359384#comment-15359384 ] ASF GitHub Bot commented on KAFKA-3854: --- GitHub user vahidhashemian reopened a pull

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #1572: KAFKA-3854: Fix issues with new consumer's subsequ...

GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/1572 KAFKA-3854: Fix issues with new consumer's subsequent regex (pattern) subscriptions This patch fixes two issues: 1. Subsequent regex subscriptions fail with the new consumer. 2. Sub

[jira] [Commented] (KAFKA-3877) Gradle compiler daemon exits with non-zero exit code while running LogOffsetTest

[ https://issues.apache.org/jira/browse/KAFKA-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359346#comment-15359346 ] Vahid Hashemian commented on KAFKA-3877: I've seen a few recent builds fail in {{

[jira] [Commented] (KAFKA-3877) Gradle compiler daemon exits with non-zero exit code while running LogOffsetTest

[ https://issues.apache.org/jira/browse/KAFKA-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359335#comment-15359335 ] Guozhang Wang commented on KAFKA-3877: -- Just saw another one of this test case failur

[jira] [Resolved] (KAFKA-3753) Add approximateNumEntries() to the StateStore interface for metrics reporting

[ https://issues.apache.org/jira/browse/KAFKA-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3753. -- Resolution: Fixed This JIRA has been merged in as PR 1486. > Add approximateNumEntries() to the

[jira] [Resolved] (KAFKA-3538) Abstract the creation/retrieval of Producer for stream sinks for unit testing

[ https://issues.apache.org/jira/browse/KAFKA-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3538. -- Resolution: Duplicate Fix Version/s: (was: 0.10.1.0) 0.10.0.1 > Ab

[jira] [Commented] (KAFKA-3538) Abstract the creation/retrieval of Producer for stream sinks for unit testing

[ https://issues.apache.org/jira/browse/KAFKA-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359264#comment-15359264 ] Guozhang Wang commented on KAFKA-3538: -- [~mdcoon] I think this has been fixed as in

[jira] [Commented] (KAFKA-3429) Remove Serdes needed for repartitioning in KTable stateful operations

[ https://issues.apache.org/jira/browse/KAFKA-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359231#comment-15359231 ] Guozhang Wang commented on KAFKA-3429: -- I think this ticket should be re-defined a bi

[jira] [Commented] (KAFKA-3923) MetricReporter interface depends on final class KafkaMetric instead of Metric interface

[ https://issues.apache.org/jira/browse/KAFKA-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359202#comment-15359202 ] ASF GitHub Bot commented on KAFKA-3923: --- GitHub user stepio opened a pull request:

[GitHub] kafka pull request #1579: KAFKA-3923: Use Metric interface instead of final ...

GitHub user stepio opened a pull request: https://github.com/apache/kafka/pull/1579 KAFKA-3923: Use Metric interface instead of final KafkaMetric in MetricsReporter and its implementations This is a patch for the previously highlighted issue: https://issues.apache.org/jira/brows

Re: [VOTE] KIP-60: Make Java client class loading more flexible

+1 (non binding) -- Edoardo Comar MQ Cloud Technologies eco...@uk.ibm.com +44 (0)1962 81 5576 IBM UK Ltd, Hursley Park, SO21 2JN IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbo

Re: [VOTE] KIP-60: Make Java client class loading more flexible

+1 -Jay On Fri, Jul 1, 2016 at 4:27 AM, Rajini Sivaram wrote: > I would like to initiate the voting process for KIP-60 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-60+-+Make+Java+client+classloading+more+flexible > ). > This is a simple set of changes that are fully compatible wi

Re: [DISCUSS] KIP-67: Queryable state for Kafka Stream

We have not used the "get" prefex in methods, like getXyz(), elsewhere in our java code, instead sticking with the scala style methods like xyz(). It'd be good to change those. -Jay On Fri, Jul 1, 2016 at 4:09 AM, Damian Guy wrote: > Hi, > > We've made some modifications to the KIP. The "Discov

Brokers are crash due to __consumer_offsets folder are deleted

Hi All, Do you encounter below issue when using kafka_2.11-0.10.0.0? All brokers are crash due to __consumer_offsets folder are deleted. sample log: [2016-06-30 12:46:32,579] FATAL [Replica Manager on Broker 2]: Halting due to unrecoverable I/O error while handling produce request: (kafka.server

[VOTE] KIP-60: Make Java client class loading more flexible

I would like to initiate the voting process for KIP-60 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-60+-+Make+Java+client+classloading+more+flexible). This is a simple set of changes that are fully compatible with the existing class loading in Kafka and enables Java clients to be run in

Re: [DISCUSS] Client Side Auto Topic Creation

Hi all, I think there are a few things being discussed and it would be good to make that explicit: 1. If and how we expose auto-topic creation in the client (under the assumption that the server auto-topic creation will be deprecated and eventually removed) 2. The ability to create topics with th

Re: [DISCUSS] KIP-67: Queryable state for Kafka Stream

Hi, We've made some modifications to the KIP. The "Discovery" API has been changed https://cwiki.apache.org/confluence/display/KAFKA/KIP-67%3A+Queryable+state+for+Kafka+Streams#KIP-67:QueryablestateforKafkaStreams-Step2inproposal:globaldiscoveryofstatestores Please take a look. Many thanks, Dami

[jira] [Updated] (KAFKA-3923) MetricReporter interface depends on final class KafkaMetric instead of Metric interface

[ https://issues.apache.org/jira/browse/KAFKA-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Stepanov updated KAFKA-3923: - Description: Hello, I'm working on exposing Kafka's consumer/producer metrics to Spring Actuator.

[jira] [Commented] (KAFKA-3923) MetricReporter interface depends on final class KafkaMetric instead of Metric interface

[ https://issues.apache.org/jira/browse/KAFKA-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358761#comment-15358761 ] Igor Stepanov commented on KAFKA-3923: -- If you consider this worth to be improved, le

[jira] [Created] (KAFKA-3923) MetricReporter interface depends on final class KafkaMetric instead of Metric interface

Igor Stepanov created KAFKA-3923: Summary: MetricReporter interface depends on final class KafkaMetric instead of Metric interface Key: KAFKA-3923 URL: https://issues.apache.org/jira/browse/KAFKA-3923

[jira] [Assigned] (KAFKA-1464) Add a throttling option to the Kafka replication tool

[ https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford reassigned KAFKA-1464: --- Assignee: Ben Stopford (was: Ismael Juma) > Add a throttling option to the Kafka replication

[jira] [Updated] (KAFKA-3919) Broker faills to start after ungraceful shutdown due to non-monotonically incrementing offsets in logs

[ https://issues.apache.org/jira/browse/KAFKA-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Coates updated KAFKA-3919: --- Description: Hi All, I encountered an issue with Kafka following a power outage that saw a proportio

[jira] [Issue Comment Deleted] (KAFKA-3919) Broker faills to start after ungraceful shutdown due to non-monotonically incrementing offsets in logs

[ https://issues.apache.org/jira/browse/KAFKA-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Coates updated KAFKA-3919: --- Comment: was deleted (was: Hi [~junrao], thanks for taking the time to look at this. Note: I've incor

[jira] [Commented] (KAFKA-3919) Broker faills to start after ungraceful shutdown due to non-monotonically incrementing offsets in logs

[ https://issues.apache.org/jira/browse/KAFKA-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358704#comment-15358704 ] Andy Coates commented on KAFKA-3919: Hi [~junrao], thanks for taking the time to look

[jira] [Updated] (KAFKA-3919) Broker faills to start after ungraceful shutdown due to non-monotonically incrementing offsets in logs

[ https://issues.apache.org/jira/browse/KAFKA-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Coates updated KAFKA-3919: --- Description: Hi All, I encountered an issue with Kafka following a power outage that saw a proportio

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Thank you, Jun. Hi all, Please let me know if you have any comments or suggestions on the updated KIP. If there are no objections, I will initiate voting next week. Thank you... On Thu, Jun 30, 2016 at 10:37 PM, Jun Rao wrote: > Rajini, > > The latest wiki looks good to me. Perhaps you want

[jira] [Comment Edited] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358566#comment-15358566 ] Jan Filipiak edited comment on KAFKA-3705 at 7/1/16 7:28 AM: -

[jira] [Commented] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358566#comment-15358566 ] Jan Filipiak commented on KAFKA-3705: - Hi, yes that is kinda where I am coming from. I

Build failed in Jenkins: kafka-trunk-jdk8 #727

See Changes: [me] MINOR: bug fixes to ducktape services -- [...truncated 6656 lines...] kafka.utils.SchedulerTest > testReentrantTaskInMockScheduler PASSED kafka.utils.SchedulerTest > testPerio

Build failed in Jenkins: kafka-trunk-jdk7 #1397

See Changes: [me] MINOR: bug fixes to ducktape services -- [...truncated 6716 lines...] kafka.utils.ReplicationUtilsTest > testUpdateLeaderAndIsr STARTED kafka.utils.ReplicationUtilsTest > tes