[GitHub] kafka pull request #1629: KAFKA-3960 - Committed offset not set after first ...

2016-07-18 Thread 13h3r
GitHub user 13h3r opened a pull request: https://github.com/apache/kafka/pull/1629 KAFKA-3960 - Committed offset not set after first assign Fixes https://issues.apache.org/jira/browse/KAFKA-3960 You can merge this pull request into a Git repository by running: $ git pull https:

[jira] [Commented] (KAFKA-3960) Committed offset not set after first assign

2016-07-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381888#comment-15381888 ] ASF GitHub Bot commented on KAFKA-3960: --- GitHub user 13h3r opened a pull request:

Helping Spread the Word about Apachecon EU 2016

2016-07-18 Thread Sharan Foga
Hi Everyone I'm forwarding the following message on behalf of Rich Bowen and the Apachecon team == As you are aware, we are holding ApacheCon in Seville in November. While this seems like a long way away, it is critical that we get on people's calendar now, so that they can plan, get b

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-18 Thread Jim Jagielski
Very cool. > On Jul 16, 2016, at 9:55 AM, Damian Guy wrote: > > Hi, > The vote is now complete and KIP-67 has been accepted and adopted. > Thanks everyone for the input etc. > > Regards, > Damian > > On Sat, 16 Jul 2016 at 06:53 Damian Guy wrote: > >> Hi, >> Jay's interpretation is correct.

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

2016-07-18 Thread Grant Henke
Hi Parth, Are you still working on this? If you need any help please don't hesitate to ask. Thanks, Grant On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao wrote: > Parth, > > Thanks for the reply. > > It makes sense to only allow the renewal by users that authenticated using > *non* delegation token m

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-07-18 Thread Vincent Rischmann (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382370#comment-15382370 ] Vincent Rischmann commented on KAFKA-3894: -- Not adding much to the conversation,

[GitHub] kafka pull request #1630: MINOR: Fix typo in Operations section

2016-07-18 Thread ssaamm
GitHub user ssaamm opened a pull request: https://github.com/apache/kafka/pull/1630 MINOR: Fix typo in Operations section This contribution is my original work, and I license the work to the project under the project's open source license. You can merge this pull request into a Git

[GitHub] kafka pull request #1631: KAFKA-3934: kafka-server-start.sh enables GC by de...

2016-07-18 Thread granthenke
GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/1631 KAFKA-3934: kafka-server-start.sh enables GC by default with no way t… …o disable You can merge this pull request into a Git repository by running: $ git pull https://github.com/granthen

Compacted topic cannot accept message without key

2016-07-18 Thread Kafka
Hi, The server log shows error as belows on broker 0.9.0. ERROR [Replica Manager on Broker 0]: Error processing append operation on partition [__consumer_offsets,5] (kafka.server.ReplicaManager) kafka.message.InvalidMessageException: Compacted topic cannot accept message without

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-07-18 Thread Peter Davis (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382511#comment-15382511 ] Peter Davis commented on KAFKA-3894: Re: "the broker seems to be working" You may reg

[jira] [Commented] (KAFKA-3934) kafka-server-start.sh enables GC by default with no way to disable

2016-07-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382484#comment-15382484 ] ASF GitHub Bot commented on KAFKA-3934: --- GitHub user granthenke opened a pull reques

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-07-18 Thread Vincent Rischmann (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382516#comment-15382516 ] Vincent Rischmann commented on KAFKA-3894: -- Yeah, that's why I was hoping for a w

[jira] [Commented] (KAFKA-3969) kafka.admin.ConsumerGroupCommand doesn't show consumer groups

2016-07-18 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382518#comment-15382518 ] Vahid Hashemian commented on KAFKA-3969: Without diving much into the code, I thin

[jira] [Updated] (KAFKA-3934) kafka-server-start.sh enables GC by default with no way to disable

2016-07-18 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3934: --- Status: Patch Available (was: Open) > kafka-server-start.sh enables GC by default with no way to disa

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

2016-07-18 Thread Harsha Chintalapani
Hi Grant, We are working on it. Will add the details to KIP about the request protocol. Thanks, Harsha On Mon, Jul 18, 2016 at 6:50 AM Grant Henke wrote: > Hi Parth, > > Are you still working on this? If you need any help please don't hesitate > to ask. > > Thanks, > Grant > > On Thu,

[jira] [Assigned] (KAFKA-3855) Guard race conditions in TopologyBuilder

2016-07-18 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3855: - Assignee: Damian Guy > Guard race conditions in TopologyBuilder > ---

[jira] [Work started] (KAFKA-3855) Guard race conditions in TopologyBuilder

2016-07-18 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3855 started by Damian Guy. - > Guard race conditions in TopologyBuilder > >

[GitHub] kafka pull request #1632: MINOR: MetadataCache brokerId is not set on first ...

2016-07-18 Thread granthenke
GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/1632 MINOR: MetadataCache brokerId is not set on first run with generated … …broker id This is because the id passed into the MetadataCache is the value from the config before the real br

[jira] [Commented] (KAFKA-3969) kafka.admin.ConsumerGroupCommand doesn't show consumer groups

2016-07-18 Thread Dieter Plaetinck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382618#comment-15382618 ] Dieter Plaetinck commented on KAFKA-3969: - thanks that works. seems like the "chec

Re: Compacted topic cannot accept message without key

2016-07-18 Thread Dustin Cote
Compacted topics require keyed messages in order for compaction to function. The solution is to provide a key for your messages. I would suggest reading the wiki on log compaction. On Mon, Jul 18, 2016 at 12:03 PM, Kafka wrote:

[jira] [Created] (KAFKA-3970) Message Listener(that can hit callback registered)

2016-07-18 Thread rajan verma (JIRA)
rajan verma created KAFKA-3970: -- Summary: Message Listener(that can hit callback registered) Key: KAFKA-3970 URL: https://issues.apache.org/jira/browse/KAFKA-3970 Project: Kafka Issue Type: New

[jira] [Updated] (KAFKA-3970) Message Listener(that can hit callback registered)

2016-07-18 Thread rajan verma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajan verma updated KAFKA-3970: --- Attachment: KafkaCallable.java KafkaAkkaConsumerQueueManager.java Consu

[GitHub] kafka pull request #1633: KAFKA-3855: Guard race conditions in TopologyBuild...

2016-07-18 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1633 KAFKA-3855: Guard race conditions in TopologyBuilder Mark all public `TopologyBuilder` methods as synchronized as they can modify data-structures and these methods could be called from multiple threads

[jira] [Commented] (KAFKA-3855) Guard race conditions in TopologyBuilder

2016-07-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382744#comment-15382744 ] ASF GitHub Bot commented on KAFKA-3855: --- GitHub user dguy opened a pull request:

[jira] [Created] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

2016-07-18 Thread Lei Wang (JIRA)
Lei Wang created KAFKA-3971: --- Summary: Consumers drop from coordinator and cannot reconnet Key: KAFKA-3971 URL: https://issues.apache.org/jira/browse/KAFKA-3971 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-3924) Data loss due to halting when LEO is larger than leader's LEO

2016-07-18 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated KAFKA-3924: --- Description: Currently the follower broker panics when its LEO is larger than its leader's LE

[jira] [Assigned] (KAFKA-3782) Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True

2016-07-18 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-3782: -- Assignee: Jason Gustafson (was: Ewen Cheslack-Postava) > Transient failure with > kaf

[jira] [Updated] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

2016-07-18 Thread Lei Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Wang updated KAFKA-3971: Description: >From time to time, we're creating new topics, and all consumers will pickup >those new topics

[jira] [Updated] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

2016-07-18 Thread Lei Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Wang updated KAFKA-3971: Attachment: KAFKA-3971.txt stacktrace of the process > Consumers drop from coordinator and cannot reconnet

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

2016-07-18 Thread Lei Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382833#comment-15382833 ] Lei Wang commented on KAFKA-3971: - consumer config: {noformat} Properties props = new

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

2016-07-18 Thread Lei Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382840#comment-15382840 ] Lei Wang commented on KAFKA-3971: - for different bad hosts, the coordinator id are differe

[GitHub] kafka pull request #1526: KAFKA-3870: Expose state store names in DSL

2016-07-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1526 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3870) Expose state store names to DSL

2016-07-18 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3870. -- Resolution: Fixed Issue resolved by pull request 1526 [https://github.com/apache/kafka/pull/1526

[jira] [Commented] (KAFKA-3870) Expose state store names to DSL

2016-07-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382875#comment-15382875 ] ASF GitHub Bot commented on KAFKA-3870: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1621: MINOR: Added simple streams benchmark to system te...

2016-07-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1621 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-07-18 Thread Vincent Rischmann (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382917#comment-15382917 ] Vincent Rischmann commented on KAFKA-3894: -- Well that doesn't work, in fact I jus

[jira] [Commented] (KAFKA-3940) Log should check the return value of dir.mkdirs()

2016-07-18 Thread Jim Jagielski (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382927#comment-15382927 ] Jim Jagielski commented on KAFKA-3940: -- Let me know if I can add myself to the assign

[jira] [Created] (KAFKA-3972) kafka java consumer poll returns 0 records after seekToBeginning

2016-07-18 Thread don caldwell (JIRA)
don caldwell created KAFKA-3972: --- Summary: kafka java consumer poll returns 0 records after seekToBeginning Key: KAFKA-3972 URL: https://issues.apache.org/jira/browse/KAFKA-3972 Project: Kafka

[GitHub] kafka pull request #1634: KAFKA-3924: Replacing halt with exit upon LEO mism...

2016-07-18 Thread maysamyabandeh
GitHub user maysamyabandeh opened a pull request: https://github.com/apache/kafka/pull/1634 KAFKA-3924: Replacing halt with exit upon LEO mismatch to trigger gra… …ceful shutdown The patch is pretty simple and the justification is explained in https://issues.apache.org/

[jira] [Commented] (KAFKA-3924) Data loss due to halting when LEO is larger than leader's LEO

2016-07-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383035#comment-15383035 ] ASF GitHub Bot commented on KAFKA-3924: --- GitHub user maysamyabandeh opened a pull re

[jira] [Updated] (KAFKA-3924) Data loss due to halting when LEO is larger than leader's LEO

2016-07-18 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated KAFKA-3924: --- Status: Patch Available (was: Open) Submitting a simple patch that replaces halt with exit, w

Build failed in Jenkins: kafka-trunk-jdk8 #754

2016-07-18 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3870: Expose state store names in DSL [wangguoz] MINOR: Added simple streams benchmark to system tests -- [...truncated 5317 lines...] kafka.log.LogCleanerIntegrat

[jira] [Commented] (KAFKA-3924) Data loss due to halting when LEO is larger than leader's LEO

2016-07-18 Thread Andrew Olson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383107#comment-15383107 ] Andrew Olson commented on KAFKA-3924: - Reviewed, looks good to me. > Data loss due to

[jira] [Updated] (KAFKA-2507) Replace ControlledShutdown{Request,Response} with org.apache.kafka.common.requests equivalent

2016-07-18 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2507: --- Fix Version/s: 0.11.0.0 > Replace ControlledShutdown{Request,Response} with > org.apache.kafka.common

[jira] [Updated] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3777: Assignee: Anna Povzner > Extract the LRU cache out of RocksDBStore > ---

[jira] [Created] (KAFKA-3973) Investigate feasibility of caching bytes vs. records

2016-07-18 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3973: --- Summary: Investigate feasibility of caching bytes vs. records Key: KAFKA-3973 URL: https://issues.apache.org/jira/browse/KAFKA-3973 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-3778) Avoiding using range queries of RocksDBWindowStore on KStream windowed aggregations

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3778: Summary: Avoiding using range queries of RocksDBWindowStore on KStream windowed aggregations (was:

[jira] [Updated] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3777: Summary: Extract the existing LRU cache out of RocksDBStore (was: Extract the LRU cache out of Rock

[jira] [Updated] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3777: Description: The LRU cache that is currently inside the RocksDbStore class. As part of KAFKA-3776 i

[jira] [Created] (KAFKA-3974) LRU cache should store bytes/object and not records

2016-07-18 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3974: --- Summary: LRU cache should store bytes/object and not records Key: KAFKA-3974 URL: https://issues.apache.org/jira/browse/KAFKA-3974 Project: Kafka Issue Type: S

[jira] [Resolved] (KAFKA-3101) Optimize Aggregation Outputs

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-3101. - Resolution: Duplicate This is a duplicate of KAFKA-3776. > Optimize Aggregation Outputs > ---

[jira] [Commented] (KAFKA-3101) Optimize Aggregation Outputs

2016-07-18 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383256#comment-15383256 ] Eno Thereska commented on KAFKA-3101: - Since this is a duplicate, will resolve. > Opt

Re: Compacted topic cannot accept message without key

2016-07-18 Thread Kafka
thanks for your answer,I know the necessity of key for compacted topics,and as you know,__consumer_offsets is a internal compacted topic in kafka,and it’s key is a triple of ,these errors are occurred when the consumer client wants to commit group offset. so why does his happen? > 在 2016年7月19日

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-07-18 Thread Jason Gustafson
Hey Simon, Sorry for the late response. The onPartitionsRevoked() hook is called before the rebalance begins (that is, before the JoinGroup is sent) and is intended to be used to flush uncommitted data and to commit corresponding offsets. One of the main purposes of the KIP is to decouple the time

[jira] [Commented] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

2016-07-18 Thread James Cheng (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383610#comment-15383610 ] James Cheng commented on KAFKA-3915: We just saw this in one of our brokers as well. W