[GitHub] kafka pull request #3333: MINOR: Mark AbstractLogCleanerIntegrationTest as a...

2017-06-13 Thread ewencp
GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/ MINOR: Mark AbstractLogCleanerIntegrationTest as an IntegrationTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka minor-mark-log-

[GitHub] kafka pull request #3332: MINOR: Enable request and other debug loggers duri...

2017-06-13 Thread ewencp
GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3332 MINOR: Enable request and other debug loggers during tests, directed at a NullAppender You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/ka

[jira] [Comment Edited] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048709#comment-16048709 ] Hongyuan Li edited comment on KAFKA-5444 at 6/14/17 6:15 AM: -

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-13 Thread Evgeny Veretennikov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048738#comment-16048738 ] Evgeny Veretennikov commented on KAFKA-5245: There are actually a lot of KStre

[jira] [Comment Edited] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048709#comment-16048709 ] Hongyuan Li edited comment on KAFKA-5444 at 6/14/17 4:58 AM: -

[jira] [Comment Edited] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048709#comment-16048709 ] Hongyuan Li edited comment on KAFKA-5444 at 6/14/17 4:57 AM: -

[jira] [Commented] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048709#comment-16048709 ] Hongyuan Li commented on KAFKA-5444: [~huxi_2b] May be not duplicate of KAF

[jira] [Updated] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated KAFKA-5444: --- Description: Frequent kafka old Producer open and close with cause the server hang with lots of error

Jenkins build is back to normal : kafka-trunk-jdk8 #1702

2017-06-13 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-5443) Consumer should use last offset from batch to set next fetch offset

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048688#comment-16048688 ] ASF GitHub Bot commented on KAFKA-5443: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request #3331: KAFKA-5443: Consumer should use last offset from b...

2017-06-13 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3331 KAFKA-5443: Consumer should use last offset from batch to set next fetch offset You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kaf

[jira] [Commented] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread huxihx (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048674#comment-16048674 ] huxihx commented on KAFKA-5444: --- Is it a duplicate of [KAFKA-3552|https://issues.apache.org

[GitHub] kafka pull request #3330: MINOR: Javadoc for ExtendedSerializer and Extended...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3330 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-13 Thread Hongyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated KAFKA-5444: --- Summary: Producer.send() will hang 8+ hours (was: Producer.send() will hang 8 hours, ) > Producer.se

[jira] [Created] (KAFKA-5444) Producer.send() will hang 8 hours,

2017-06-13 Thread Hongyuan Li (JIRA)
Hongyuan Li created KAFKA-5444: -- Summary: Producer.send() will hang 8 hours, Key: KAFKA-5444 URL: https://issues.apache.org/jira/browse/KAFKA-5444 Project: Kafka Issue Type: Bug Rep

[jira] [Commented] (KAFKA-3455) Connect custom processors with the streams DSL

2017-06-13 Thread Bobby Calderwood (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048626#comment-16048626 ] Bobby Calderwood commented on KAFKA-3455: - Hi Michal Borowiecki, Sorry for the la

[jira] [Commented] (KAFKA-5432) producer and consumer SocketTimeoutException

2017-06-13 Thread huxihx (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048615#comment-16048615 ] huxihx commented on KAFKA-5432: --- I might not see rolling new log segment as an indicator of

Build failed in Jenkins: kafka-trunk-jdk8 #1701

2017-06-13 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-5438; Fix UnsupportedOperationException in WriteTxnMarkersRequest [me] MINOR: Add unit tests for PluginDesc in Connect. -- [...truncated 4.21 M

[jira] [Commented] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-06-13 Thread huxihx (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048605#comment-16048605 ] huxihx commented on KAFKA-5007: --- [~joseph.alias...@gmail.com] org.apache.kafka.common.networ

[jira] [Commented] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-06-13 Thread Joseph Aliase (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048586#comment-16048586 ] Joseph Aliase commented on KAFKA-5007: -- [~huxi_2b] Please let me know the class this

[GitHub] kafka pull request #3319: MINOR: Verify mocks in all WorkerTest tests and do...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3319 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #3330: MINOR: Javadoc for ExtendedSerializer and Extended...

2017-06-13 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3330 MINOR: Javadoc for ExtendedSerializer and ExtendedDeserializer And add warning about usage. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/k

[jira] [Resolved] (KAFKA-5406) NoNodeException result in rebalance failed

2017-06-13 Thread xiaoguy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaoguy resolved KAFKA-5406. Resolution: Done it's the zk problem, the project used low version zk ,and it's can't work as normal when t

[jira] [Updated] (KAFKA-5435) Produce state lost if no snapshot retained

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5435: --- Status: Patch Available (was: Open) > Produce state lost if no snapshot retained > --

[jira] [Updated] (KAFKA-5214) Re-add KafkaAdminClient#apiVersions

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5214: --- Description: We removed KafkaAdminClient#apiVersions just before 0.11.0.0 to give us a bit more time t

[jira] [Updated] (KAFKA-5214) Re-add KafkaAdminClient#apiVersions

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5214: --- Summary: Re-add KafkaAdminClient#apiVersions (was: KafkaAdminClient#apiVersions should return a publi

[GitHub] kafka pull request #3324: Temporarily remove the apiVersions API for 0.11

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3324 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-5214) KafkaAdminClient#apiVersions should return a public class

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5214: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > KafkaAdminClient#apiVersions should r

[jira] [Created] (KAFKA-5443) Consumer should use last offset from batch to set next fetch offset

2017-06-13 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-5443: -- Summary: Consumer should use last offset from batch to set next fetch offset Key: KAFKA-5443 URL: https://issues.apache.org/jira/browse/KAFKA-5443 Project: Kafka

[jira] [Updated] (KAFKA-5413) Log cleaner fails due to large offset in segment file

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5413: --- Fix Version/s: 0.11.0.1 > Log cleaner fails due to large offset in segment file >

[GitHub] kafka pull request #3309: MINOR: NetworkClient#disconnect should not erase c...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3309 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-764) Race Condition in Broker Registration after ZooKeeper disconnect

2017-06-13 Thread Robert P. Thille (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert P. Thille updated KAFKA-764: --- Attachment: BPPF_2900-Broker_Logs.tbz2 I believe the issues started somewhere around the time o

[jira] [Commented] (KAFKA-764) Race Condition in Broker Registration after ZooKeeper disconnect

2017-06-13 Thread Robert P. Thille (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048552#comment-16048552 ] Robert P. Thille commented on KAFKA-764: I believe we saw this issue, or something

[jira] [Assigned] (KAFKA-5020) Update protocol documentation to mention message format v2

2017-06-13 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta reassigned KAFKA-5020: --- Assignee: Apurva Mehta > Update protocol documentation to mention message format v2 > ---

[jira] [Updated] (KAFKA-5413) Log cleaner fails due to large offset in segment file

2017-06-13 Thread Kelvin Rutt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kelvin Rutt updated KAFKA-5413: --- Attachment: kafka-5413.patch This is the patch I did. It's basic but does the job. > Log cleaner fai

[GitHub] kafka pull request #3326: MINOR: Add unit tests for PluginDesc in Connect.

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3326 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-5317) Update KIP-98 to reflect changes during implementation.

2017-06-13 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta resolved KAFKA-5317. - Resolution: Fixed > Update KIP-98 to reflect changes during implementation. >

[jira] [Commented] (KAFKA-5317) Update KIP-98 to reflect changes during implementation.

2017-06-13 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048515#comment-16048515 ] Apurva Mehta commented on KAFKA-5317: - I updated both the wiki and the google doc to r

[jira] [Updated] (KAFKA-5052) We shouldn't pass the underlying exception to RetriableCommitFailedException when an async offset commit fails.

2017-06-13 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta updated KAFKA-5052: Description: * This is a follow up to : https://issues.apache.org/jira/browse/KAFKA-3527 We current

[jira] [Commented] (KAFKA-5438) UnsupportedOperationException in WriteTxnMarkers handler

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048514#comment-16048514 ] ASF GitHub Bot commented on KAFKA-5438: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3313: KAFKA-5438: Fix UnsupportedOperationException in w...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3313 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-5438) UnsupportedOperationException in WriteTxnMarkers handler

2017-06-13 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5438. Resolution: Fixed Issue resolved by pull request 3313 [https://github.com/apache/kafka/pull/

[jira] [Updated] (KAFKA-5442) Streams producer `client.id` are not unique for EOS

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5442: --- Status: Patch Available (was: In Progress) > Streams producer `client.id` are not unique for

[jira] [Commented] (KAFKA-5442) Streams producer `client.id` are not unique for EOS

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048509#comment-16048509 ] ASF GitHub Bot commented on KAFKA-5442: --- GitHub user mjsax opened a pull request:

[GitHub] kafka pull request #3329: KAFKA-5442: Streams producer client.id are not uni...

2017-06-13 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3329 KAFKA-5442: Streams producer client.id are not unique for EOS You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka kafka-5442-producer-id-c

[GitHub] kafka pull request #3328: HOTFIX: log level should be ERROR for conflicting ...

2017-06-13 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3328 HOTFIX: log level should be ERROR for conflicting metric names You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfix-app-info-log Al

[GitHub] kafka pull request #3327: HOTFIX: Introduce max wait time for retry-and-back...

2017-06-13 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3327 HOTFIX: Introduce max wait time for retry-and-backoff while creating tasks You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfix-back

[jira] [Work started] (KAFKA-5442) Streams producer `client.id` are not unique for EOS

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5442 started by Matthias J. Sax. -- > Streams producer `client.id` are not unique for EOS > -

[jira] [Created] (KAFKA-5442) Streams producer `client.id` are not unique for EOS

2017-06-13 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-5442: -- Summary: Streams producer `client.id` are not unique for EOS Key: KAFKA-5442 URL: https://issues.apache.org/jira/browse/KAFKA-5442 Project: Kafka Issue T

[jira] [Comment Edited] (KAFKA-5348) kafka-consumer-groups.sh refuses to remove groups without ids

2017-06-13 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048486#comment-16048486 ] Vahid Hashemian edited comment on KAFKA-5348 at 6/13/17 10:36 PM: --

[GitHub] kafka pull request #3326: MINOR: Add unit tests for PluginDesc in Connect.

2017-06-13 Thread kkonstantine
GitHub user kkonstantine opened a pull request: https://github.com/apache/kafka/pull/3326 MINOR: Add unit tests for PluginDesc in Connect. Related to https://github.com/apache/kafka/pull/3321 You can merge this pull request into a Git repository by running: $ git pull https://g

[jira] [Commented] (KAFKA-5348) kafka-consumer-groups.sh refuses to remove groups without ids

2017-06-13 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048486#comment-16048486 ] Vahid Hashemian commented on KAFKA-5348: [~bobrik] In the scenario you described I

[jira] [Updated] (KAFKA-5441) Fix transaction marker grouping by producerId in TC

2017-06-13 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta updated KAFKA-5441: Labels: exactly-once (was: ) > Fix transaction marker grouping by producerId in TC > --

[jira] [Created] (KAFKA-5441) Fix transaction marker grouping by producerId in TC

2017-06-13 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-5441: -- Summary: Fix transaction marker grouping by producerId in TC Key: KAFKA-5441 URL: https://issues.apache.org/jira/browse/KAFKA-5441 Project: Kafka Issue T

[jira] [Updated] (KAFKA-4585) Offset fetch and commit requests use the same permissions

2017-06-13 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-4585: --- Labels: kip (was: needs-kip) > Offset fetch and commit requests use the same permissions > --

[jira] [Work started] (KAFKA-5370) Replace uses of old consumer with the new consumer

2017-06-13 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5370 started by Vahid Hashemian. -- > Replace uses of old consumer with the new consumer > -

[jira] [Assigned] (KAFKA-5212) Consumer ListOffsets request can starve group heartbeats

2017-06-13 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-5212: -- Assignee: (was: Jason Gustafson) > Consumer ListOffsets request can starve group he

[GitHub] kafka pull request #3321: HOTFIX: Handle Connector version returning 'null' ...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

2017-06-13 Thread Vahid S Hashemian
Hi Michal, Thanks a lot for your feedback. Your statement about Heartbeat is fair and makes sense. I'll update the KIP accordingly. --Vahid From: Michal Borowiecki To: us...@kafka.apache.org, Vahid S Hashemian , dev@kafka.apache.org Date: 06/13/2017 01:35 AM Subject:Re: [D

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-13 Thread Bill Bejeck
+1 on deferring discussion on Serdes until API improvements are ironed out. On Tue, Jun 13, 2017 at 2:06 PM, Matthias J. Sax wrote: > Hi, > > I am just catching up on this thread. (1) as most people agree, we > should not add anything to KStreamBuilder (btw: we actually plan to move > #merge() t

[jira] [Commented] (KAFKA-5413) Log cleaner fails due to large offset in segment file

2017-06-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048429#comment-16048429 ] Jun Rao commented on KAFKA-5413: [~Kelvinrutt], yes, that seems to be the issue. segs.head

[DISCUSS] KIP-167: Add interface for the state store restoration process

2017-06-13 Thread Bill Bejeck
The KIP has been updated to incorporate support for users to receive notification of state store restore progress for either pre-existing or custom state stores. A PR has been submitted concurrently https://github.com/apache/kafka/pull/3325 Thanks, Bill

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-06-13 Thread Jeyhun Karimov
Thanks for the comment Matthias. After all the discussion (thanks to all participants), I think this (single method that passes in a RecordContext object) is the best alternative. Just a side note: I think KAFKA-3907 [1] can also be integrated into the KIP by adding related method inside RecordCont

[jira] [Commented] (KAFKA-5413) Log cleaner fails due to large offset in segment file

2017-06-13 Thread Nicholas Ngorok (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048394#comment-16048394 ] Nicholas Ngorok commented on KAFKA-5413: This happened again today on another clus

[jira] [Commented] (KAFKA-5363) Add ability to batch restore and receive restoration stats.

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048378#comment-16048378 ] ASF GitHub Bot commented on KAFKA-5363: --- GitHub user bbejeck opened a pull request:

[GitHub] kafka pull request #3325: KAFKA-5363 [WIP] : Initial cut at implementing bul...

2017-06-13 Thread bbejeck
GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3325 KAFKA-5363 [WIP] : Initial cut at implementing bulk load for persistent stat… …e stores You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbej

[GitHub] kafka pull request #3324: Temporarily remove the apiVersions API for 0.11

2017-06-13 Thread cmccabe
GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/3324 Temporarily remove the apiVersions API for 0.11 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cmccabe/kafka removeApiVersions Alternatively y

Re: [VOTE] KIP-117: Add a public AdminClient API for Kafka admin operations

2017-06-13 Thread Colin McCabe
Hi all, We're getting close to the final 0.11 release and we still haven't gotten the version retrieval API for AdminClient in, as described by KAFKA-5214. Let's put this off until the next release so that we have some time to let it soak before shipping. This will just mean that we will remove

Re: [DISCUSS] KIP-132: Augment KStream.print to allow extra parameters in the printed string

2017-06-13 Thread Matthias J. Sax
As there was no response from Marc, and we got KIP-160 as duplicate, I am "closing" this KIP without a vote. If there are any objections please let me know. -Matthias On 6/7/17 5:24 PM, Matthias J. Sax wrote: > Hi, > > as it turns out, we got two KIPs for the same Jira. KIP-160 duplicates > KIP

[jira] [Updated] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5354: --- Status: Patch Available (was: Open) > MirrorMaker not preserving headers > --

[jira] [Updated] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5354: --- Fix Version/s: 0.11.0.0 > MirrorMaker not preserving headers > -- > >

[jira] [Updated] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5354: --- Priority: Blocker (was: Major) > MirrorMaker not preserving headers > ---

[jira] [Work started] (KAFKA-4868) Optimize RocksDb config for fast recovery/bulk load

2017-06-13 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4868 started by Bill Bejeck. -- > Optimize RocksDb config for fast recovery/bulk load > -

[jira] [Work started] (KAFKA-5363) Add ability to batch restore and receive restoration stats.

2017-06-13 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5363 started by Bill Bejeck. -- > Add ability to batch restore and receive restoration stats. > -

[jira] [Comment Edited] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Michael Andre Pearce (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048320#comment-16048320 ] Michael Andre Pearce edited comment on KAFKA-5354 at 6/13/17 7:58 PM: --

[jira] [Comment Edited] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Michael Andre Pearce (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048320#comment-16048320 ] Michael Andre Pearce edited comment on KAFKA-5354 at 6/13/17 7:58 PM: --

[jira] [Comment Edited] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Michael Andre Pearce (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048320#comment-16048320 ] Michael Andre Pearce edited comment on KAFKA-5354 at 6/13/17 7:57 PM: --

[jira] [Commented] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread Michael Andre Pearce (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048320#comment-16048320 ] Michael Andre Pearce commented on KAFKA-5354: - [~junrao] yes it was a bug, I'v

[jira] [Commented] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048318#comment-16048318 ] ASF GitHub Bot commented on KAFKA-5354: --- GitHub user michaelandrepearce opened a pul

[GitHub] kafka pull request #3323: KAFKA-5354: MirrorMaker not preserving headers

2017-06-13 Thread michaelandrepearce
GitHub user michaelandrepearce opened a pull request: https://github.com/apache/kafka/pull/3323 KAFKA-5354: MirrorMaker not preserving headers Add test case to ensure fix and avoid regression Update mirror maker for new consumer to preserve headers You can merge this pull reques

[jira] [Commented] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048298#comment-16048298 ] ASF GitHub Bot commented on KAFKA-5354: --- GitHub user michaelandrepearce opened a pul

[GitHub] kafka pull request #3322: KAFKA-5354: MirrorMaker not preserving headers

2017-06-13 Thread michaelandrepearce
GitHub user michaelandrepearce opened a pull request: https://github.com/apache/kafka/pull/3322 KAFKA-5354: MirrorMaker not preserving headers Add test case to ensure fix and avoid regression Update mirror maker for new consumer to preserve headers You can merge this pull reques

[GitHub] kafka pull request #3321: HOTFIX: Handle Connector version returning 'null' ...

2017-06-13 Thread kkonstantine
GitHub user kkonstantine opened a pull request: https://github.com/apache/kafka/pull/3321 HOTFIX: Handle Connector version returning 'null' during plugin loading. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kkonstantine/kafka

[jira] [Commented] (KAFKA-5370) Replace uses of old consumer with the new consumer

2017-06-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048236#comment-16048236 ] ASF GitHub Bot commented on KAFKA-5370: --- GitHub user vahidhashemian opened a pull re

[GitHub] kafka pull request #3320: KAFKA-5370 (WIP): Replace uses of the old consumer...

2017-06-13 Thread vahidhashemian
GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/3320 KAFKA-5370 (WIP): Replace uses of the old consumer with the new consumer when possible Also, methods in `ClientUtils` that are called by server or tools code should be introduced in `AdminUt

[jira] [Updated] (KAFKA-4653) Improve test coverage of RocksDBStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4653: --- Description: {{putAll}} - not covered {{putInternal}} - exceptions was: {{putAll}} - not

[jira] [Updated] (KAFKA-4653) Improve test coverage of RocksDBStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4653: --- Status: Patch Available (was: Open) > Improve test coverage of RocksDBStore > ---

[jira] [Updated] (KAFKA-4658) Improve test coverage InMemoryKeyValueLoggedStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4658: --- Status: Patch Available (was: Open) > Improve test coverage InMemoryKeyValueLoggedStore > ---

[jira] [Updated] (KAFKA-4658) Improve test coverage InMemoryKeyValueLoggedStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4658: --- Description: {{putAll}} not covered (was: {{putAll} not covered) > Improve test coverage InMe

[GitHub] kafka pull request #3319: MINOR: Verify mocks in all WorkerTest tests and do...

2017-06-13 Thread ewencp
GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3319 MINOR: Verify mocks in all WorkerTest tests and don't unnecessarily mockStatic the Plugins class. You can merge this pull request into a Git repository by running: $ git pull https://github.co

[jira] [Updated] (KAFKA-4656) Improve test coverage of CompositeReadOnlyKeyValueStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4656: --- Status: Patch Available (was: Open) > Improve test coverage of CompositeReadOnlyKeyValueStore

[jira] [Updated] (KAFKA-4659) Improve test coverage of CachingKeyValueStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4659: --- Status: Patch Available (was: Open) > Improve test coverage of CachingKeyValueStore > ---

[jira] [Updated] (KAFKA-4655) Improve test coverage of CompositeReadOnlySessionStore

2017-06-13 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4655: --- Status: Patch Available (was: Open) > Improve test coverage of CompositeReadOnlySessionStore

[GitHub] kafka pull request #3317: MINOR: Clean up in DeleteConsumerGroupTest even if...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3317 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-5382) Log recovery can fail if topic names contain one of the index suffixes

2017-06-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-5382: - Fix Version/s: (was: 0.11.0.0.) 0.11.0.0 > Log recovery can

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-13 Thread Matthias J. Sax
Hi, I am just catching up on this thread. (1) as most people agree, we should not add anything to KStreamBuilder (btw: we actually plan to move #merge() to KStream and deprecate it on KStreamBuilder as it's a quite unnatural API atm). About specifying Serdes: there is still the idea to improve to

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-06-13 Thread Matthias J. Sax
Hi, I would like to push this discussion further. It seems we got nice alternatives (thanks for the summary Jeyhun!). With respect to RichFunctions and allowing them to be stateful, I have my doubt as expressed already. From my understanding, the idea was to give access to record metadata informa

[GitHub] kafka pull request #3318: HOTFIX: Fix invalid long format conversion in requ...

2017-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3318 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-06-13 Thread Matthias J. Sax
I agree with Guozhang's second point. This change does not seem backward compatible. As we don't have to support lambdas, it might be the easiest thing to just extend the current interface: > public interface ValueTransformerWithKey extends > ValueTransformer When plugging the topology togethe

[GitHub] kafka pull request #3318: HOTFIX: Fix invalid long format conversion in requ...

2017-06-13 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3318 HOTFIX: Fix invalid long format conversion in request logger message You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka hotfix-re

  1   2   >