[jira] [Updated] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

2017-05-26 Thread James Cheng (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng updated KAFKA-5191: --- Reviewer: Guozhang Wang (was: Jason Gustafson) Changing reviewer to [~guozhang], since he's currently

[jira] [Created] (KAFKA-5332) When resize the index file, maybe caused the content disappear

2017-05-26 Thread xuzq (JIRA)
xuzq created KAFKA-5332: --- Summary: When resize the index file, maybe caused the content disappear Key: KAFKA-5332 URL: https://issues.apache.org/jira/browse/KAFKA-5332 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #3150: modify code to fix KAFKA-5332

2017-05-26 Thread ZanderXu
GitHub user ZanderXu opened a pull request: https://github.com/apache/kafka/pull/3150 modify code to fix KAFKA-5332 this modify is to fix [KAFKA-5332](https://issues.apache.org/jira/browse/KAFKA-5332) You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-5332) When resize the index file, maybe caused the content disappear

2017-05-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026011#comment-16026011 ] ASF GitHub Bot commented on KAFKA-5332: --- GitHub user ZanderXu opened a pull request:

[GitHub] kafka pull request #3145: MINOR: Add test case for duplicate check after log...

2017-05-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3145 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Eno Thereska
Replying to Avi's and Matthias' questions in one go inline: > On 25 May 2017, at 19:27, Matthias J. Sax wrote: > > Thanks for the KIP Eno! > > Couple of comments: > > I think we don't need `RecordContext` in `RecordExceptionHandler#handle` > because the `ConsumerRecord` provides all this infor

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Eno Thereska
Thanks Jan, The record passed to the handler will always be the problematic record. There are 2 cases/types of exceptions for the purposes of this KIP: 1) any exception during deserialization. The bad record + the exception (i.e. DeserializeException) will be passed to the handler. The handler

[jira] [Commented] (KAFKA-5329) Replica list in the metadata cache on the broker may have different order from zookeeper

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026049#comment-16026049 ] Ismael Juma commented on KAFKA-5329: Are you sure that this is an issue [~becket_qin]?

[jira] [Commented] (KAFKA-5332) When resize the index file, maybe caused the content disappear

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026050#comment-16026050 ] Ismael Juma commented on KAFKA-5332: cc [~junrao] > When resize the index file, maybe

[jira] [Commented] (KAFKA-5328) consider switching json parser from scala to jackson

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026052#comment-16026052 ] Ismael Juma commented on KAFKA-5328: Duplicate of KAFKA-1595, which had a PR. If there

[GitHub] kafka pull request #3151: KAFKA-5331:MirrorMaker can't guarantee data is con...

2017-05-26 Thread ZanderXu
GitHub user ZanderXu opened a pull request: https://github.com/apache/kafka/pull/3151 KAFKA-5331:MirrorMaker can't guarantee data is consistent between clusters this modify is to fix [KAFKA-5331](https://issues.apache.org/jira/browse/KAFKA-5331) You can merge this pull request into

[jira] [Commented] (KAFKA-5331) MirrorMaker can't guarantee data is consistent between clusters

2017-05-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026062#comment-16026062 ] ASF GitHub Bot commented on KAFKA-5331: --- GitHub user ZanderXu opened a pull request:

Re: GlobalKTable limitations

2017-05-26 Thread Damian Guy
We did originally think of allowing `filter`, `map` etc on a GlobalKTable, though it would have been slightly different to what you are suggesting, i.e., we'd materialize the topic as a Store and then provide views on top of that. They'd also be global, but not materialized as physical state stores

[jira] [Commented] (KAFKA-5308) TC should handle UNSUPPORTED_FOR_MESSAGE_FORMAT in WriteTxnMarker response

2017-05-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026073#comment-16026073 ] ASF GitHub Bot commented on KAFKA-5308: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #3152: KAFKA-5308: TC should handle UNSUPPORTED_FOR_MESSA...

2017-05-26 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3152 KAFKA-5308: TC should handle UNSUPPORTED_FOR_MESSAGE_FORMAT in WriteTxnMarker response Return UNSUPPORTED_MESSAGE_FORMAT in handleWriteTxnMarkers when a topic is not the correct message format. Re

Build failed in Jenkins: kafka-trunk-jdk7 #2271

2017-05-26 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Preserve the base offset of the original record batch in V2 -- [...truncated 901.23 KB...] kafka.producer.ProducerTest > testUpdateBrokerParti

[jira] [Updated] (KAFKA-5308) TC should handle UNSUPPORTED_FOR_MESSAGE_FORMAT in WriteTxnMarker response

2017-05-26 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-5308: -- Status: Patch Available (was: Open) > TC should handle UNSUPPORTED_FOR_MESSAGE_FORMAT in WriteTxnMarker

[jira] [Commented] (KAFKA-5031) Additional validation in validateMessagesAndAssignOffsets

2017-05-26 Thread Janek P (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026127#comment-16026127 ] Janek P commented on KAFKA-5031: [~apurva] thanks for the clarification. I am working on t

[jira] [Commented] (KAFKA-5332) When resize the index file, maybe caused the content disappear

2017-05-26 Thread xuzq (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026130#comment-16026130 ] xuzq commented on KAFKA-5332: - [~junrao] > When resize the index file, maybe caused the cont

Jenkins build is back to normal : kafka-trunk-jdk8 #1597

2017-05-26 Thread Apache Jenkins Server
See

Jenkins build is back to normal : kafka-trunk-jdk7 #2272

2017-05-26 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-5032) Think through implications of max.message.size affecting record batches in message format V2

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026211#comment-16026211 ] Ismael Juma commented on KAFKA-5032: [~apurva], docs sound good. Maybe we should consi

[jira] [Resolved] (KAFKA-5250) handleFetchRequest should do down conversion after throttling

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5250. Resolution: Fixed > handleFetchRequest should do down conversion after throttling >

[jira] [Updated] (KAFKA-5025) FetchRequestTest should use batches with more than one message

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5025: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > FetchRequestTest should use batches w

[jira] [Updated] (KAFKA-5030) Suggested clean-ups from message format V2 review

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5030: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Suggested clean-ups from message form

[jira] [Updated] (KAFKA-5235) GetOffsetShell: retrieve offsets for all given topics and partitions with single request to the broker

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5235: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > GetOffsetShell: retrieve offsets for

[jira] [Updated] (KAFKA-5017) Consider making baseOffset the first offset in message format v2

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5017: --- Fix Version/s: (was: 0.11.0.0) > Consider making baseOffset the first offset in message format v2

[jira] [Updated] (KAFKA-5293) Do not apply exponential backoff if users have overridden reconnect.backoff.ms

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5293: --- Reporter: Ismael Juma (was: Colin P. McCabe) > Do not apply exponential backoff if users have overrid

[jira] [Updated] (KAFKA-5234) GetOffsetShell: retrieve offsets for multiple topics with single request

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5234: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > GetOffsetShell: retrieve offsets for

[jira] [Updated] (KAFKA-3554) Generate actual data with specific compression ratio and add multi-thread support in the ProducerPerformance tool.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3554: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Generate actual data with specific co

[jira] [Updated] (KAFKA-5185) Adding the RecordMetadata that is returned by the producer to the commitRecord method for SourceTask

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5185: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Adding the RecordMetadata that is ret

[jira] [Updated] (KAFKA-4764) Improve diagnostics for SASL authentication failures

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4764: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve diagnostics for SASL authenti

[GitHub] kafka pull request #3153: MINOR: clarify partition option behavior for conso...

2017-05-26 Thread cotedm
GitHub user cotedm opened a pull request: https://github.com/apache/kafka/pull/3153 MINOR: clarify partition option behavior for console consumer Console Consumer help doesn't say that ``--partition`` option needs ``--offset`` otherwise will consume from the end of the partition. Th

[jira] [Resolved] (KAFKA-4421) Update release process so that Scala 2.12 artifacts are published

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4421. Resolution: Fixed Assignee: Ewen Cheslack-Postava > Update release process so that Scala 2.12

[jira] [Updated] (KAFKA-5133) Changing "connection refused" message from logging level "debug" into "warn"

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5133: --- Fix Version/s: (was: 0.11.0.0) > Changing "connection refused" message from logging level "debug"

[jira] [Comment Edited] (KAFKA-5133) Changing "connection refused" message from logging level "debug" into "warn"

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026218#comment-16026218 ] Ismael Juma edited comment on KAFKA-5133 at 5/26/17 12:36 PM: --

[jira] [Resolved] (KAFKA-5133) Changing "connection refused" message from logging level "debug" into "warn"

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5133. Resolution: Duplicate This was resolved via a different ticket so that we get warnings if the initia

[jira] [Resolved] (KAFKA-5108) Add support for reading PID snapshot files to DumpLogSegments

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5108. Resolution: Fixed > Add support for reading PID snapshot files to DumpLogSegments >

[jira] [Updated] (KAFKA-1894) Avoid long or infinite blocking in the consumer

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1894: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Avoid long or infinite blocking in th

[jira] [Updated] (KAFKA-5116) Controller updates to ISR holds the controller lock for a very long time

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5116: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Controller updates to ISR holds the c

[jira] [Updated] (KAFKA-5092) KIP 141 - ProducerRecord Interface Improvements

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5092: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > KIP 141 - ProducerRecord Interface Im

[jira] [Updated] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3042: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > updateIsr should stop after failed se

[jira] [Updated] (KAFKA-4423) Drop support for Java 7

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4423: --- Fix Version/s: (was: 0.11.0.0) 0.12.0.0 > Drop support for Java 7 > ---

[jira] [Updated] (KAFKA-4501) Support Java 9

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4501: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Support Java 9 > -- > >

[jira] [Commented] (KAFKA-5116) Controller updates to ISR holds the controller lock for a very long time

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026224#comment-16026224 ] Ismael Juma commented on KAFKA-5116: Moving to the next release given that the code fr

[jira] [Updated] (KAFKA-4862) Kafka client connect to a shutdown node will block for a long time

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4862: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Kafka client connect to a shutdown no

[jira] [Commented] (KAFKA-3955) Kafka log recovery doesn't truncate logs on non-monotonic offsets, leading to failed broker boot

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026227#comment-16026227 ] Ismael Juma commented on KAFKA-3955: Unless we get a PR for this very soon, it won't h

[jira] [Updated] (KAFKA-5011) Replica fetchers may need to down-convert messages during a selective message format upgrade

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5011: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Replica fetchers may need to down-con

[jira] [Updated] (KAFKA-5023) Remove cyclic dependency between record and protocol packages

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5023: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Remove cyclic dependency between reco

[jira] [Resolved] (KAFKA-4973) Transient failure of AdminClientTest.testDeleteRecordsWithException

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4973. Resolution: Fixed > Transient failure of AdminClientTest.testDeleteRecordsWithException > --

[jira] [Updated] (KAFKA-3436) Speed up controlled shutdown.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3436: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Speed up controlled shutdown. > -

[jira] [Updated] (KAFKA-5022) Improve CRC tests so that we verify which fields are included in the CRC

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5022: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve CRC tests so that we verify w

[jira] [Updated] (KAFKA-4922) Fix several FindBugs warnings in Clients and Connect

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4922: --- Resolution: Fixed Status: Resolved (was: Patch Available) I believe these were fixed via a di

[jira] [Updated] (KAFKA-1326) New consumer checklist

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1326: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > New consumer checklist >

[jira] [Updated] (KAFKA-3186) KIP-50: Move Authorizer and related classes to separate package.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3186: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > KIP-50: Move Authorizer and related c

[jira] [Updated] (KAFKA-3300) Calculate the initial/max size of offset index files and reduce the memory footprint for memory mapped index files.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3300: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Calculate the initial/max size of off

[jira] [Updated] (KAFKA-4602) KIP-72 Allow putting a bound on memory consumed by Incoming requests

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4602: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > KIP-72 Allow putting a bound on memor

[jira] [Updated] (KAFKA-1342) Slow controlled shutdowns can result in stale shutdown requests

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1342: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Slow controlled shutdowns can result

[jira] [Updated] (KAFKA-2983) Remove old Scala clients and all related code, tests, and tools.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2983: --- Fix Version/s: (was: 0.11.0.0) 0.12.0.0 > Remove old Scala clients and all rela

[jira] [Updated] (KAFKA-3177) Kafka consumer can hang when position() is called on a non-existing partition.

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3177: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Kafka consumer can hang when position

[jira] [Updated] (KAFKA-3190) KafkaProducer should not invoke callback in send()

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3190: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > KafkaProducer should not invoke callb

[jira] [Updated] (KAFKA-3331) Refactor TopicCommand to make it testable and add unit tests

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3331: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Refactor TopicCommand to make it test

[jira] [Updated] (KAFKA-4440) Make producer RecordMetadata non-final

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4440: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Make producer RecordMetadata non-fina

[jira] [Updated] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3297: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > More optimally balanced partition ass

[jira] [Updated] (KAFKA-3135) Unexpected delay before fetch response transmission

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3135: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Unexpected delay before fetch respons

[jira] [Updated] (KAFKA-3247) kafka-topic tool can use an option to see just "unbalanced partitions"

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3247: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > kafka-topic tool can use an option to

[jira] [Updated] (KAFKA-4345) Run ducktape test for each pull request

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4345: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Run ducktape test for each pull reque

[jira] [Updated] (KAFKA-4459) rat license check not running in Jenkins

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4459: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > rat license check not running in Jenk

[jira] [Created] (KAFKA-5333) Remove Broker ACL resource type

2017-05-26 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-5333: -- Summary: Remove Broker ACL resource type Key: KAFKA-5333 URL: https://issues.apache.org/jira/browse/KAFKA-5333 Project: Kafka Issue Type: Improvement

[jira] [Issue Comment Deleted] (KAFKA-5332) When resize the index file, maybe caused the content disappear

2017-05-26 Thread xuzq (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuzq updated KAFKA-5332: Comment: was deleted (was: [~junrao] ) > When resize the index file, maybe caused the content disappear > -

[GitHub] kafka pull request #3154: KAFKA-5333; Remove Broker ACL resource type

2017-05-26 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3154 KAFKA-5333; Remove Broker ACL resource type You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-5333-remove-broker-acl-resource-typ

[jira] [Commented] (KAFKA-5333) Remove Broker ACL resource type

2017-05-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026259#comment-16026259 ] ASF GitHub Bot commented on KAFKA-5333: --- GitHub user ijuma opened a pull request:

[jira] [Updated] (KAFKA-5333) Remove Broker ACL resource type

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5333: --- Status: Patch Available (was: Open) > Remove Broker ACL resource type > -

[jira] [Updated] (KAFKA-5333) Remove Broker ACL resource type

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5333: --- Fix Version/s: 0.11.0.0 > Remove Broker ACL resource type > --- > >

[jira] [Updated] (KAFKA-5333) Remove Broker ACL resource type

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5333: --- Priority: Critical (was: Major) > Remove Broker ACL resource type > --- >

[jira] [Updated] (KAFKA-5157) Options for handling corrupt data during deserialization

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5157: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Options for handling corrupt data d

[jira] [Updated] (KAFKA-5217) Improve Streams internal exception handling

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5217: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve Streams internal exception

[jira] [Updated] (KAFKA-4563) State transitions error PARTITIONS_REVOKED to NOT_RUNNING

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-4563: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > State transitions error PARTITIONS_

[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5158: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Options for handling exceptions dur

[jira] [Updated] (KAFKA-5300) Improve exception handling on producer path

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5300: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve exception handling on produ

[jira] [Updated] (KAFKA-5313) Improve exception handling on coordinator interactions

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5313: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve exception handling on coord

[jira] [Updated] (KAFKA-5302) Improve exception handling on streams client (communication with brokers)

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5302: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve exception handling on strea

[jira] [Updated] (KAFKA-5314) Improve exception handling for state stores

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5314: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve exception handling for stat

[jira] [Updated] (KAFKA-5219) Move transaction expiration logic and scheduling to the Transaction Manager

2017-05-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5219: --- Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Move transaction expiration logic and

KIP-162: Enable topic deletion by default

2017-05-26 Thread Gwen Shapira
Hi Kafka developers, users and friends, I've added a KIP to improve our out-of-the-box usability a bit: KIP-162: Enable topic deletion by default: https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default Pretty simple :) Discussion and feedback are welcome. Gw

[jira] [Updated] (KAFKA-5226) NullPointerException (NPE) in SourceNodeRecordDeserializer.deserialize

2017-05-26 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-5226: --- Fix Version/s: 0.11.0.0 > NullPointerException (NPE) in SourceNodeRecordDeserializer.deserialize > ---

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Matthias J. Sax
+1 On 5/26/17 7:03 AM, Gwen Shapira wrote: > Hi Kafka developers, users and friends, > > I've added a KIP to improve our out-of-the-box usability a bit: > KIP-162: Enable topic deletion by default: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default > >

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Jorge Esteban Quilcate Otoya
+1 El vie., 26 may. 2017 a las 16:14, Matthias J. Sax () escribió: > +1 > > On 5/26/17 7:03 AM, Gwen Shapira wrote: > > Hi Kafka developers, users and friends, > > > > I've added a KIP to improve our out-of-the-box usability a bit: > > KIP-162: Enable topic deletion by default: > > > https://cwik

[jira] [Created] (KAFKA-5334) rocksdb.config.setter must be a class instance, not a class name

2017-05-26 Thread Tommy Becker (JIRA)
Tommy Becker created KAFKA-5334: --- Summary: rocksdb.config.setter must be a class instance, not a class name Key: KAFKA-5334 URL: https://issues.apache.org/jira/browse/KAFKA-5334 Project: Kafka

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Ismael Juma
Thanks for the KIP, sounds good to me. One comment: not sure we need to add the config to server.properties. Do we expect people to change this default? On Fri, May 26, 2017 at 3:03 PM, Gwen Shapira wrote: > Hi Kafka developers, users and friends, > > I've added a KIP to improve our out-of-the-b

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Matthias J. Sax
About `LogAndThresholdExceptionHandler`: If the handler needs to keep track of number of failed messages, than it becomes stateful -- not sure if we should do that. But maybe we can introduce 2 metrics (might be an interesting metric to report to the user anyway) and allow programmatic access to t

[GitHub] kafka pull request #3155: Fix KAKFA-5334.

2017-05-26 Thread twbecker
GitHub user twbecker opened a pull request: https://github.com/apache/kafka/pull/3155 Fix KAKFA-5334. Handle` rocksdb.config.setter` being set as a class name or class instance. You can merge this pull request into a Git repository by running: $ git pull https://github.com/

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Damian Guy
+1 Also agree with what Ismael said. On Fri, 26 May 2017 at 15:26 Ismael Juma wrote: > Thanks for the KIP, sounds good to me. One comment: not sure we need to add > the config to server.properties. Do we expect people to change this > default? > > On Fri, May 26, 2017 at 3:03 PM, Gwen Shapira w

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Tom Crayford
+1 (non-binding) On Fri, May 26, 2017 at 3:38 PM, Damian Guy wrote: > +1 > Also agree with what Ismael said. > > On Fri, 26 May 2017 at 15:26 Ismael Juma wrote: > > > Thanks for the KIP, sounds good to me. One comment: not sure we need to > add > > the config to server.properties. Do we expect

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Jim Jagielski
> On May 26, 2017, at 5:13 AM, Eno Thereska wrote: > > >> >> >> With regard to `DeserializationException`, do you thing it might make >> sense to have a "dead letter queue" as a feature to provide out-of-the-box? > > We could provide a special topic where bad messages go to, and then we'd ha

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Gwen Shapira
This was a discussion, not a vote (sorry for mangling the title), but thanks for all the +1 anyway. Regarding Ismael's feedback: The current server.properties includes the following: # Switch to enable topic deletion or not, default value is false #delete.topic.enable=true We can't leave it as is

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Matthias J. Sax
"bad" for this case would mean, that we got an `DeserializationException`. I am not sure if any other processing error should be covered? @Eno: this raises one one question. Might it be better to allow for two handlers instead of one? One for deserialization exception and one for all other excepti

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Eno Thereska
Hi Jan, You're right. I think I got carried away and broadened the scope of this KIP beyond it's original purpose. This handler will only be there for deserialization errors, i.e., "poison pills" and is not intended to be a catch-all handler for all sorts of other problems (e.g., NPE exception

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-26 Thread Eno Thereska
See latest reply to Jan's note. I think I unnecessarily broadened the scope of this KIP to the point where it sounded like it handles all sorts of exceptions. The scope should be strictly limited to "poison pill" records for now. Will update KIP, Thanks Eno > On 26 May 2017, at 16:16, Matthias

[jira] [Updated] (KAFKA-5301) Improve exception handling on consumer path

2017-05-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5301: Fix Version/s: (was: 0.11.0.0) 0.11.1.0 > Improve exception handling on consu

  1   2   >