[jira] [Reopened] (KAFKA-4579) Schema Registry client should have a proper caching mechanism for schema IDs

[ https://issues.apache.org/jira/browse/KAFKA-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Manchester reopened KAFKA-4579: -- Re-opening and chaning project. > Schema Registry client should have a proper caching mecha

[jira] [Resolved] (KAFKA-4579) Schema Registry client should have a proper caching mechanism for schema IDs

[ https://issues.apache.org/jira/browse/KAFKA-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Manchester resolved KAFKA-4579. -- Resolution: Invalid > Schema Registry client should have a proper caching mechanism for

[jira] [Issue Comment Deleted] (KAFKA-4579) Schema Registry client should have a proper caching mechanism for schema IDs

[ https://issues.apache.org/jira/browse/KAFKA-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Manchester updated KAFKA-4579: - Comment: was deleted (was: Re-opening and chaning project.) > Schema Registry client shou

[jira] [Assigned] (KAFKA-3739) Add no-arg constructor for library provided serdes

[ https://issues.apache.org/jira/browse/KAFKA-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi reassigned KAFKA-3739: --- Assignee: huxi > Add no-arg constructor for library provided serdes > ---

[GitHub] kafka pull request #2308: kafka-3739:Add no-arg constructor for library prov...

GitHub user amethystic opened a pull request: https://github.com/apache/kafka/pull/2308 kafka-3739:Add no-arg constructor for library provided serdes Add default constructor for library provided serdes You can merge this pull request into a Git repository by running: $ git pull

Jenkins build is back to normal : kafka-trunk-jdk8 #1145

See

[jira] [Commented] (KAFKA-4582) KStream job fails in multi-thread mode

[ https://issues.apache.org/jira/browse/KAFKA-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797674#comment-15797674 ] Manuel Romero commented on KAFKA-4582: -- Hi Matthias. Thanks for the quick response!

Re: [VOTE] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Closing this thread. On Tue, Jan 3, 2017 at 6:00 PM Ben Stopford wrote: > Hi All > > Happy New Year! > > Hopefully everyone has had an opportunity to review this KIP now, if they > wished to, so I'd like to call a vote. > > As a reminder the KIP proposes a change to the replication protocol to >

[VOTE] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Hi All Happy New Year! Hopefully everyone has had an opportunity to review this KIP now, if they wished to, so I'd like to call a vote. As a reminder the KIP proposes a change to the replication protocol to remove the potential for replicas to diverge. https://cwiki.apache.org/confluence/displa

[jira] [Updated] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-4583: -- Assignee: (was: Rajini Sivaram) > KafkaConsumerTest.testGracefulClose transient failure > --

[jira] [Assigned] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram reassigned KAFKA-4583: - Assignee: Rajini Sivaram > KafkaConsumerTest.testGracefulClose transient failure > --

[jira] [Assigned] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram reassigned KAFKA-4583: - Assignee: Rajini Sivaram > KafkaConsumerTest.testGracefulClose transient failure > --

[GitHub] kafka pull request #2307: kafka-4434: KafkaProducer configuration is logged ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2307 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4434) KafkaProducer configuration is logged twice

[ https://issues.apache.org/jira/browse/KAFKA-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi resolved KAFKA-4434. - Resolution: Fixed Reviewer: Ismael Juma > KafkaProducer configuration is logged twice >

[jira] [Updated] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4497: --- Labels: reliability (was: ) > log cleaner breaks on timeindex > --- > >

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797900#comment-15797900 ] Ismael Juma commented on KAFKA-4497: [~andyh1983], is the error exactly the same? Can

[jira] [Updated] (KAFKA-4557) ConcurrentModificationException in KafkaProducer event loop

[ https://issues.apache.org/jira/browse/KAFKA-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4557: --- Priority: Critical (was: Major) > ConcurrentModificationException in KafkaProducer event loop > -

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797906#comment-15797906 ] Ismael Juma commented on KAFKA-4477: Has anyone been able to test 0.10.1.1 to see if t

[jira] [Updated] (KAFKA-4576) Log segments close to max size break on fetch

[ https://issues.apache.org/jira/browse/KAFKA-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4576: --- Labels: reliability (was: ) > Log segments close to max size break on fetch > ---

[jira] [Commented] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797950#comment-15797950 ] ASF GitHub Bot commented on KAFKA-4583: --- GitHub user rajinisivaram opened a pull req

[GitHub] kafka pull request #2309: KAFKA-4583: Fix KafkaConsumerTest.testGracefulClos...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/2309 KAFKA-4583: Fix KafkaConsumerTest.testGracefulClose transient failure You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka

[jira] [Created] (KAFKA-4588) QueryableStateIntegrationTest.shouldNotMakeStoreAvailableUntilAllStoresAvailable is occasionally failing on jenkins

Damian Guy created KAFKA-4588: - Summary: QueryableStateIntegrationTest.shouldNotMakeStoreAvailableUntilAllStoresAvailable is occasionally failing on jenkins Key: KAFKA-4588 URL: https://issues.apache.org/jira/browse/

[jira] [Updated] (KAFKA-4588) QueryableStateIntegrationTest.shouldNotMakeStoreAvailableUntilAllStoresAvailable is occasionally failing on jenkins

[ https://issues.apache.org/jira/browse/KAFKA-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4588: -- Description: org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldNotMakeStoreAva

[GitHub] kafka pull request #2310: KAFKA-4497 - Ignore offset -1 in TimeIndex avoidin...

GitHub user ig-michaelpearce opened a pull request: https://github.com/apache/kafka/pull/2310 KAFKA-4497 - Ignore offset -1 in TimeIndex avoiding exception issue Ignore offset -1 in TimeIndex avoiding exception issue Killing compaction thread. As it seems issue remains, in

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797997#comment-15797997 ] ASF GitHub Bot commented on KAFKA-4497: --- GitHub user ig-michaelpearce opened a pull

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798006#comment-15798006 ] Michael Andre Pearce (IG) commented on KAFKA-4497: -- [~ijuma] it seems exa

[jira] [Issue Comment Deleted] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Andre Pearce (IG) updated KAFKA-4497: - Comment: was deleted (was: [~ijuma] it seems exactly the same, we note tha

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798007#comment-15798007 ] Michael Andre Pearce (IG) commented on KAFKA-4497: -- [~ijuma] it seems exa

[jira] [Created] (KAFKA-4589) Add documentation for SASL/SCRAM

Rajini Sivaram created KAFKA-4589: - Summary: Add documentation for SASL/SCRAM Key: KAFKA-4589 URL: https://issues.apache.org/jira/browse/KAFKA-4589 Project: Kafka Issue Type: Task C

[jira] [Created] (KAFKA-4590) Add system test for SASL/SCRAM

Rajini Sivaram created KAFKA-4590: - Summary: Add system test for SASL/SCRAM Key: KAFKA-4590 URL: https://issues.apache.org/jira/browse/KAFKA-4590 Project: Kafka Issue Type: Task Com

Re: [VOTE] KIP-99: Add Global Tables to Kafka Streams

Hi, The vote for KIP-99 has now concluded. The KIP has been accepted with: 5 binding votes (Gwen, Neha, Ewen, Sriram, Guozhang) and 2 non-binding (Matthias, Bill). Thanks everyone. On Tue, 3 Jan 2017 at 21:12 Sriram Subramanian wrote: > +1 > > On Tue, Jan 3, 2017 at 11:43 AM, Neha Narkhede w

[GitHub] kafka pull request #2309: KAFKA-4583: Fix KafkaConsumerTest.testGracefulClos...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2309 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798055#comment-15798055 ] ASF GitHub Bot commented on KAFKA-4583: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4583) KafkaConsumerTest.testGracefulClose transient failure

[ https://issues.apache.org/jira/browse/KAFKA-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4583. Resolution: Fixed Issue resolved by pull request 2309 [https://github.com/apache/kafka/pull/2309] >

[jira] [Reopened] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-4497: Reopening then. > log cleaner breaks on timeindex > --- > >

Re: KIP-103: Separation of Internal and External traffic

Hi Colin, Thanks for the feedback. It's a good question regarding the name `protocol label`. It was an easy starting name given that the security protocol was replaced by a label in the listener string. However, I agree that it's perhaps not as clear as it could be. Maybe `listener key` would be a

Build failed in Jenkins: kafka-trunk-jdk8 #1146

See Changes: [ismael] KAFKA-4434; KafkaProducer configuration is logged twice -- [...truncated 33052 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldReturnFalseOnCloseWhenThreadsHa

[jira] [Resolved] (KAFKA-4550) current trunk unstable

[ https://issues.apache.org/jira/browse/KAFKA-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4550. Resolution: Duplicate For the flush issue, there's https://issues.apache.org/jira/browse/KAFKA-3155

[jira] [Commented] (KAFKA-3155) Transient Failure in kafka.api.PlaintextProducerSendTest.testFlush

[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798157#comment-15798157 ] Ismael Juma commented on KAFKA-3155: Some logs in https://issues.apache.org/jira/brows

[GitHub] kafka pull request #2263: KAFKA-4508. Create system tests that run newer ver...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2263 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4508) Create system tests that run newer versions of the client against older versions of the broker

[ https://issues.apache.org/jira/browse/KAFKA-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798318#comment-15798318 ] ASF GitHub Bot commented on KAFKA-4508: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4508) Create system tests that run newer versions of the client against older versions of the broker

[ https://issues.apache.org/jira/browse/KAFKA-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4508. Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request 2263 [https://github

[jira] [Updated] (KAFKA-4508) Create system tests that run client against older versions of the broker

[ https://issues.apache.org/jira/browse/KAFKA-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4508: --- Summary: Create system tests that run client against older versions of the broker (was: Create system

Build failed in Jenkins: kafka-trunk-jdk8 #1147

See Changes: [ismael] KAFKA-4583; Fix KafkaConsumerTest.testGracefulClose transient failure -- [...truncated 17670 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldNotGetAllTasksWit

[jira] [Updated] (KAFKA-4582) KStream job fails in multi-thread mode

[ https://issues.apache.org/jira/browse/KAFKA-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manuel Romero updated KAFKA-4582: - Affects Version/s: 0.10.1.1 > KStream job fails in multi-thread mode > --

Re: [DISCUSS] KIP-107: Add purgeDataBefore() API in AdminClient

a major motivation for this KIP is cost savings. lots of internal systems at LI use kafka as an intermediate pipe, and set the topic retention period to a "safe enough" amount of time to be able to recover from crashes/downtime and catch up to "now". this results in a few days' worth of retention

Re: [DISCUSS] KIP-107: Add purgeDataBefore() API in AdminClient

in summary - i'm not opposed to the idea of a per-topic clean up config that tracks some set of consumer groups' offsets (which would probably work for 80% of use cases), but i definitely see a need to expose a simple API for the more advanced/obscure/custom use cases (the other 20%). On Wed, Jan

Re: 0.10.2.0 release planning

Thanks for running the release and being so transparent around plans, it's a great boon to the community. Looking at the plan ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.10.2.0), I see KIP-4 and KIP-35 under "Open Issues", but not under "Planned KIP Content". Are those KIPs

[jira] [Resolved] (KAFKA-3304) KIP-35 - Retrieving protocol version

[ https://issues.apache.org/jira/browse/KAFKA-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3304. Resolution: Fixed > KIP-35 - Retrieving protocol version > > >

[jira] [Commented] (KAFKA-4582) KStream job fails in multi-thread mode

[ https://issues.apache.org/jira/browse/KAFKA-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798610#comment-15798610 ] Manuel Romero commented on KAFKA-4582: -- Hi again. The error continues happening using

[jira] [Updated] (KAFKA-3308) Use ProtocolVersion during rolling upgrades to decide on a mutually agreed protocol version between brokers, eliminating the need of inter.broker.protocol.version.

[ https://issues.apache.org/jira/browse/KAFKA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3308: --- Issue Type: Improvement (was: Sub-task) Parent: (was: KAFKA-3304) > Use ProtocolVersion d

Re: [DISCUSS] KIP-107: Add purgeDataBefore() API in AdminClient

one more example of complicated config - mirror maker. we definitely cant trust each and every topic owner to configure their topics not to purge before they've been mirrored. which would mean there's a per-topic config (set by the owner) and a "global" config (where mirror makers are specified) a

Re: 0.10.2.0 release planning

Hi Tom, Thanks for the heads-up. Most of KIP-35 was included in 0.10.0.0. The remaining PR was merged to trunk (will be in 0.10.2.0), so I closed the JIRA (and changed KAFKA-3308 to be a standalone task as it wasn't part of KIP-35). About KIP-4, that's a big KIP and it will continue to be open af

Re: [DISCUSS] KIP-102 - Add close with timeout for consumers

Hi Jason, Thank you for the review. During close(), if there is a rebalance and the coordinator has to be rediscovered, close terminates without trying to find the coordinator. The poll() loop within close terminates if the coordinator is not known (as it does now) or if the timeout expires. At t

Re: 0.10.2.0 release planning

Ismael, Thanks. Does this mean there's partially implemented KIP-4 functionality landing in 0.10.2.0? It seems like having release notes for that functionality would be quite important. On Wed, Jan 4, 2017 at 4:16 PM, Ismael Juma wrote: > Hi Tom, > > Thanks for the heads-up. Most of KIP-35 was

Re: 0.10.2.0 release planning

0.10.1.0 was the release with partial KIP-4 functionality: we added create topics and delete topics to the Kafka protocol. 0.10.2.0 doesn't improve things so far. Ismael On Wed, Jan 4, 2017 at 4:28 PM, Tom Crayford wrote: > Ismael, > > Thanks. Does this mean there's partially implemented KIP-4

Jenkins build is back to normal : kafka-trunk-jdk8 #1148

See

[jira] [Commented] (KAFKA-4573) Producer sporadic timeout

[ https://issues.apache.org/jira/browse/KAFKA-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798700#comment-15798700 ] Ankur C commented on KAFKA-4573: Outgage lasted 5 hours and after we restarted broker the

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798702#comment-15798702 ] Jun Rao commented on KAFKA-4497: [~michael.andre.pearce], thanks for reporting this. It wo

[jira] [Created] (KAFKA-4591) Create Topic Policy

Ismael Juma created KAFKA-4591: -- Summary: Create Topic Policy Key: KAFKA-4591 URL: https://issues.apache.org/jira/browse/KAFKA-4591 Project: Kafka Issue Type: Improvement Reporter: I

[jira] [Commented] (KAFKA-4566) Can't Symlink to Kafka bins

[ https://issues.apache.org/jira/browse/KAFKA-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798750#comment-15798750 ] Colin P. McCabe commented on KAFKA-4566: The MacOS version of readlink does not ha

[jira] [Updated] (KAFKA-4507) The client should send older versions of requests to the broker if necessary

[ https://issues.apache.org/jira/browse/KAFKA-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4507: --- Status: Patch Available (was: In Progress) > The client should send older versions of request

Re: [VOTE] Vote for KIP-101 - Leader Epochs

Looks good to me! +1 (non-binding) On Wed, Jan 4, 2017 at 9:24 AM, Ben Stopford wrote: > Hi All > > We’re having some problems with this thread being subsumed by the > [Discuss] thread. Hopefully this one will appear distinct. If you see more > than one, please use this one. > > KIP-101 should

[VOTE] Vote for KIP-101 - Leader Epochs

Hi All We’re having some problems with this thread being subsumed by the [Discuss] thread. Hopefully this one will appear distinct. If you see more than one, please use this one. KIP-101 should now be ready for a vote. As a reminder the KIP proposes a change to the replication protocol to rem

Re: [VOTE] Vote for KIP-101 - Leader Epochs

+1 - thanks for tackling those old and painful bugs! On Wed, Jan 4, 2017 at 9:24 AM, Ben Stopford wrote: > Hi All > > We’re having some problems with this thread being subsumed by the [Discuss] > thread. Hopefully this one will appear distinct. If you see more than one, > please use this one. >

[jira] [Commented] (KAFKA-4582) KStream job fails in multi-thread mode

[ https://issues.apache.org/jira/browse/KAFKA-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798809#comment-15798809 ] Matthias J. Sax commented on KAFKA-4582: IIRC (and the JIRA also indicates it, as

Re: [VOTE] Vote for KIP-101 - Leader Epochs

+1 On Wed, Jan 4, 2017 at 5:28 PM, Gwen Shapira wrote: > +1 - thanks for tackling those old and painful bugs! > > On Wed, Jan 4, 2017 at 9:24 AM, Ben Stopford wrote: > > Hi All > > > > We’re having some problems with this thread being subsumed by the > [Discuss] thread. Hopefully this one will

Re: [DISCUSS] KIP-102 - Add close with timeout for consumers

Hi Rajini, Thanks for the clarification. I looked again at the patch and I see what you're saying now. I was confused because I assumed the request timeout was being enforced on the requests themselves, but it is more that the request timeout bounds the attempt to send them in addition to the time

Re: [DISCUSS] KIP-104: Granular Sensors for Streams

Thanks for the review, Guozhang, addressed comment 1, 3 on the KIP and left 2 for Eno to comment on. (He is back tomorrow) -aarti On Tue, Jan 3, 2017 at 11:12 AM, Guozhang Wang wrote: > Thanks for the proposed KIP. Overall LGTM. > > A few comments: > > 1. "at the granularity of each processor

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798847#comment-15798847 ] Michael Andre Pearce (IG) commented on KAFKA-4497: -- [~junrao] This is cau

[jira] [Comment Edited] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798847#comment-15798847 ] Michael Andre Pearce (IG) edited comment on KAFKA-4497 at 1/4/17 5:43 PM: --

[jira] [Commented] (KAFKA-4573) Producer sporadic timeout

[ https://issues.apache.org/jira/browse/KAFKA-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798862#comment-15798862 ] Jason Gustafson commented on KAFKA-4573: [~maverick2202] Do you know if the under-

Re: [DISCUSS] KIP-105: Addition of Record Level for Sensors

Thanks for the review, Guozhang, Addressed 2 out of the three comments, 1. Fixed and updated KIP (swapped code variable name METRICS_RECORD_LEVEL_CONFIG with config name metrics.record.level) 3. >>Could you elaborate on the "shouldRecord()" function, e.g. which class it will be added to? Does it

[jira] [Created] (KAFKA-4592) Kafka Producer Metrics Invalid Valu

AJ Jwair created KAFKA-4592: --- Summary: Kafka Producer Metrics Invalid Valu Key: KAFKA-4592 URL: https://issues.apache.org/jira/browse/KAFKA-4592 Project: Kafka Issue Type: Bug Components:

[jira] [Updated] (KAFKA-4592) Kafka Producer Metrics Invalid Value

[ https://issues.apache.org/jira/browse/KAFKA-4592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] AJ Jwair updated KAFKA-4592: Summary: Kafka Producer Metrics Invalid Value (was: Kafka Producer Metrics Invalid Valu) > Kafka Producer

[VOTE] KIP-66: Single Message Transforms for Kafka Connect

Hi all, I'd like to start voting on KIP-66: https://cwiki.apache.org/confluence/display/KAFKA/KIP-66%3A+Single+Message+Transforms+for+Kafka+Connect Best, Shikhar

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Thanks for the proposal. A few more detailed comments. 100. Security: It seems that if an app is mistakenly configured with the appId of an existing producer, it can take over the pid and prevent the existing app from publishing. So, I am wondering if it makes sense to add ACLs at the TransactionR

[jira] [Commented] (KAFKA-4436) Provide builder pattern for StreamsConfig

[ https://issues.apache.org/jira/browse/KAFKA-4436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799096#comment-15799096 ] Matthias J. Sax commented on KAFKA-4436: I guess we would need a KIP for this anyw

Re: [DISCUSS] KIP-102 - Add close with timeout for consumers

Hi Jason, Yes, we do potentially timeout even before sending pending commits after the request timeout (default is > 5 minutes, so this should only happen when there are real issues or when brokers are shutdown). I have updated the KIP to use a default timeout of 30 seconds for the existing close(

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

+1 On Wed, Jan 4, 2017 at 1:49 AM, Becket Qin wrote: > +1 > > On Tue, Jan 3, 2017 at 5:41 PM, Joel Koshy wrote: > > > +1 > > > > On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford wrote: > > > > > Hi All > > > > > > Please find the below KIP which proposes changing the setting > > > unclean.leader.

[GitHub] kafka pull request #2311: KAFKA-4523; Fix crash at shutdown due to the group...

GitHub user steveniemitz opened a pull request: https://github.com/apache/kafka/pull/2311 KAFKA-4523; Fix crash at shutdown due to the group coordinator attemp… …ting to write to a closed log. Solution: shut down the group manager before shutting down the log manager to

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799192#comment-15799192 ] Jun Rao commented on KAFKA-4497: [~michael.andre.pearce], sorry for the production issue t

[jira] [Commented] (KAFKA-4523) Controlled shutdown fails if consumer group restabilizes during shutdown

[ https://issues.apache.org/jira/browse/KAFKA-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799183#comment-15799183 ] ASF GitHub Bot commented on KAFKA-4523: --- GitHub user steveniemitz opened a pull requ

[jira] [Updated] (KAFKA-4523) Controlled shutdown fails if consumer group restabilizes during shutdown

[ https://issues.apache.org/jira/browse/KAFKA-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4523: --- Fix Version/s: 0.10.2.0 > Controlled shutdown fails if consumer group restabilizes during shutdown > -

[jira] [Updated] (KAFKA-4523) Controlled shutdown fails if consumer group restabilizes during shutdown

[ https://issues.apache.org/jira/browse/KAFKA-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4523: --- Status: Patch Available (was: Open) > Controlled shutdown fails if consumer group restabilizes during

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hey Jun, We had a proposal like this previously. The suppression scheme was slightly different. Rather than than attempting to recopy or swap, there was instead an aborted offset index maintained along with each segment containing a sequential list of aborted offsets. The filtering would happen at

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

I would have preferred not to bundle transformations, but since SMT capability is a much needed feature, I'll take it in its current form. +1 On Wed, Jan 4, 2017 at 10:47 AM, Shikhar Bhushan wrote: > Hi all, > > I'd like to start voting on KIP-66: > https://cwiki.apache.org/confluence/display/KA

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

+1 On Wed, Jan 4, 2017 at 1:29 PM, Gwen Shapira wrote: > I would have preferred not to bundle transformations, but since SMT > capability is a much needed feature, I'll take it in its current form. > > +1 > > On Wed, Jan 4, 2017 at 10:47 AM, Shikhar Bhushan > wrote: > > Hi all, > > > > I'd like

[jira] [Commented] (KAFKA-4558) throttling_test fails if the producer starts too fast.

[ https://issues.apache.org/jira/browse/KAFKA-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799448#comment-15799448 ] Apurva Mehta commented on KAFKA-4558: - That sounds like a good idea in general, to mak

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

+1 I agree with Gwen that this needs to be well communicated to users. Anyone who is using the current default may see their brokers start to go exit when they upgrade to this new version. On a somewhat related subject, occasionally I'll need to look up how my Kafka brokers are configured. Typ

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799490#comment-15799490 ] Michael Andre Pearce (IG) commented on KAFKA-4497: -- [~junrao]We had this

[jira] [Comment Edited] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799490#comment-15799490 ] Michael Andre Pearce (IG) edited comment on KAFKA-4497 at 1/4/17 10:09 PM: -

Re: KIP-103: Separation of Internal and External traffic

Hi, Ismael, Thanks for the proposal. Looks good to me overall. Just a couple of minor comments. 1. Sasl also has some properties provided through the login context in the jaas file. Do we want to extend that to allow different login context for different protocol labels on the server side (e.g. L

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

+1 Gwen, re: bundling transformations, would it help at all to isolate them to a separate jar or is the concern purely about maintaining them as part of Kafka? -Ewen On Wed, Jan 4, 2017 at 1:31 PM, Sriram Subramanian wrote: > +1 > > On Wed, Jan 4, 2017 at 1:29 PM, Gwen Shapira wrote: > > > I

[jira] [Commented] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799561#comment-15799561 ] Vahid Hashemian commented on KAFKA-4547: So here's what I've found so far. The in

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799565#comment-15799565 ] Jun Rao commented on KAFKA-4497: [~michael.andre.pearce], if you could attach the problema

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

The concern is maintaining a potentially unbounded list of add-ons as part of Kafka. I think the pros/cons were well discussed, I am happy we added a provision specifically excluding transformations that depend on specific data sources, and I'd much rather get SMT than continuing to debate how man

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

I do plan on introducing a new `connect:transforms` module (which `connect:runtime` will depend on), so they will live in a separate module in the source tree and output. ( https://github.com/apache/kafka/pull/2299 ) On Wed, Jan 4, 2017 at 2:28 PM Ewen Cheslack-Postava wrote: > +1 > > Gwen, re:

[jira] [Updated] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Andre Pearce (IG) updated KAFKA-4497: - Attachment: com_ig_trade_v1_order_event--demo--compacted-7.tar.bz2 [~junra

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799594#comment-15799594 ] Michael Andre Pearce (IG) commented on KAFKA-4497: -- [~junrao] just attach

  1   2   >