Jenkins build is back to normal : kafka-trunk-jdk8 #1941

2017-08-25 Thread Apache Jenkins Server
See

Re: [ANNOUNCE] New Kafka PMC member: Jiangjie (Becket) Qin

2017-08-25 Thread James Cheng
Congrats Becket! -James > On Aug 23, 2017, at 10:20 PM, Joel Koshy wrote: > > Hi everyone, > > Jiangjie (Becket) Qin has been a Kafka committer in October 2016 and has > contributed significantly to several major patches, reviews and discussions > since. I am glad to announce that Becket is no

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
Matthias, i agree so i've added those two overloads. Thanks, Damian On Thu, 24 Aug 2017 at 21:54 Matthias J. Sax wrote: > Thanks for clarification. I see your point. Java varargs are problematic > in general IMHO as they force you to put them as last argument making > parameter ordering unnatura

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
On Thu, 24 Aug 2017 at 18:31 Xavier Léauté wrote: > A few comments on the KIP: > - I'm a bit confused about the BytesStoreSupplier interface. Nothing in its > definition is really specific to Bytes, and > when I see return types like BytesStoreSupplier byte[]>>, it seems redundant to have "Bytes"

[jira] [Resolved] (KAFKA-5595) Illegal state in SocketServer; attempt to send with another send in progress

2017-08-25 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5595. Resolution: Fixed Assignee: Rajini Sivaram Fix Version/s: 1.0.0 > Illegal state in S

[jira] [Reopened] (KAFKA-5595) Illegal state in SocketServer; attempt to send with another send in progress

2017-08-25 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-5595: > Illegal state in SocketServer; attempt to send with another send in progress > ---

[jira] [Resolved] (KAFKA-5595) Illegal state in SocketServer; attempt to send with another send in progress

2017-08-25 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5595. Resolution: Fixed I'm going to close this. If we see the issue again, we can reopen it. > Illegal s

Build failed in Jenkins: kafka-trunk-jdk8 #1942

2017-08-25 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: KafkaService should print node hostname on failure -- [...truncated 2.46 MB...] org.apache.kafka.streams.integration.RegexSourceIntegrationTe

[jira] [Created] (KAFKA-5785) Always close connection if KafkaChannel.setSend throws exception

2017-08-25 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-5785: -- Summary: Always close connection if KafkaChannel.setSend throws exception Key: KAFKA-5785 URL: https://issues.apache.org/jira/browse/KAFKA-5785 Project: Kafka I

[jira] [Created] (KAFKA-5786) Yet another exception is causing that streamming app is zombie

2017-08-25 Thread Seweryn Habdank-Wojewodzki (JIRA)
Seweryn Habdank-Wojewodzki created KAFKA-5786: - Summary: Yet another exception is causing that streamming app is zombie Key: KAFKA-5786 URL: https://issues.apache.org/jira/browse/KAFKA-5786

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

2017-08-25 Thread Manikumar
+1 (non-binding) On Fri, Aug 25, 2017 at 12:37 AM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > +1 > > Thanks Rajini. > > --Vahid > > > > From: Edoardo Comar > To: dev@kafka.apache.org > Date: 08/24/2017 10:55 AM > Subject:Re: [VOTE] KIP-152 - Improve diagnostics for

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

2017-08-25 Thread Mickael Maison
Huge +1 ! Clients getting no feedback when failing SASL authentication is a major source of confusion. Thanks for the KIP On Fri, Aug 25, 2017 at 9:48 AM, Manikumar wrote: > +1 (non-binding) > > > On Fri, Aug 25, 2017 at 12:37 AM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > >> +1 >>

[GitHub] kafka pull request #3734: KAFKA-5785; Always close connection if KafkaChanne...

2017-08-25 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3734 KAFKA-5785; Always close connection if KafkaChannel.setSend throws exception You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-57

[jira] [Created] (KAFKA-5787) StoreChangeLogReader needs to restore partitions that were added post initialization

2017-08-25 Thread Damian Guy (JIRA)
Damian Guy created KAFKA-5787: - Summary: StoreChangeLogReader needs to restore partitions that were added post initialization Key: KAFKA-5787 URL: https://issues.apache.org/jira/browse/KAFKA-5787 Project:

[GitHub] kafka pull request #3735: KAFKA-4869: Update 0.10.2.0 upgrade notes

2017-08-25 Thread omkreddy
GitHub user omkreddy opened a pull request: https://github.com/apache/kafka/pull/3735 KAFKA-4869: Update 0.10.2.0 upgrade notes You can merge this pull request into a Git repository by running: $ git pull https://github.com/omkreddy/kafka cleanup10 Alternatively you can revie

[GitHub] kafka pull request #3736: KAFKA-5787: StoreChangelogReader needs to restore ...

2017-08-25 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3736 KAFKA-5787: StoreChangelogReader needs to restore partitions that were added post initialization If a task fails during initialization due to a LockException, its changelog partitions are not immediat

[GitHub] kafka pull request #3737: KAFKA-5771: org.apache.kafka.streams.state.interna...

2017-08-25 Thread radzish
GitHub user radzish opened a pull request: https://github.com/apache/kafka/pull/3737 KAFKA-5771: org.apache.kafka.streams.state.internals.Segments#segments method returns incorrect results when segments were added out of order Suggested fix for the bug You can merge this pull reque

[jira] [Created] (KAFKA-5788) "IllegalArgumentException: long is not a value type" when running ReassignPartitionsCommand

2017-08-25 Thread Ansel Zandegran (JIRA)
Ansel Zandegran created KAFKA-5788: -- Summary: "IllegalArgumentException: long is not a value type" when running ReassignPartitionsCommand Key: KAFKA-5788 URL: https://issues.apache.org/jira/browse/KAFKA-5788

[GitHub] kafka pull request #3738: MINOR: Remove try/finally from processNewResponses

2017-08-25 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3738 MINOR: Remove try/finally from processNewResponses The previous code would skip a response if an exception was thrown. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Created] (KAFKA-5789) Deleted topic is recreated when consumer subscribe the deleted one

2017-08-25 Thread Takafumi Saito (JIRA)
Takafumi Saito created KAFKA-5789: - Summary: Deleted topic is recreated when consumer subscribe the deleted one Key: KAFKA-5789 URL: https://issues.apache.org/jira/browse/KAFKA-5789 Project: Kafka

[GitHub] kafka pull request #3735: KAFKA-4869: Update 0.10.2.0 upgrade notes

2017-08-25 Thread omkreddy
Github user omkreddy closed the pull request at: https://github.com/apache/kafka/pull/3735 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] kafka pull request #3739: KAFKA-5789: Deleted topic is recreated when consum...

2017-08-25 Thread stakafum
GitHub user stakafum opened a pull request: https://github.com/apache/kafka/pull/3739 KAFKA-5789: Deleted topic is recreated when consumer subscribe the deleted one You can merge this pull request into a Git repository by running: $ git pull https://github.com/stakafum/kafka

[jira] [Resolved] (KAFKA-4869) 0.10.2.0 release notes incorrectly include KIP-115

2017-08-25 Thread Manikumar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-4869. -- Resolution: Fixed Assignee: Manikumar > 0.10.2.0 release notes incorrectly include KIP-115 > -

[GitHub] kafka pull request #3740: MINOR: reduce logging to trace in NetworkClient wh...

2017-08-25 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3740 MINOR: reduce logging to trace in NetworkClient when an old server API is being used logging in `NetworkClient#doSend´ at debug level is spamming the logs when you have a producer that is sending man

[GitHub] kafka pull request #3465: MINOR: Fixed the invalid link to Log Compaction

2017-08-25 Thread Kamal15
Github user Kamal15 closed the pull request at: https://github.com/apache/kafka/pull/3465 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[jira] [Resolved] (KAFKA-5771) org.apache.kafka.streams.state.internals.Segments#segments method returns incorrect results when segments were added out of order

2017-08-25 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-5771. --- Resolution: Fixed Fix Version/s: 0.11.0.1 1.0.0 Issue resolved by pull reque

[GitHub] kafka pull request #3737: KAFKA-5771: org.apache.kafka.streams.state.interna...

2017-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3737 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (KAFKA-5790) SocketServer.processNewResponses should not skip a response if exception is thrown

2017-08-25 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-5790: -- Summary: SocketServer.processNewResponses should not skip a response if exception is thrown Key: KAFKA-5790 URL: https://issues.apache.org/jira/browse/KAFKA-5790 Project:

Build failed in Jenkins: kafka-trunk-jdk7 #2675

2017-08-25 Thread Apache Jenkins Server
See Changes: [damian.guy] KAFKA-5771; org.apache.kafka.streams.state.internals.Segments#segments -- [...truncated 916.05 KB...] kafka.server.epoch.LeaderEpochFileCacheTe

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
Hi, I've just realised we need to add two methods to StateStoreBuilder or it isn't going to work: Map logConfig(); boolean loggingEnabled(); These are needed when we are building the topology and determining changelog topic names and configs. I've also update the KIP to add StreamBuil

Build failed in Jenkins: kafka-trunk-jdk7 #2676

2017-08-25 Thread Apache Jenkins Server
See -- [...truncated 915.80 KB...] kafka.server.epoch.LeaderEpochFileCacheTest > shouldNotResetEpochHistoryTailIfUndefinedPassed STARTED kafka.server.epoch.LeaderEpochFileCacheTest >

[GitHub] kafka pull request #3741: KAFKA-5762; LogContext used to capture the clientI...

2017-08-25 Thread Kamal15
GitHub user Kamal15 opened a pull request: https://github.com/apache/kafka/pull/3741 KAFKA-5762; LogContext used to capture the clientId implicitly in the… … logs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Kamal15/kafka

Jenkins build is back to normal : kafka-trunk-jdk8 #1943

2017-08-25 Thread Apache Jenkins Server
See

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Matthias J. Sax
Thanks Damian. Great KIP! +1 -Matthias On 8/25/17 6:45 AM, Damian Guy wrote: > Hi, > > I've just realised we need to add two methods to StateStoreBuilder or it > isn't going to work: > > Map logConfig(); > boolean loggingEnabled(); > > These are needed when we are building the topolo

Re: [DISCUSS] KIP-91 Provide Intuitive User Timeouts in The Producer

2017-08-25 Thread Jun Rao
Hi, Becket, Good point on expiring inflight requests. Perhaps we can expire an inflight request after min(remaining delivery.timeout.ms, request.timeout.ms). This way, if a user sets a high delivery.timeout.ms, we can still recover from broker power outage sooner. Thanks, Jun On Thu, Aug 24, 20

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

2017-08-25 Thread Jun Rao
Hi, Rajini, Thanks for the KIP. +1 Jun On Thu, Aug 24, 2017 at 10:29 AM, Rajini Sivaram wrote: > Hi all, > > I would like to start vote on KIP-152 to improve diagnostics of > authentication failures and to update clients to treat authentication > failures as fatal exceptions rather than transi

[jira] [Resolved] (KAFKA-1944) Rename LogCleaner and related classes to LogCompactor

2017-08-25 Thread Pranav Maniar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pranav Maniar resolved KAFKA-1944. -- Resolution: Won't Fix As discussed on Mailing List and KIP-184 discussion. This name changes wi

Re: [DISCUSS] KIP-184 Rename LogCleaner and related classes to LogCompactor

2017-08-25 Thread Pranav Maniar
Yes I can take it up deprecation of "log.cleaner.enable" Will it require KIP ? Since as per my understanding we will be honoring value set for "log.cleaner.enable" till the time it is around. For now just a warning message about deprecation will be logged only. Or should we remove the cofig now

[jira] [Resolved] (KAFKA-3826) Sampling on throughput / latency metrics recording in Streams

2017-08-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3826. -- Resolution: Fixed This is been done as part of KAFKA-5152. > Sampling on throughput / latency m

[GitHub] kafka pull request #3673: MINOR: Consolidate broker request/response handlin...

2017-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3673 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4787) KafkaStreams close() is not reentrant

2017-08-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4787. -- Resolution: Duplicate > KafkaStreams close() is not reentrant >

[GitHub] kafka pull request #3703: KAFKA-5755: KafkaProducer should be refactored to ...

2017-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3703 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Mayuresh Gharat
Hi Jason, Thanks a lot for the KIP and sorry for the delayed response. I had a few questions : - The KIP says that a user can have a class that extends KafkaPrincipal. Would this extended class be used when constructing the Session object in the SocketServer instead of constructing a n

[GitHub] kafka pull request #3742: [KAFKA-4380] Document the purpose of clean shutdow...

2017-08-25 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/kafka/pull/3742 [KAFKA-4380] Document the purpose of clean shutdown file in the code. Remove the previous TODO to remove the clean shutdown file with some of the discussion from https://github.com/apache/kafka/pull

[GitHub] kafka pull request #2104: [KAFKA-4380] Remove cleanshutdownfile

2017-08-25 Thread holdenk
Github user holdenk closed the pull request at: https://github.com/apache/kafka/pull/2104 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[jira] [Created] (KAFKA-5791) Add HTTPS support to the fault injector

2017-08-25 Thread Colin P. McCabe (JIRA)
Colin P. McCabe created KAFKA-5791: -- Summary: Add HTTPS support to the fault injector Key: KAFKA-5791 URL: https://issues.apache.org/jira/browse/KAFKA-5791 Project: Kafka Issue Type: Sub-tas

Jenkins build is back to normal : kafka-trunk-jdk7 #2677

2017-08-25 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk8 #1944

2017-08-25 Thread Apache Jenkins Server
See Changes: [jason] MINOR: Consolidate broker request/response handling -- [...truncated 2.46 MB...] org.apache.kafka.streams.kstream.internals.KTableImplTest > shouldN

[GitHub] kafka pull request #3734: KAFKA-5785; Always close connection if KafkaChanne...

2017-08-25 Thread ijuma
Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/3734 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] kafka pull request #3738: KAFKA-5790: SocketServer.processNewResponses shoul...

2017-08-25 Thread ijuma
Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/3738 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Jason Gustafson
Hey Mayuresh, Thanks for the comments. - The KIP says that a user can have a class that extends KafkaPrincipal. >Would this extended class be used when constructing the Session object > in >the SocketServer instead of constructing a new KafkaPrincipal? Yes, that's correct. We want to

[jira] [Resolved] (KAFKA-5776) Add the Trogdor fault injection daemon

2017-08-25 Thread Rajini Sivaram (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-5776. --- Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 3699 [https://git

[GitHub] kafka pull request #3699: Add the Trogdor fault injection daemon

2017-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3699 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #3743: KAFKA-5494: enable idempotence with max.in.flight....

2017-08-25 Thread apurvam
GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3743 KAFKA-5494: enable idempotence with max.in.flight.requests.per.connection > 1 Here we introduce client and broker changes to support multiple inflight requests while still guaranteeing idempotence.

[GitHub] kafka pull request #3728: Add group-id to the metrics tags

2017-08-25 Thread EtaCassiopeia
Github user EtaCassiopeia closed the pull request at: https://github.com/apache/kafka/pull/3728 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #3397: KAFKA-5413: Port fix to 0.10.2 branch

2017-08-25 Thread kelvinrutt
Github user kelvinrutt closed the pull request at: https://github.com/apache/kafka/pull/3397 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #3744: Add group-id to the metrics tags

2017-08-25 Thread EtaCassiopeia
GitHub user EtaCassiopeia opened a pull request: https://github.com/apache/kafka/pull/3744 Add group-id to the metrics tags It is better to have group-id in the JMX metrics. It would improve debuggability of systems: ```kafka_consumer_consumer_fetch_manager_metrics_test_0_recor

[jira] [Resolved] (KAFKA-5452) Aggressive log compaction ratio appears to have no negative effect on log-compacted topics

2017-08-25 Thread Jeff Chao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Chao resolved KAFKA-5452. -- Resolution: Resolved Following up after a long while. After talking offline with [~wushujames], the ori

Jenkins build is back to normal : kafka-trunk-jdk8 #1945

2017-08-25 Thread Apache Jenkins Server
See

[jira] [Created] (KAFKA-5792) Transient failure in KafkaAdminClientTest.testHandleTimeout

2017-08-25 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5792: --- Summary: Transient failure in KafkaAdminClientTest.testHandleTimeout Key: KAFKA-5792 URL: https://issues.apache.org/jira/browse/KAFKA-5792 Project: Kafka Issu

Build failed in Jenkins: kafka-trunk-jdk8 #1946

2017-08-25 Thread Apache Jenkins Server
See Changes: [rajinisivaram] KAFKA-5776; Add the Trogdor fault injection daemon -- [...truncated 2.02 MB...] org.apache.kafka.common.security.scram.ScramMessagesTest > i

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Mayuresh Gharat
Hi Jason, Thanks for the replies. I think it would be better to discuss with an example that we were trying to address with KIP-111 and see if the current mentioned solution would address it. Let's consider a third party library called authz_lib that is provided by some Security team at some co

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Jason Gustafson
Hi Mayuresh, To clarify, the intention is to use the KafkaPrincipal object built by the KafkaPrincipalBuilder inside the Session. So we would remove the logic to construct a new KafkaPrincipal using only the name from the Principal. Then it should be possible to pass the `AuthzPrincipal` to the un

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Mayuresh Gharat
Perfect. As long as there is a way we can access the originally created Principal in the Authorizer, it would solve the KIP-111 issue. This is really helpful, thanks again. Thanks, Mayuresh On Fri, Aug 25, 2017 at 3:13 PM, Jason Gustafson wrote: > Hi Mayuresh, > > To clarify, the intention is

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Jason Gustafson
No problem. I'll add a note to the KIP to emphasize that we will use the same object built by the KafkaPrincipalBuilder in the Session object passed to the authorizer. -Jason On Fri, Aug 25, 2017 at 3:17 PM, Mayuresh Gharat wrote: > Perfect. > As long as there is a way we can access the origina

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Don Bosco Durai
Jason Do you anticipate any backward compatibility issues with existing custom implementation of the authorization interface/plugins? Thanks Bosco On 8/25/17, 3:22 PM, "Jason Gustafson" wrote: No problem. I'll add a note to the KIP to emphasize that we will use the same object built

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Jason Gustafson
Hi Don, I don't think so. We are not making any changes to the Authorizer interface itself. The KafkaPrincipal object does not change either, though we now explicitly allow it to be extended. That means you have to exercise a little caution when combining a custom PrincipalBuilder with a custom Au

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Don Bosco Durai
Jason, thanks for confirming that. Since there are existing custom plugins, we might have to give enough time for them to start using the newer interface. I quickly glanced over the KIP, it looks good. Here is one comment: --- In the future, we may add support for groups to Kafka. This was b

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Jason Gustafson
Hey Don, That is not actually part of the KIP. It was a (somewhat pedantic) example used to illustrate how the kafka principal semantics could be applied to authorizers which understood group-level ACLs. The key point is this: although a principal is identified only by its type and name, the Kafka

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-25 Thread Don Bosco Durai
Jason, thanks for the clarification. Bosco On 8/25/17, 4:59 PM, "Jason Gustafson" wrote: Hey Don, That is not actually part of the KIP. It was a (somewhat pedantic) example used to illustrate how the kafka principal semantics could be applied to authorizers which understoo

Re: [DISCUSS] KIP-91 Provide Intuitive User Timeouts in The Producer

2017-08-25 Thread Becket Qin
Hi Jason, I see what you mean. That makes sense. So in the above case after the producer resets PID, when it retry batch_0_tp1, the batch will still have the old PID even if the producer has already got a new PID. @Jun, do you mean max(remaining delivery.timeout.ms, request.timeout.ms) instead of

[jira] [Created] (KAFKA-5793) Tighten up situations where OutOfOrderSequence may be returned

2017-08-25 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5793: --- Summary: Tighten up situations where OutOfOrderSequence may be returned Key: KAFKA-5793 URL: https://issues.apache.org/jira/browse/KAFKA-5793 Project: Kafka I

[jira] [Created] (KAFKA-5794) Introduce new idempotence mode to gracefully deal with topics on the older message format

2017-08-25 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5794: --- Summary: Introduce new idempotence mode to gracefully deal with topics on the older message format Key: KAFKA-5794 URL: https://issues.apache.org/jira/browse/KAFKA-5794

[jira] [Created] (KAFKA-5795) Make the idempotent producer the default producer setting

2017-08-25 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5795: --- Summary: Make the idempotent producer the default producer setting Key: KAFKA-5795 URL: https://issues.apache.org/jira/browse/KAFKA-5795 Project: Kafka Issue T

[jira] [Created] (KAFKA-5796) Understand performance implications of acks=all and potential ways to reduce it

2017-08-25 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5796: --- Summary: Understand performance implications of acks=all and potential ways to reduce it Key: KAFKA-5796 URL: https://issues.apache.org/jira/browse/KAFKA-5796 Project:

[jira] [Resolved] (KAFKA-5640) Look into making acks=all the default setting

2017-08-25 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta resolved KAFKA-5640. - Resolution: Duplicate This is a dup of https://issues.apache.org/jira/browse/KAFKA-5796 > Look in