Build failed in Jenkins: kafka-trunk-jdk7 #1867

See Changes: [wangguoz] KAFKA-4671: Fix Streams window retention policy [jason] MINOR: Add offset information to consumer_test error messages -- [...truncated 19456 lines...] org.apache.kafka.s

Re: [DISCUSS] KIP-111 : Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Rajini, Thanks a lot for the review. Please see the comments inline : It feels like the goal is to expose custom Principal as an opaque object between PrincipalBuilder and Authorizer so that Kafka doesn't really need to know anything about additional stuff added for customization. But kafka-ac

[GitHub] kafka pull request #2428: HOTIFX: streams system test do not start up correc...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2428 HOTIFX: streams system test do not start up correctly You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfixSystemTests Alternatively

Re: [VOTE] KIP-81: Bound Fetch memory usage in the consumer

Good point. The consumer does use a separate connection to the coordinator, so perhaps the connection itself could be tagged for normal heap allocation? -Jason On Mon, Jan 23, 2017 at 10:26 AM, Onur Karaman wrote: > I only did a quick scan but I wanted to point out what I think is an > incorrec

[GitHub] kafka pull request #2426: MINOR: Add offset information to consumer_test err...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2426 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2401: KAFKA-4671: Fix Streams window retention policy

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2401 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-107 Add purgeDataBefore() API in AdminClient

Hi all, When I am implementing the patch, I realized that the current usage of "low_watermark" is a bit confusing. So I made the following interface changes in the KIP: - The newly added checkpoint file will be named log-begin-offset-checkpoint - Replace low_watermark with log_begin_offset in Fet

[GitHub] kafka pull request #2427: KAFKA-4673: Fix thread-safety of Python Verifiable...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/2427 KAFKA-4673: Fix thread-safety of Python VerifiableConsumer class You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka kafka-4673-verifia

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hi Roger, SCRAM uses the PBKDF2 mechanism, here's a comparison between PBKDF2 and bcrypt: http://security.stackexchange.com/questions/4781/do-any-secu rity-experts-recommend-bcrypt-for-password-storage/6415#6415 It may be worth supporting bcrypt, but not sure it would make sense to do it in the

[GitHub] kafka pull request #2426: MINOR: Add offset information to consumer_test err...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2426 MINOR: Add offset information to consumer_test error messages You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka improve-consume

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Sorry for the late question but is there a reason to choose SHA-1 and SHA-256 instead of bcrypt? https://codahale.com/how-to-safely-store-a-password/ On Fri, Nov 11, 2016 at 5:30 AM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > I think all the comments and suggestions on this thread h

[GitHub] kafka pull request #2425: KAFKA-4687: Fix InvalidTopicException due to topic...

GitHub user noslowerdna opened a pull request: https://github.com/apache/kafka/pull/2425 KAFKA-4687: Fix InvalidTopicException due to topic creation race condition Pull request for https://issues.apache.org/jira/browse/KAFKA-4687, complete details can be found in the Jira.

Re: [VOTE] KIP-74: Add FetchResponse size limit in bytes

Thanks, all. I edited the wiki to reflect the implemented behavior by dropping references to special behavior when max_bytes was INT_MAX. cheers, Colin On Sat, Jan 21, 2017, at 09:44, radai wrote: > +1 > > On Fri, Jan 20, 2017 at 9:51 PM, Apurva Mehta > wrote: > > > +1 > > > > On Fri, Jan 20

Build failed in Jenkins: kafka-trunk-jdk8 #1205

See Changes: [wangguoz] KAFKA-4060 Follow-up: update docs accordingly [jason] KAFKA-4633; Always using regex pattern subscription in StreamThread -- [...truncated 8401 lines...] kafka.integrat

Jenkins build is back to normal : kafka-trunk-jdk7 #1866

See

[GitHub] kafka pull request #2424: KAFKA-4688: The Docker image should contain versio...

GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/2424 KAFKA-4688: The Docker image should contain version 0.10.1.0 of Kafka You can merge this pull request into a Git repository by running: $ git pull https://github.com/cmccabe/kafka KAFKA-4688

Re: Request to provide rights to assign JIRA tickets to myself

Done. You should be able to assign yourself the JIRA now. -Jason On Mon, Jan 23, 2017 at 2:27 AM, Akhilesh Naidu < akhilesh_na...@persistent.com> wrote: > Hi, > > > I would be interested to contribute to the Kafka project, > > and was presently looking into the below issue > > KAFKA-4566 > Can't

Build failed in Jenkins: kafka-trunk-jdk7 #1865

See Changes: [wangguoz] HOTFIX: KAFKA-4060 and KAFKA-4476 follow up [wangguoz] KAFKA-4060 Follow-up: update docs accordingly -- [...truncated 8392 lines...] kafka.log.LogTest > testReadWithToo

Build failed in Jenkins: kafka-trunk-jdk8 #1204

See Changes: [ismael] MINOR: Improve scaladoc for Authorizer [wangguoz] HOTFIX: KAFKA-4060 and KAFKA-4476 follow up -- [...truncated 28498 lines...] org.apache.kafka.common.security.scram.Scram

Re: [VOTE] KIP-107 Add purgeDataBefore() API in AdminClient

Thanks for the comment Jun. Yeah, I think there is use-case where this can be useful. Allowing for asynchronous delete will be useful if an application doesn't need strong guarantee of purgeDataFrom(), e.g. if it is done to help reduce disk usage of kafka. The application may want to purge data fo

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

Thanks. Please see my comment inline. On Mon, Jan 23, 2017 at 6:45 AM, Alexey Ozeritsky wrote: > > > 13.01.2017, 22:29, "Dong Lin" : > > Hey Alexey, > > > > Thanks for your review and the alternative approach. Here is my > > understanding of your patch. kafka's background threads are used to mov

[GitHub] kafka pull request #2379: KAFKA-4633: Always using regex pattern subscriptio...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2379 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-107 Add purgeDataBefore() API in AdminClient

I feel that it's simpler to just keep the format of the checkpoint file as it is and just add a separate checkpoint for low watermark. Low watermark and high watermark are maintained independently. So, not sure if there is significant benefit of storing them together. Looking at the KIP again. I a

Re: [VOTE] KIP-81: Bound Fetch memory usage in the consumer

I only did a quick scan but I wanted to point out what I think is an incorrect assumption in the KIP's caveats: " There is a risk using the MemoryPool that, after we fill up the memory with fetch data, we can starve the coordinator's connection ... To alleviate this issue, only messages larger than

[GitHub] kafka pull request #2377: Kafka 4060 docs update

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2377 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk8 #1203

See Changes: [ismael] MINOR: Pass RecordingLevel to MetricConfig in the broker -- [...truncated 20927 lines...] org.apache.kafka.connect.runtime.WorkerSinkTaskThreadedTest > testCommitTimeout P

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

I understand your argument, but do not agree with it. Your first version (even if the "flow" is not as nice) is more explicit than the second version. Adding a stateStoreName parameter is quite implicit but has a huge impact -- thus, I prefer the rather more verbose but explicit version. -Matthi

Re: [VOTE] KIP-81: Bound Fetch memory usage in the consumer

I've updated the KIP to address all the comments raised here and from the "DISCUSS" thread. See: https://cwiki.apache.org/confluence/display/KAFKA/KIP-81%3A+Bound+Fetch+memory+usage+in+the+consumer Now, I'd like to restart the vote. On Tue, Dec 6, 2016 at 9:02 AM, Rajini Sivaram wrote: > Hi Mic

[GitHub] kafka pull request #2375: Kafka 4060 remove zk client dependency in kafka st...

Github user hjafarpour closed the pull request at: https://github.com/apache/kafka/pull/2375 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2418: HOTFIX: KAFKA-4060 and KAFKA-4476 follow up

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2418 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-111 : Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Mayuresh, Thanks for updating the KIP. A couple of questions: 1. PrincipalBuilder implements Configurable and gets a map of properties via the `configure` method. Do we really need a new `buildPrincipal` method given that? 2. Jun suggested in the JIRA that it may make sense to pass the `chann

SASL for ZK\Kafka

Hi I was trying to secure communication between ZK and Kafka. We generate the keytab file with principal We were following this document - https://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption/ (really detailed doc) For Kafka - kafka/xx--xx.x

[GitHub] kafka pull request #2151: Feature/authorizer name reference

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2350: MINOR: Finished exposing the broker config

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-1379) Partition reassignment resets clock for time-based retention

[ https://issues.apache.org/jira/browse/KAFKA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated KAFKA-1379: Component/s: log > Partition reassignment resets clock for time-based retention > --

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

13.01.2017, 22:29, "Dong Lin" : > Hey Alexey, > > Thanks for your review and the alternative approach. Here is my > understanding of your patch. kafka's background threads are used to move > data between replicas. When data movement is triggered, the log will be > rolled and the new logs will be

Re: [DISCUSS] KIP-111 : Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Mayuresh, The part about exposing Principal from custom PrincipalBuilder to custom Authorizer sounds good. The definition of ACLs using kafka-acls.sh is less clear to me. It feels like the goal is to expose custom Principal as an opaque object between PrincipalBuilder and Authorizer so that Kaf

Re: KafkaException: Message payload is null shutting down broker

Hi Ismael, thank you very much. I create the issue: https://issues.apache.org/jira/browse/KAFKA-4686. 2017-01-23 11:46 GMT-02:00 Ismael Juma : > Hi Rodrigo, > > Please file a JIRA so that this can be investigated. > > Ismael > > On Mon, Jan 23, 2017 at 1:32 PM, Rodrigo Queiroz Saramago < > rod

[jira] [Created] (KAFKA-4686) Null Message payload is shutting down broker

Rodrigo Queiroz Saramago created KAFKA-4686: --- Summary: Null Message payload is shutting down broker Key: KAFKA-4686 URL: https://issues.apache.org/jira/browse/KAFKA-4686 Project: Kafka

Re: KafkaException: Message payload is null shutting down broker

Hi Rodrigo, Please file a JIRA so that this can be investigated. Ismael On Mon, Jan 23, 2017 at 1:32 PM, Rodrigo Queiroz Saramago < rodrigo.saram...@zup.com.br> wrote: > Hello, I have a test environment with 3 brokers and 1 zookeeper nodes, in > which clients connect using two-way ssl authentic

KafkaException: Message payload is null shutting down broker

Hello, I have a test environment with 3 brokers and 1 zookeeper nodes, in which clients connect using two-way ssl authentication. I use kafka version 0.10.1.1, the system works as expected for a while, but if the node goes down and then is restarted, something got corrupted and is not possible star

[jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos

[ https://issues.apache.org/jira/browse/KAFKA-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834464#comment-15834464 ] postmas...@inn.ru commented on KAFKA-1207: -- Delivery is delayed to these recipien

[jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos

[ https://issues.apache.org/jira/browse/KAFKA-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834463#comment-15834463 ] postmas...@inn.ru commented on KAFKA-1207: -- Delivery is delayed to these recipien

[jira] [Commented] (KAFKA-4679) Remove unstable markers from Connect APIs

[ https://issues.apache.org/jira/browse/KAFKA-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834456#comment-15834456 ] ASF GitHub Bot commented on KAFKA-4679: --- GitHub user baluchicken opened a pull reque

[GitHub] kafka pull request #2423: KAFKA-4679 Remove unstable markers from Connect AP...

GitHub user baluchicken opened a pull request: https://github.com/apache/kafka/pull/2423 KAFKA-4679 Remove unstable markers from Connect APIs @ewencp ignore this PR if you are already started to work on this ticket. You can merge this pull request into a Git repository by running:

[jira] [Commented] (KAFKA-4679) Remove unstable markers from Connect APIs

[ https://issues.apache.org/jira/browse/KAFKA-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834440#comment-15834440 ] ASF GitHub Bot commented on KAFKA-4679: --- Github user baluchicken closed the pull req

[GitHub] kafka pull request #2422: KAFKA-4679 Remove unstable markers from Connect AP...

Github user baluchicken closed the pull request at: https://github.com/apache/kafka/pull/2422 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4679) Remove unstable markers from Connect APIs

[ https://issues.apache.org/jira/browse/KAFKA-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834437#comment-15834437 ] ASF GitHub Bot commented on KAFKA-4679: --- GitHub user baluchicken opened a pull reque

[GitHub] kafka pull request #2422: KAFKA-4679 Remove unstable markers from Connect AP...

GitHub user baluchicken opened a pull request: https://github.com/apache/kafka/pull/2422 KAFKA-4679 Remove unstable markers from Connect APIs @ewencp ignore this PR if you are already started to work on this ticket. You can merge this pull request into a Git repository by running:

[GitHub] kafka pull request #2421: HOTFIX: Fix cmd line argument order in streams sys...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2421 HOTFIX: Fix cmd line argument order in streams system tests Fix the ordering of cmd line arguments passed to the system tests. You can merge this pull request into a Git repository by running: $ g

[jira] [Commented] (KAFKA-4679) Remove unstable markers from Connect APIs

[ https://issues.apache.org/jira/browse/KAFKA-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834402#comment-15834402 ] Balint Molnar commented on KAFKA-4679: -- [~ewencp] can I help with this? > Remove uns

Re: [VOTE] KIP-108: Create Topic Policy

Hi all, During PR review a few minor improvements to the KIP were suggested: 1. Rename TopicDetails to RequestMetadata and make it an inner class of the CreateTopicPolicy interface. By using a more generic name, we have more flexibility with regards to additional data we may want to pass to the p

[jira] [Commented] (KAFKA-4677) Avoid unnecessary task movement across threads of the same process during rebalance

[ https://issues.apache.org/jira/browse/KAFKA-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834239#comment-15834239 ] ASF GitHub Bot commented on KAFKA-4677: --- Github user dguy closed the pull request at

[GitHub] kafka pull request #2411: [WIP] KAFKA-4677: Avoid unnecessary task movement ...

Github user dguy closed the pull request at: https://github.com/apache/kafka/pull/2411 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

Request to provide rights to assign JIRA tickets to myself

Hi, I would be interested to contribute to the Kafka project, and was presently looking into the below issue KAFKA-4566 Can't Symlink to Kafka bins Could some one please provide the rights to assign JIRA tickets to myself. Regards Akhilesh DISCLAIMER == This e-mail may contain

[jira] [Commented] (KAFKA-4685) All partitions offline, no conroller znode in ZK

[ https://issues.apache.org/jira/browse/KAFKA-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834209#comment-15834209 ] Sinóros-Szabó Péter commented on KAFKA-4685: Yes. I was able to check ZK only

[jira] [Commented] (KAFKA-3450) Producer blocks on send to topic that doesn't exist if auto create is disabled

[ https://issues.apache.org/jira/browse/KAFKA-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834170#comment-15834170 ] Michal Turek commented on KAFKA-3450: - Hi Esoga, I don't think this issue is duplicati

[jira] [Commented] (KAFKA-4685) All partitions offline, no conroller znode in ZK

[ https://issues.apache.org/jira/browse/KAFKA-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834169#comment-15834169 ] huxi commented on KAFKA-4685: - Server logs showed that all the brokers have come back to regis

[jira] [Assigned] (KAFKA-2363) ProducerSendTest.testCloseWithZeroTimeoutFromCallerThread Transient Failure

[ https://issues.apache.org/jira/browse/KAFKA-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford reassigned KAFKA-2363: --- Assignee: (was: Ben Stopford) > ProducerSendTest.testCloseWithZeroTimeoutFromCallerThread

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

I'm not a fan of materialize. I think it interrupts the flow, i.e, table.mapValue(..).materialize().join(..).materialize() compared to: table.mapValues(..).join(..) I know which one i prefer. My preference is stil to provide overloaded methods where people can specify the store names if they want

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834117#comment-15834117 ] Ismael Juma commented on KAFKA-4669: The reason why there's no try/catch in `handleCom

[jira] [Created] (KAFKA-4685) All partitions offline, no conroller znode in ZK

Sinóros-Szabó Péter created KAFKA-4685: -- Summary: All partitions offline, no conroller znode in ZK Key: KAFKA-4685 URL: https://issues.apache.org/jira/browse/KAFKA-4685 Project: Kafka Is

[jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos

[ https://issues.apache.org/jira/browse/KAFKA-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834096#comment-15834096 ] postmas...@inn.ru commented on KAFKA-1207: -- Delivery is delayed to these recipien

[jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos

[ https://issues.apache.org/jira/browse/KAFKA-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834090#comment-15834090 ] postmas...@inn.ru commented on KAFKA-1207: -- Delivery is delayed to these recipien

[jira] [Updated] (KAFKA-4684) Kafka does not offer kafka-configs.bat on Windows box

[ https://issues.apache.org/jira/browse/KAFKA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi updated KAFKA-4684: Labels: newbie (was: ) > Kafka does not offer kafka-configs.bat on Windows box > --

[jira] [Comment Edited] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834053#comment-15834053 ] huxi edited comment on KAFKA-4669 at 1/23/17 8:25 AM: -- Maybe we could

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834053#comment-15834053 ] huxi commented on KAFKA-4669: - Maybe we could invoke an alternative method `ProduceRequestResu