[jira] [Commented] (KAFKA-5278) kafka-console-consumer: `--value-deserializer` is not working but `--property value.deserializer` does

[ https://issues.apache.org/jira/browse/KAFKA-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028800#comment-16028800 ] ASF GitHub Bot commented on KAFKA-5278: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-5278) kafka-console-consumer: `--value-deserializer` is not working but `--property value.deserializer` does

[ https://issues.apache.org/jira/browse/KAFKA-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-5278. -- Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 3100 [https://gi

[GitHub] kafka pull request #3100: KAFKA-5278: ConsoleConsumer should honor `--value-...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Jenkins build is back to normal : kafka-trunk-jdk7 #2283

See

[jira] [Resolved] (KAFKA-3151) kafka-consumer-groups.sh fail with sasl enabled

[ https://issues.apache.org/jira/browse/KAFKA-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balint Molnar resolved KAFKA-3151. -- Resolution: Not A Problem > kafka-consumer-groups.sh fail with sasl enabled > -

Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

+1 It's a mystery how this didn't already exist as it's one of the key cluster's health indicator On Mon, May 29, 2017 at 9:18 AM, Gwen Shapira wrote: > Hi, > > Sounds good. I was sure this existed already for some reason :) > > On Sun, May 28, 2017 at 11:06 AM Dong Lin wrote: > >> Hi, >> >> We

[jira] [Commented] (KAFKA-5290) docs need clarification on meaning of 'committed' to the log

[ https://issues.apache.org/jira/browse/KAFKA-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029248#comment-16029248 ] Edoardo Comar commented on KAFKA-5290: -- https://github.com/apache/kafka/pull/3035 >

Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

Hi Dong, many thanks for the KIP. It's a very useful metric. by saying > Unavailable partitions could be most easily defined as “The number of partitions that this broker leads for which the ISR is insufficient to meet the minimum ISR required.” I presume you meant to call 'Unavailable' the p

Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

What about simply calling them 'BelowIsrPartitionCount' and 'BelowIsr' ? On Tue, May 30, 2017 at 11:40 AM, Edoardo Comar wrote: > Hi Dong, > > many thanks for the KIP. It's a very useful metric. > > by saying >> Unavailable partitions could be most easily defined as “The number of > partitions th

[GitHub] kafka pull request #3079: MINOR: Removed the redundant code snippets.

Github user Kamal15 closed the pull request at: https://github.com/apache/kafka/pull/3079 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] kafka pull request #3166: KAFKA-5316: Follow-up with ByteBufferOutputStream ...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3166 KAFKA-5316: Follow-up with ByteBufferOutputStream and other misc improvements ByteBufferOutputStream improvements: * Document pitfalls * Improve efficiency when dealing with direct byte buffer

[jira] [Commented] (KAFKA-5316) Log cleaning can increase message size and cause cleaner to crash with buffer overflow

[ https://issues.apache.org/jira/browse/KAFKA-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029331#comment-16029331 ] ASF GitHub Bot commented on KAFKA-5316: --- GitHub user ijuma opened a pull request:

Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

Thanks for the KIP, Dong. I agree that that the metrics are useful. Like Edoardo and Mickael said, it seems like it may be better to choose a different name. A couple of additional suggestions: `UnderMinIsrPartitionCount` and `UnderMinIsr`. Ismael On Tue, May 30, 2017 at 12:04 PM, Mickael Maison

[GitHub] kafka pull request #3167: MINOR: Remove unused configuration setting

GitHub user reftel opened a pull request: https://github.com/apache/kafka/pull/3167 MINOR: Remove unused configuration setting The configuration setting zookeeper.sync.time.ms was not used for anything. Remove it to reduce confusion. This changes rebalance.backoff.ms fro

[GitHub] kafka pull request #3168: KAFKA-4956: System test for request quotas

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/3168 KAFKA-4956: System test for request quotas You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka KAFKA-4956 Alternatively y

[jira] [Commented] (KAFKA-4956) Add system tests for request quotas

[ https://issues.apache.org/jira/browse/KAFKA-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029461#comment-16029461 ] ASF GitHub Bot commented on KAFKA-4956: --- GitHub user rajinisivaram opened a pull req

[jira] [Resolved] (KAFKA-4907) compacted topic shouldn't reject messages with old timestamp

[ https://issues.apache.org/jira/browse/KAFKA-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4907. Resolution: Won't Fix We decided to change the default of `message.timestamp.difference.max.ms` back

Kafka Monitors

Hello, I need your help. We use Kafka in our organization, and I would like to create a perl script which will run monitoring metrics. How can I run the metrics on Linux so I would see it`s output ?

[GitHub] kafka pull request #3169: MINOR: Fix doc for producer throttle time metrics

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/3169 MINOR: Fix doc for producer throttle time metrics You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka MINOR-producer-metri

[jira] [Created] (KAFKA-5346) Kafka Producer Failure After Idle Time

Manikandan P created KAFKA-5346: --- Summary: Kafka Producer Failure After Idle Time Key: KAFKA-5346 URL: https://issues.apache.org/jira/browse/KAFKA-5346 Project: Kafka Issue Type: Bug E

[jira] [Updated] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4340: --- Fix Version/s: (was: 0.11.0.0) > Change the default value of log.message.timestamp.difference.max.

[jira] [Updated] (KAFKA-5344) Change message.timestamp.difference.max.ms back to Long.MaxValue

[ https://issues.apache.org/jira/browse/KAFKA-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5344: --- Description: Per discussion in KAFKA-4340 and PR-2705 (https://github.com/apache/kafka/pull/2705), we

[jira] [Resolved] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4340. Resolution: Won't Fix Compacted topics means that we would also need KAFKA-4907 to avoid rejecting

[jira] [Updated] (KAFKA-5344) Change message.timestamp.difference.max.ms back to Long.MaxValue

[ https://issues.apache.org/jira/browse/KAFKA-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5344: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 3163

[GitHub] kafka pull request #3163: KAFKA-5344: set message.timestamp.difference.max.m...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3163 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-5344) Change message.timestamp.difference.max.ms back to Long.MaxValue

[ https://issues.apache.org/jira/browse/KAFKA-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029513#comment-16029513 ] ASF GitHub Bot commented on KAFKA-5344: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029555#comment-16029555 ] Tiberian Sun commented on KAFKA-1194: - Hello, Is this being resolved? I have been expe

KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

Hi, I started a new KIP to improve the minimum required ACL permissions of some of the APIs: https://cwiki.apache.org/confluence/display/KAFKA/KIP-163%3A+Lower+the+Minimum+Required+ACL+Permission+of+OffsetFetch The KIP is to address KAFKA-4585. Feedback and suggestions are welcome! Thanks. --V

Kafka Time Based Index - Server Property ?

Hi All , I've started exploring SearchMessagesByTimestamp https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index#KIP-33-Addatimebasedlogindex-Searchmessagebytimestamp . Kafka Producer produces the record with timestamp . When i try to search Timestamps few cases i

Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

Thanks Edoardo and everyone for the comment! That is a very good point. I have updated to wiki to use UnderMinIsrPartitionCount as the per-broker metric name and UnderMinIsr as the per-partition metric name. The motivation section has also been updated to clarify how the existence of UnderMinIsrPar

[jira] [Commented] (KAFKA-5295) Allow Kafka Connect source connectors to specify topic-specific settings for new topics

[ https://issues.apache.org/jira/browse/KAFKA-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029625#comment-16029625 ] Randall Hauch commented on KAFKA-5295: -- Created [KIP-158|https://cwiki.apache.org/co

[jira] [Updated] (KAFKA-5341) Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

[ https://issues.apache.org/jira/browse/KAFKA-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-5341: Description: We currently have under replicated partitions, but we do not have a metric to track the number

[jira] [Updated] (KAFKA-5341) Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

[ https://issues.apache.org/jira/browse/KAFKA-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-5341: Summary: Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics (was: Add unavailablePartition

[jira] [Assigned] (KAFKA-5342) Distinguish abortable failures in transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-5342: -- Assignee: Jason Gustafson > Distinguish abortable failures in transactional producer >

[GitHub] kafka pull request #3170: KAFKA-5054: ChangeLoggingKeyValueByteStore delete ...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3170 KAFKA-5054: ChangeLoggingKeyValueByteStore delete and putIfAbsent should be synchronized These involve multiple operations on the underlying store. The underlying store access is synchronized, but the

[jira] [Commented] (KAFKA-5054) ChangeLoggingKeyValueByteStore delete and putIfAbsent should be synchronized

[ https://issues.apache.org/jira/browse/KAFKA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029777#comment-16029777 ] ASF GitHub Bot commented on KAFKA-5054: --- GitHub user dguy opened a pull request:

Jenkins build is back to normal : kafka-trunk-jdk8 #1610

See

Re: [DISCUSS] KIP-148: Add a connect timeout for client

On Mon, May 29, 2017, at 15:46, Guozhang Wang wrote: > On Wed, May 24, 2017 at 9:59 AM, Colin McCabe wrote: > > > On Tue, May 23, 2017, at 19:07, Guozhang Wang wrote: > > > I think using a single config to cover end-to-end latency with connecting > > > and request round-trip may not be best appro

[jira] [Created] (KAFKA-5347) OutOfSequence error should be fatal

Jason Gustafson created KAFKA-5347: -- Summary: OutOfSequence error should be fatal Key: KAFKA-5347 URL: https://issues.apache.org/jira/browse/KAFKA-5347 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-5347) OutOfSequence error should be fatal

[ https://issues.apache.org/jira/browse/KAFKA-5347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-5347: --- Issue Type: Sub-task (was: Bug) Parent: KAFKA-4815 > OutOfSequence error should be fa

[jira] [Created] (KAFKA-5348) kafka-consumer-groups.sh refuses to remove groups without ids

Ivan Babrou created KAFKA-5348: -- Summary: kafka-consumer-groups.sh refuses to remove groups without ids Key: KAFKA-5348 URL: https://issues.apache.org/jira/browse/KAFKA-5348 Project: Kafka Issu

[jira] [Assigned] (KAFKA-5348) kafka-consumer-groups.sh refuses to remove groups without ids

[ https://issues.apache.org/jira/browse/KAFKA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-5348: -- Assignee: Vahid Hashemian > kafka-consumer-groups.sh refuses to remove groups without i

[GitHub] kafka pull request #3166: KAFKA-5316: Follow-up with ByteBufferOutputStream ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3166 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-5316) Log cleaning can increase message size and cause cleaner to crash with buffer overflow

[ https://issues.apache.org/jira/browse/KAFKA-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030029#comment-16030029 ] ASF GitHub Bot commented on KAFKA-5316: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-5347) OutOfSequence error should be fatal

[ https://issues.apache.org/jira/browse/KAFKA-5347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta updated KAFKA-5347: Labels: exactly-once (was: ) > OutOfSequence error should be fatal > --

Re: Reg: [VOTE] KIP 157 - Add consumer config options to streams reset tool

+1. Thanks! On Tue, May 16, 2017 at 1:12 AM, Eno Thereska wrote: > +1 thanks. > > Eno > > On 16 May 2017, at 04:20, BigData dev wrote: > > > > Hi All, > > Given the simple and non-controversial nature of the KIP, I would like to > > start the voting process for KIP-157: Add consumer config opti

[GitHub] kafka-site pull request #57: Updated description for uber card on the powere...

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/57 Updated description for uber card on the powered by page Swapped in new description for the Uber "Powered By" card You can merge this pull request into a Git repository by running: $ git

[GitHub] kafka-site issue #48: Filled in og meta tags

Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/48 @gwenshap sorry forgot to follow up on this PR. Yes, this is still relevant. I just rebased this with the latest changes. Let me know if the changes look ok. --- If your project is set up for

[GitHub] kafka-site pull request #44: 0102 fixes

Github user derrickdoo closed the pull request at: https://github.com/apache/kafka-site/pull/44 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka-site pull request #46: Fix routing bugs

Github user derrickdoo closed the pull request at: https://github.com/apache/kafka-site/pull/46 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: KIP-160 - Augment KStream.print() to allow users pass in extra parameters in the printed string

Overall +1. One comment about the wiki itself: Could you replace the general description of "Argument KStream.print() which is KStream.print(KeyValueMapper)" with the actual added overloaded functions in the wiki page? Guozhang On Mon, May 22, 2017 at 12:21 AM, James Chain wrote: > Hi All, >

Re: Reg: [VOTE] KIP 157 - Add consumer config options to streams reset tool

Hi All, Updated the KIP, as the consumer configurations are required for both Admin Client and Consumer in Stream reset tool. Updated the KIP to use command-config option, similar to other tools like kafka-consumer-groups.sh *https://cwiki.apache.org/confluence/display/KAFKA/KIP+157+-+Add+consume

[GitHub] kafka pull request #2349: KAFKA-4603 Command parsed error, change all Option...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2349 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4603) the argument of shell in doc wrong and command parsed error

[ https://issues.apache.org/jira/browse/KAFKA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4603. -- Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2349 [https://gi

[jira] [Commented] (KAFKA-4603) the argument of shell in doc wrong and command parsed error

[ https://issues.apache.org/jira/browse/KAFKA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030106#comment-16030106 ] ASF GitHub Bot commented on KAFKA-4603: --- Github user asfgit closed the pull request

Build failed in Jenkins: kafka-trunk-jdk7 #2286

See Changes: [jason] KAFKA-5316; Follow-up with ByteBufferOutputStream and other misc -- [...truncated 1.93 MB...] org.apache.kafka.connect.runtime.WorkerConnectorTest >

[jira] [Commented] (KAFKA-5332) When resize the index file, maybe caused the content disappear

[ https://issues.apache.org/jira/browse/KAFKA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030120#comment-16030120 ] Guozhang Wang commented on KAFKA-5332: -- Does look like a bug, [~junrao] could you tak

[jira] [Commented] (KAFKA-5339) Transactions system test with hard broker bounces fails sporadically

[ https://issues.apache.org/jira/browse/KAFKA-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030147#comment-16030147 ] Apurva Mehta commented on KAFKA-5339: - Another puzzling one: {noformat} 21:02:37,45

Re: [DISCUSS] KIP-108: Create Topic Policy

Hey Ismael, Thanks for the KIP. This is definitely useful. Does the KIP apply the topic creation policy to ZK-based topic creation? If not, which seems to be the case from my understanding, should we have a new broker config to disable ZK-based topic creation? This seems necessary to prevent user

Re: [DISCUSS] KIP-108: Create Topic Policy

Hi Dong, No, ZK-based topic creation doesn't go through the policy since it doesn't go through the broker. Given that, I am not sure how the broker config would work. Can you please elaborate? It seems like the way forward is to limit ZK access to brokers only. Ismael On Tue, May 30, 2017 at 10:

[jira] [Issue Comment Deleted] (KAFKA-5339) Transactions system test with hard broker bounces fails sporadically

[ https://issues.apache.org/jira/browse/KAFKA-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta updated KAFKA-5339: Comment: was deleted (was: Another puzzling one: {noformat} 21:02:37,450] ERROR Uncaught error in

Re: Jira-Spam on Dev-Mailinglist

I actually do not know.. Maybe Jun knows better than me? Guozhang On Mon, May 29, 2017 at 12:58 PM, Gwen Shapira wrote: > I agree. > > Guozhang, do you know how to implement the suggestion? JIRA to Apache > Infra? Or is this something we can do ourselves somehow? > > On Mon, May 29, 2017 at 9:

Build failed in Jenkins: kafka-trunk-jdk8 #1611

See Changes: [jason] KAFKA-5316; Follow-up with ByteBufferOutputStream and other misc -- [...truncated 2.96 MB...] kafka.utils.UtilsTest > testReadBytes PASSED kafka.uti

[jira] [Updated] (KAFKA-5202) Handle topic deletion for ongoing transactions

[ https://issues.apache.org/jira/browse/KAFKA-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-5202: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull requ

[GitHub] kafka pull request #3130: KAFKA-5202: Handle topic deletion while trying to ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3130 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-5202) Handle topic deletion for ongoing transactions

[ https://issues.apache.org/jira/browse/KAFKA-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030217#comment-16030217 ] ASF GitHub Bot commented on KAFKA-5202: --- Github user asfgit closed the pull request

Re: [DISCUSS] KIP-108: Create Topic Policy

Hey Ismael, I agree that it makes sense not to cover ZK-based topic creation with the topic creation policy and limit ZK access to brokers only going forward. My point is that we need a way to disable ZK-based topic creation so that all topic creation goes through the topic creation policy as spec

[jira] [Created] (KAFKA-5349) KafkaConsumer occasionally hits IllegalStateException

Apurva Mehta created KAFKA-5349: --- Summary: KafkaConsumer occasionally hits IllegalStateException Key: KAFKA-5349 URL: https://issues.apache.org/jira/browse/KAFKA-5349 Project: Kafka Issue Type:

Jenkins build is back to normal : kafka-trunk-jdk7 #2287

See

[jira] [Commented] (KAFKA-5349) KafkaConsumer occasionally hits IllegalStateException

[ https://issues.apache.org/jira/browse/KAFKA-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030276#comment-16030276 ] Apurva Mehta commented on KAFKA-5349: - cc [~hachikuji] > KafkaConsumer occasionally h

Re: [DISCUSS] KIP-108: Create Topic Policy

Just for completeness, I think one option is to: 1. Upgrade broker-side auto topic creation to send CreateTopicsRequest. This is just to unify the topic creation flow and policy enforcement. 2. Migrate all clients away from topic creation from zookeeper and instead send CreateTopicsRequest 3. ACL o

Re: [DISCUSS] KIP-108: Create Topic Policy

It seems like, to make it really secure, we need the enforcement to be done at the ZooKepeer level. Any broker or client-side configuration can just be ignored by a malicious client. Do we have documentation or code that configures ZK to prevent unprivileged users from modifying the topic configu

[jira] [Commented] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

[ https://issues.apache.org/jira/browse/KAFKA-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030291#comment-16030291 ] ASF GitHub Bot commented on KAFKA-5150: --- Github user ijuma closed the pull request a

[GitHub] kafka pull request #3164: KAFKA-5150: Reduce lz4 decompression overhead (wit...

Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/3164 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs

Hi all, In the course of reviewing the AdminClient#apiVersions API, we found out that it was using an internal class, org.apache.kafka.clients.NodeApiVersions. This internal class references more internal stuff, including things in org.apache.kafka.common.protocol. So we filed KAFKA-5214 to crea

Re: [DISCUSS] KIP-108: Create Topic Policy

Do we have an old version of bin/kafka-topics.sh which creates topic via ZK and still allows user to access ZK with ACL? Another concern is that some user may not have ACL service deployed in their cluster. If neither of these is issue, then I would prefer the zookeeper approach instead of adding

Re: [DISCUSS] KIP-108: Create Topic Policy

Hey Colin, I think one big advantage of the broker side config is that it can not be ignored by the malicious client, right? Thanks, Dong On Tue, May 30, 2017 at 3:53 PM, Dong Lin wrote: > Do we have an old version of bin/kafka-topics.sh which creates topic via > ZK and still allows user to ac

Re: [DISCUSS] KIP-108: Create Topic Policy

@Colin checkout kafka.admin.ZkSecurityMigrator. This is what I meant in my earlier comment on "acl off zookeeper" On Tue, May 30, 2017 at 3:39 PM Colin McCabe wrote: > It seems like, to make it really secure, we need the enforcement to be > done at the ZooKepeer level. Any broker or client-side

[GitHub] kafka-site pull request #58: [PLEASE DO NOT MERGE] Further break apart Strea...

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/58 [PLEASE DO NOT MERGE] Further break apart Streams docs This is just to illustrate necessary changes for changing the structure of Streams documentation. Please don't merge yet.

Re: [DISCUSS] KIP-108: Create Topic Policy

On Tue, May 30, 2017, at 15:55, Dong Lin wrote: > Hey Colin, > > I think one big advantage of the broker side config is that it can not be > ignored by the malicious client, right? Hi Dong, The scenario I was thinking of is where a malicious client communicates directly with ZooKeeper, bypassing

[GitHub] kafka pull request #3169: MINOR: Fix doc for producer throttle time metrics

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3169 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-108: Create Topic Policy

On Tue, May 30, 2017 at 4:26 PM, Colin McCabe wrote: > On Tue, May 30, 2017, at 15:55, Dong Lin wrote: > > Hey Colin, > > > > I think one big advantage of the broker side config is that it can not be > > ignored by the malicious client, right? > > Hi Dong, > > The scenario I was thinking of is wh

[GitHub] kafka pull request #3171: MINOR: A few cleanups in KafkaApis and Transaction...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3171 MINOR: A few cleanups in KafkaApis and TransactionMarkerChannelManager You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-t

Re: [DISCUSS] KIP-108: Create Topic Policy

I am not sure if the additional complexity in the Controller is worth it for this use case. It seems like it would be better to swap the tools to use AdminClient and then restrict access to ZK (via ACLs and/or network segmentation). Either way, that proposal should be done via a separate KIP as KIP

Re: [DISCUSS] KIP-108: Create Topic Policy

Certainly. I think it is reasonable to create a separate KIP to enforce the topic creation policy. After all the administrator needs a guarantee that the policy that they have specified in the broker will be enforce -- otherwise the feature doesn't seem complete. On Tue, May 30, 2017 at 4:45 PM, I

[jira] [Created] (KAFKA-5350) Modify Unstable annotations in Streams API

Guozhang Wang created KAFKA-5350: Summary: Modify Unstable annotations in Streams API Key: KAFKA-5350 URL: https://issues.apache.org/jira/browse/KAFKA-5350 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #2935: MINOR: onControllerResignation should be invoked i...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2935 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030397#comment-16030397 ] ASF GitHub Bot commented on KAFKA-3761: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-5350) Modify Unstable annotations in Streams API

[ https://issues.apache.org/jira/browse/KAFKA-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030395#comment-16030395 ] ASF GitHub Bot commented on KAFKA-5350: --- GitHub user guozhangwang opened a pull requ

[GitHub] kafka pull request #3172: KAFKA-5350: Modify unstable annotations in Streams...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/3172 KAFKA-5350: Modify unstable annotations in Streams API You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K5350-compatibil

Jenkins build is back to normal : kafka-trunk-jdk8 #1612

See

Re: [DISCUSS] KIP-108: Create Topic Policy

Yes, and the only way to enforce it is by restricting ZK access to brokers only. If ZK access is available to users, they can get around the broker config proposal as well. And then the question is about benefit versus cost. We can have that discussion when the KIP is proposed. Ismael On Wed, May

[GitHub] kafka-site pull request #59: Add auto syntax highlighter

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/59 Add auto syntax highlighter Added plugin to do auto syntax highlighting for code snippets in our docs. Here's an example of how to syntax highlight some java code: `` Here

Build failed in Jenkins: kafka-0.11.0-jdk7 #50

See Changes: [junrao] MINOR: Fix doc for producer throttle time metrics -- [...truncated 906.15 KB...] kafka.controller.ControllerEventManagerTest > testSuccessfulEvent PA

Build failed in Jenkins: kafka-trunk-jdk7 #2289

See Changes: [junrao] MINOR: Fix doc for producer throttle time metrics -- [...truncated 905.43 KB...] kafka.server.DelayedOperationTest > testRequestPurge PASSED kafka.

[GitHub] kafka pull request #2576: kafka-4767: KafkaProducer is not joining its IO th...

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2576 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2317: kafka-4592: Kafka Producer Metrics Invalid Value

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2317 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2054: kafka-4295: ConsoleConsumer does not delete the te...

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2054 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2967: KAFKA-5150 reduce lz4 decompression overhead

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2967 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

[ https://issues.apache.org/jira/browse/KAFKA-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030468#comment-16030468 ] ASF GitHub Bot commented on KAFKA-5150: --- Github user asfgit closed the pull request

  1   2   >