[jira] [Commented] (KAFKA-3719) Pattern regex org.apache.kafka.common.utils.Utils.HOST_PORT_PATTERN is too narrow

[ https://issues.apache.org/jira/browse/KAFKA-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288595#comment-15288595 ] Balazs Kossovics commented on KAFKA-3719: - [~gwenshap] Oh, you are right. Actually

[jira] [Commented] (KAFKA-3719) Pattern regex org.apache.kafka.common.utils.Utils.HOST_PORT_PATTERN is too narrow

[ https://issues.apache.org/jira/browse/KAFKA-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288597#comment-15288597 ] Balazs Kossovics commented on KAFKA-3719: - Oh, you are right. Actually it was a [

[jira] [Issue Comment Deleted] (KAFKA-3719) Pattern regex org.apache.kafka.common.utils.Utils.HOST_PORT_PATTERN is too narrow

[ https://issues.apache.org/jira/browse/KAFKA-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Kossovics updated KAFKA-3719: Comment: was deleted (was: [~gwenshap] Oh, you are right. Actually it was a [bug|https://gi

[jira] [Resolved] (KAFKA-3719) Pattern regex org.apache.kafka.common.utils.Utils.HOST_PORT_PATTERN is too narrow

[ https://issues.apache.org/jira/browse/KAFKA-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Kossovics resolved KAFKA-3719. - Resolution: Not A Bug > Pattern regex org.apache.kafka.common.utils.Utils.HOST_PORT_PATTER

[jira] [Created] (KAFKA-3725) Update documentation with regards to XFS

Ismael Juma created KAFKA-3725: -- Summary: Update documentation with regards to XFS Key: KAFKA-3725 URL: https://issues.apache.org/jira/browse/KAFKA-3725 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-3725) Update documentation with regards to XFS

[ https://issues.apache.org/jira/browse/KAFKA-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288623#comment-15288623 ] Ismael Juma commented on KAFKA-3725: [~toddpalino], can you please check our current r

Re: Perf producer/consumers for compacted topics

Hi, There is a kafka.tools.TestLogCleaning tool, which is used to stress test the compaction feature. This tool validates the correctness of compaction process. This tool can be improved for perf testing. I think you want to benchmark server side compaction process. Currently we have few compact

Re: Perf producer/consumers for compacted topics

Hi, I'm interested in benchmarking the impact of compaction on producers and consumers and long term cluster stability. That's not *quite* the impact of it on the server side, but it certainly plays into it. For example, I'd like to be able to answer "in configuration X, if we write N messages int

[jira] [Updated] (KAFKA-3726) Enable cold storage option

[ https://issues.apache.org/jira/browse/KAFKA-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radoslaw Gruchalski updated KAFKA-3726: --- Attachment: kafka-cold-storage.txt The text version of the mentioned article. > Enabl

[jira] [Created] (KAFKA-3726) Enable cold storage option

Radoslaw Gruchalski created KAFKA-3726: -- Summary: Enable cold storage option Key: KAFKA-3726 URL: https://issues.apache.org/jira/browse/KAFKA-3726 Project: Kafka Issue Type: Wish

[jira] [Commented] (KAFKA-3637) Add method that checks if streams are initialised

[ https://issues.apache.org/jira/browse/KAFKA-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289271#comment-15289271 ] Thomas Szymanski commented on KAFKA-3637: - Hi, I would like to take this one to st

download - 0.10.0.0 RC6

Hello, A naïve question. How do I download 0.10.0 RC6 (or RC5) version? I clicked on "Download zip" under 0.10.0 trunk @ github, downloaded kafka-0.10.0.zip today, compiled & started the server. Mbeans shows the version to be 0.10.0-SNAPSHOT and am not sure where / how to download RC6. Please

[jira] [Created] (KAFKA-3727) inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subscribed

Edoardo Comar created KAFKA-3727: Summary: inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subscribed Key: KAFKA-3727 URL: https://issues.apache.org/jira/browse/KAFKA-3727

Re: download - 0.10.0.0 RC6

Hi From Gwen’s announcement of RC6 last night: > * Release artifacts to be voted upon (source and binary): > http://home.apache.org/~gwenshap/0.10.0.0-rc6/ Regards Ian. --- Ian Wrigley Director, Education Services Confluent, Inc Cell: (323) 819-4075 > On May 18, 2016, at 1:04 PM, Ramanan, Buv

[jira] [Updated] (KAFKA-3727) inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subscribed

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar updated KAFKA-3727: - Description: The behavior of a consumer on poll() for a non-existing topic is surprisingly differ

[jira] [Updated] (KAFKA-3727) inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subscribed

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar updated KAFKA-3727: - Description: The behavior of a consumer on poll() for a non-existing topic is surprisingly differ

[jira] [Updated] (KAFKA-3727) Consumer.poll() stuck in loop on non-existent topic manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar updated KAFKA-3727: - Summary: Consumer.poll() stuck in loop on non-existent topic manually assigned (was: inconsistent

[jira] [Updated] (KAFKA-3727) inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subscribed

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar updated KAFKA-3727: - Description: Inconsistent behavior of Consumer.poll() on non-existent topic when assigned vs subs

[jira] [Updated] (KAFKA-3727) Consumer.poll() stuck in loop on non-existent topic manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar updated KAFKA-3727: - Description: The behavior of a consumer on poll() for a non-existing topic is surprisingly differ

RE: [COMMERCIAL] Re: download - 0.10.0.0 RC6

Ian, Thanks a lot for the prompt response. What is the difference between the following? 1) kafka-0.10.0.0-src.tgz 2) kafka_2.10-0.10.0.0.tgz 3) kafka_2.11-0.10.0.0.tgz I suppose the 1st one is the source and the 2nd & 3rd are binaries... I am not able to figure out the difference between 2nd

[GitHub] kafka pull request: MINOR: Use `Record` instead of `ByteBufferMess...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1357 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk8 #638

See Changes: [wangguoz] MINOR: Use `Record` instead of `ByteBufferMessageSet` in -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-5 (docke

[jira] [Created] (KAFKA-3728) inconsistent behavior of Consumer.poll() when assigned vs subscribed

Edoardo Comar created KAFKA-3728: Summary: inconsistent behavior of Consumer.poll() when assigned vs subscribed Key: KAFKA-3728 URL: https://issues.apache.org/jira/browse/KAFKA-3728 Project: Kafka

[jira] [Commented] (KAFKA-3396) Unauthorized topics are returned to the user

[ https://issues.apache.org/jira/browse/KAFKA-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289551#comment-15289551 ] Edoardo Comar commented on KAFKA-3396: -- I'm held back by https://issues.apache.org/j

Build failed in Jenkins: kafka-trunk-jdk7 #1303

See Changes: [wangguoz] MINOR: Use `Record` instead of `ByteBufferMessageSet` in -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker Ub

[jira] [Commented] (KAFKA-3726) Enable cold storage option

[ https://issues.apache.org/jira/browse/KAFKA-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289608#comment-15289608 ] Ben Stopford commented on KAFKA-3726: - The standard approach to this sort of problem w

Re: [COMMERCIAL] Re: download - 0.10.0.0 RC6

It's for different Scala versions, ie, 2.10 and 2.11, respectively. -Matthias On 05/18/2016 07:28 PM, Ramanan, Buvana (Nokia - US) wrote: > Ian, > > Thanks a lot for the prompt response. > > What is the difference between the following? > > 1) kafka-0.10.0.0-src.tgz > 2) kafka_2.10-0.10.0.0.t

[jira] [Created] (KAFKA-3729) Auto-configure non-default SerDes passed alongside the topology builder

Fred Patton created KAFKA-3729: -- Summary: Auto-configure non-default SerDes passed alongside the topology builder Key: KAFKA-3729 URL: https://issues.apache.org/jira/browse/KAFKA-3729 Project: Kafka

[GitHub] kafka pull request: MINOR: Bump system test ducktape dependency to...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1397 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk8 #639

See Changes: [me] MINOR: Bump system test ducktape dependency to 0.5.1 -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-2 (docker Ubuntu u

Build failed in Jenkins: kafka-0.10.0-jdk7 #99

See Changes: [me] MINOR: Bump system test ducktape dependency to 0.5.1 -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-4 (docker Ubuntu u

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

Generally, this seems like a sensible proposal to me. Regarding (1): time and message count seem sensible. I can’t think of a specific use case for bytes but it seems like there could be one. Regarding (2): The setting log.cleaner.min.cleanable.ratio currently seems to have two uses. It con

[jira] [Created] (KAFKA-3730) Problem when updating from 0.8.2 to 0.9.0

Clint Hillerman created KAFKA-3730: -- Summary: Problem when updating from 0.8.2 to 0.9.0 Key: KAFKA-3730 URL: https://issues.apache.org/jira/browse/KAFKA-3730 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-3728) inconsistent behavior of Consumer.poll() when assigned vs subscribed

[ https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290084#comment-15290084 ] Ismael Juma commented on KAFKA-3728: cc [~hachikuji] > inconsistent behavior of Consu

[jira] [Commented] (KAFKA-3717) Support building aggregate javadoc for all project modules

[ https://issues.apache.org/jira/browse/KAFKA-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290102#comment-15290102 ] ASF GitHub Bot commented on KAFKA-3717: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3717; Support building aggregate javadoc...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1398 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-3717) Support building aggregate javadoc for all project modules

[ https://issues.apache.org/jira/browse/KAFKA-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3717: --- Resolution: Fixed Fix Version/s: 0.10.0.0 Status: Resolved (was: Patch Available) I

Build failed in Jenkins: kafka-trunk-jdk7 #1304

See Changes: [me] MINOR: Bump system test ducktape dependency to 0.5.1 -- [...truncated 3305 lines...] kafka.log.LogTest > testAsyncDelete PASSED kafka.log.LogTest > testReadOutOfRange PASSED

Build failed in Jenkins: kafka-trunk-jdk8 #640

See Changes: [ismael] KAFKA-3717; Support building aggregate javadoc for all project modules -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubu

Build failed in Jenkins: kafka-0.10.0-jdk7 #100

See Changes: [ismael] KAFKA-3717; Support building aggregate javadoc for all project modules -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H1

[GitHub] kafka pull request: KAFKA-3664 (WIP): Commit offset of unsubscribe...

GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/1363 KAFKA-3664 (WIP): Commit offset of unsubscribed partitions of the new consumer on a subscription change When users are using group management, if they call consumer.subscribe() to change t

[GitHub] kafka pull request: KAFKA-3664 (WIP): Commit offset of unsubscribe...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1363 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (KAFKA-3664) When subscription set changes on new consumer, the partitions may be removed without offset being committed.

[ https://issues.apache.org/jira/browse/KAFKA-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290181#comment-15290181 ] ASF GitHub Bot commented on KAFKA-3664: --- GitHub user vahidhashemian reopened a pull

[jira] [Commented] (KAFKA-3664) When subscription set changes on new consumer, the partitions may be removed without offset being committed.

[ https://issues.apache.org/jira/browse/KAFKA-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290179#comment-15290179 ] ASF GitHub Bot commented on KAFKA-3664: --- Github user vahidhashemian closed the pull

[GitHub] kafka pull request: MINOR: Catch Throwable in CommitSourceTask()

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1402 MINOR: Catch Throwable in CommitSourceTask() You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka source-task-commit-record Altern

[jira] [Updated] (KAFKA-3718) propagate all KafkaConfig __consumer_offsets configs to OffsetConfig instantiation

[ https://issues.apache.org/jira/browse/KAFKA-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-3718: Fix Version/s: 0.10.0.0 > propagate all KafkaConfig __consumer_offsets configs to OffsetConfig > in

[GitHub] kafka pull request: KAFKA-3722 : The PlaintextChannelBuilder shoul...

GitHub user MayureshGharat opened a pull request: https://github.com/apache/kafka/pull/1403 KAFKA-3722 : The PlaintextChannelBuilder should always use the DefaultPrincipalBuilder Consider this scenario : 1) We have a Kafka Broker running on PlainText and SSL port simul

[jira] [Commented] (KAFKA-3722) PlaintextChannelBuilder should not use ChannelBuilders.createPrincipalBuilder(configs) for creating instance of PrincipalBuilder

[ https://issues.apache.org/jira/browse/KAFKA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290233#comment-15290233 ] ASF GitHub Bot commented on KAFKA-3722: --- GitHub user MayureshGharat opened a pull re

Question regarding enhancement for Apache Kafka

Hello, I Janagan Sivagnanasundaram from Sri Lanka, and pursuing final year undergraduate studies under software engineering at University of Colombo School of Computing, Sri Lanka. As a part of the fulfillment of the degree we have to do a final year software engineering project with 3 members. So

Jenkins build is back to normal : kafka-trunk-jdk7 #1305

See

[GitHub] kafka pull request: KAFKA-2935: Remove vestigial WorkerConfig.CLUS...

GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1404 KAFKA-2935: Remove vestigial WorkerConfig.CLUSTER_CONFIG You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka kafka-2935 Alternatively

[jira] [Commented] (KAFKA-2935) Remove vestigial CLUSTER_CONFIG in WorkerConfig

[ https://issues.apache.org/jira/browse/KAFKA-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290286#comment-15290286 ] ASF GitHub Bot commented on KAFKA-2935: --- GitHub user shikhar opened a pull request:

[jira] [Commented] (KAFKA-2935) Remove vestigial CLUSTER_CONFIG in WorkerConfig

[ https://issues.apache.org/jira/browse/KAFKA-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290294#comment-15290294 ] Shikhar Bhushan commented on KAFKA-2935: I could not find any documentation refere

[jira] [Commented] (KAFKA-3335) Kafka Connect hangs in shutdown hook

[ https://issues.apache.org/jira/browse/KAFKA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290333#comment-15290333 ] Shikhar Bhushan commented on KAFKA-3335: Currently {{start()}} looks like {code:

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

Interesting! This needs to be double checked by someone with more experience, but reading the code, it looks like "log.cleaner.min.cleanable.ratio" controls *just* the second property, and I'm not even convinced about that. Few facts: 1. Each cleaner thread cleans one log at a time. It always go

[jira] [Commented] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290440#comment-15290440 ] Jay Kreps commented on KAFKA-3511: -- It just seems really weird to bake one or two aggrega

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

The sad part is I actually did think pretty hard about how to configure that stuff so I guess *I* think the config makes sense! Clearly trying to prevent my being shot :-) I agree the name could be improved and the documentation is quite spartan--no guidance at all on how to set it or what it trad

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

So in summary we never considered this a mechanism to give the consumer time to consume prior to compaction, just a mechanism to control space wastage. It sort of accidentally gives you that but it's super hard to reason about it as an SLA since it is relative to the log size rather than absolute.

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

Oops :) The docs are definitely not doing the feature any favors, but I didn't mean to imply the feature is thoughtless. Here's the thing I'm not getting: You are trading off disk space for IO efficiency. Thats reasonable. But why not allow users to specify space in bytes? Basically tell the Log