[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

2015-06-22 Thread Honghai Chen (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595450#comment-14595450 ] Honghai Chen commented on KAFKA-1646: - See the commit https://git-wip-us.apache.org/r

[jira] [Assigned] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Chris Black (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Black reassigned KAFKA-2290: -- Assignee: Chris Black > OffsetIndex should open RandomAccessFile consistently > ---

[jira] [Updated] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Chris Black (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Black updated KAFKA-2290: --- Attachment: KAFKA-2290.patch > OffsetIndex should open RandomAccessFile consistently > ---

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jiangjie Qin
Very useful KIP. I have no clear opinion over where to put the framework will be better yet. I agree with Gwen on the benefits we can get from have a separate project for Copycat. But still have a few questions: 1. As far as code is concerned, Copycat would be some datasource adapters + Kafka clie

[jira] [Commented] (KAFKA-2092) New partitioning for better load balancing

2015-06-22 Thread Gianmarco De Francisci Morales (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595510#comment-14595510 ] Gianmarco De Francisci Morales commented on KAFKA-2092: --- Any more th

[jira] [Comment Edited] (KAFKA-2092) New partitioning for better load balancing

2015-06-22 Thread Gianmarco De Francisci Morales (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589634#comment-14589634 ] Gianmarco De Francisci Morales edited comment on KAFKA-2092 at 6/22/15 8:42 AM:

[jira] [Updated] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Ivan Simoneko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Simoneko updated KAFKA-2235: - Attachment: KAFKA-2235_v2.patch > LogCleaner offset map overflow > --

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Ivan Simoneko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595798#comment-14595798 ] Ivan Simoneko commented on KAFKA-2235: -- [~junrao] thank you for review. Please check

[jira] [Resolved] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2290. Resolution: Fixed Fix Version/s: 0.8.3 Thanks for the patch. +1 and committed to trunk. > OffsetInde

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review88796 --- Ship it! I did not review it thoroughly but the design looks clean

[jira] [Resolved] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2235. Resolution: Fixed Fix Version/s: 0.8.3 Assignee: Ivan Simoneko (was: Jay Kreps) Thanks for

Re: Review Request 35677: Patch for KAFKA-2288

2015-06-22 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35677/ --- (Updated June 22, 2015, 5:02 p.m.) Review request for kafka. Bugs: KAFKA-2288

[jira] [Updated] (KAFKA-2288) Follow-up to KAFKA-2249 - reduce logging and testing

2015-06-22 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2288: Attachment: KAFKA-2288_2015-06-22_10:02:27.patch > Follow-up to KAFKA-2249 - reduce logging and test

[jira] [Commented] (KAFKA-2288) Follow-up to KAFKA-2249 - reduce logging and testing

2015-06-22 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596233#comment-14596233 ] Gwen Shapira commented on KAFKA-2288: - Updated reviewboard https://reviews.apache.org/

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

2015-06-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596242#comment-14596242 ] Jason Gustafson commented on KAFKA-2120: Hey [~mgharat] and [~becket_qin], how is

[jira] [Created] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2293: -- Summary: IllegalFormatConversionException in Partition.scala Key: KAFKA-2293 URL: https://issues.apache.org/jira/browse/KAFKA-2293 Project: Kafka Issue T

[jira] [Updated] (KAFKA-2245) Add response tests for ConsumerCoordinator

2015-06-22 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2245: -- Resolution: Fixed Reviewer: Joel Koshy Status: Resolved (was: Patch Available) Thanks for

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

2015-06-22 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596265#comment-14596265 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~jasong35], we have began work on thi

Review Request 35734: Patch for KAFKA-2293

2015-06-22 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35734/ --- Review request for kafka. Bugs: KAFKA-2293 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596296#comment-14596296 ] Aditya A Auradkar commented on KAFKA-2293: -- Created reviewboard https://reviews.a

[jira] [Updated] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2293: - Attachment: KAFKA-2293.patch > IllegalFormatConversionException in Partition.scala > -

[jira] [Commented] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596298#comment-14596298 ] Aditya Auradkar commented on KAFKA-2293: [~junrao] Can you take a look at this min

[jira] [Created] (KAFKA-2294) javadoc compile error due to illegal , build failing (jdk 8)

2015-06-22 Thread Jeremy Fields (JIRA)
Jeremy Fields created KAFKA-2294: Summary: javadoc compile error due to illegal , build failing (jdk 8) Key: KAFKA-2294 URL: https://issues.apache.org/jira/browse/KAFKA-2294 Project: Kafka

[jira] [Commented] (KAFKA-2276) Initial patch for KIP-25

2015-06-22 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596624#comment-14596624 ] Geoffrey Anderson commented on KAFKA-2276: -- Pull request is here: https://github.

Re: [GitHub] kafka pull request: Kafka 2276

2015-06-22 Thread Geoffrey Anderson
Hi, I'm pinging the dev list regarding KAFKA-2276 (KIP-25 initial patch) again since it sounds like at least one person I spoke with did not see the initial pull request. Pull request: https://github.com/apache/kafka/pull/70/ JIRA: https://issues.apache.org/jira/browse/KAFKA-2276 Thanks! Geoff

[jira] [Comment Edited] (KAFKA-2276) Initial patch for KIP-25

2015-06-22 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596624#comment-14596624 ] Geoffrey Anderson edited comment on KAFKA-2276 at 6/22/15 9:00 PM: -

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jay Kreps
Hey Roshan, That is definitely the key question in this space--what can we do that other systems don't? It's true that there are a number of systems that copy data between things. At a high enough level of abstraction I suppose they are somewhat the same. But I think this area is the source of ra

Re: Review Request 35734: Patch for KAFKA-2293

2015-06-22 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35734/#review88866 --- Ship it! Minor comment: would use `%s` with `TopicAndPartition` ins

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jay Kreps
Hey Gwen, That makes a lot of sense. Here was the thinking on our side. I guess there are two questions, where does Copycat go and where do the connectors go? I'm in favor of Copycat being in Kafka and the connectors being federated. Arguments for federating connectors: - There will be like >>

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Ewen Cheslack-Postava
I'll respond to specific comments, but at the bottom of this email I've included some comparisons with other connector frameworks and Kafka import/export tools. This definitely isn't an exhaustive list, but hopefully will clarify how I'm thinking about Copycat should live wrt these other systems.

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 22, 2015, 11:35 p.m.) Review request for kafka. Bugs: KAFKA-216

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-06-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596850#comment-14596850 ] Jason Gustafson commented on KAFKA-2168: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-06-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2168: --- Attachment: KAFKA-2168_2015-06-22_16:34:37.patch > New consumer poll() can block other calls l

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35655/#review88877 --- Ship it! LGTM. There were 9 question marks when 10 characters were

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
> On June 22, 2015, 11:43 p.m., Ewen Cheslack-Postava wrote: > > LGTM. There were 9 question marks when 10 characters were requested, so the > > problem was probably just that a whitespace character at the start or end > > would get trimmed during AbstractConfig's parsing. That's what I though

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
> On June 21, 2015, 5:49 a.m., Guozhang Wang wrote: > > I think the main reason is that util.Random.nextString() may include other > > non-ascii chars and hence its toString may not be well-defined: > > > > http://alvinalexander.com/scala/creating-random-strings-in-scala > > > > So I think bot

[jira] [Commented] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

2015-06-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596965#comment-14596965 ] Aditya Auradkar commented on KAFKA-2205: [~junrao] - Can you review this patch? >

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Ewen Cheslack-Postava
> On June 22, 2015, 11:43 p.m., Ewen Cheslack-Postava wrote: > > LGTM. There were 9 question marks when 10 characters were requested, so the > > problem was probably just that a whitespace character at the start or end > > would get trimmed during AbstractConfig's parsing. > > Jason Gustafson

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Roshan Naik
Thanks Jay and Ewen for the response. >@Jay > > 3. This has a built in notion of parallelism throughout. It was not obvious how it will look like or differ from existing systemsÅ  since all of existing ones do parallelize data movement. @Ewen, >Import: Flume is just one of many similar syste

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review88914 --- Ship it! Thanks for the latest patch. Looks good overall. To avoid

Re: [GitHub] kafka pull request: Kafka 2276

2015-06-22 Thread Gwen Shapira
Thanks, I indeed missed the original :) Is the plan to squash the commits and merge a pull request with single commit that matches the JIRA #? This will be more in line with how commits were organized until now and will make life much easier when cherry-picking. Gwen On Mon, Jun 22, 2015 at 1:58