[jira] [Commented] (KAFKA-1736) Improve parition-broker assignment strategy for better availaility in majority durability modes

2014-10-30 Thread Kyle Banker (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191318#comment-14191318 ] Kyle Banker commented on KAFKA-1736: Thanks for all your notes on this, [~jkreps], [~n

[jira] [Commented] (KAFKA-1686) Implement SASL/Kerberos

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191258#comment-14191258 ] Gwen Shapira commented on KAFKA-1686: - Hi [~harsha_ch], I assume you already started

Re: A more scalable Kafka to Hadoop InputFormat

2014-10-30 Thread Bhavesh Mistry
HI Casey, Do you think Camus Job can benefit from this ? It pulls data from Kafka and stores it into timestamps bucket. Here is link to source: https://github.com/linkedin/camus/blob/master/camus-etl-kafka/src/main/java/com/linkedin/camus/etl/kafka/mapred/EtlInputFormat.java Thanks, Bhavesh O

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191241#comment-14191241 ] Joel Koshy commented on KAFKA-1555: --- ack never mind.. I see what you mean. && requiredA

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191239#comment-14191239 ] Joel Koshy commented on KAFKA-1555: --- Yeah what I wrote earlier was incorrect - sorry. It

[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191229#comment-14191229 ] Pradeep commented on KAFKA-1738: Yes. We tried sending data but it gave the exception. k.

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191206#comment-14191206 ] Gwen Shapira commented on KAFKA-1695: - Makes sense. Thanks for pointing this out. >

[jira] [Assigned] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira reassigned KAFKA-1082: --- Assignee: Gwen Shapira > zkclient dies after UnknownHostException in zk reconnect > -

[jira] [Commented] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191205#comment-14191205 ] Gwen Shapira commented on KAFKA-1082: - Assigning to me so I can track this - we want t

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191202#comment-14191202 ] Gwen Shapira commented on KAFKA-1555: - " it is independent of the producer's ack confi

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191196#comment-14191196 ] Joel Koshy commented on KAFKA-1555: --- Sorry for the late feedback. configuration.html: -

[jira] [Assigned] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy reassigned KAFKA-1729: - Assignee: Joel Koshy > add doc for Kafka-based offset management in 0.8.2 > -

[jira] [Commented] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191115#comment-14191115 ] Jun Rao commented on KAFKA-1481: Thanks for the patch. Looks good overall. Some minor comm

Build failed in Jenkins: Kafka-trunk #321

2014-10-30 Thread Apache Jenkins Server
See Changes: [jjkoshy] KAFKA-1647; Create replicas on follower transition even if leader is -- [...truncated 1761 lines...] kafka.server.SimpleFetchTest > testReadFromLog PASSED kafka.server.ServerSh

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1647: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch - committed to 0

[jira] [Commented] (KAFKA-1501) transient unit tests failures due to port already in use

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191022#comment-14191022 ] Guozhang Wang commented on KAFKA-1501: -- OK. I finally got to re-produce the error aft

Re: Review Request 26373: Patch for KAFKA-1647

2014-10-30 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26373/#review59274 --- Ship it! Ship It! - Joel Koshy On Oct. 30, 2014, 10:50 p.m., Jia

[jira] [Updated] (KAFKA-1333) Add consumer co-ordinator module to the server

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1333: - Description: Scope of this JIRA is to just add a consumer co-ordinator module that do the followi

[jira] [Updated] (KAFKA-1740) Merge Offset manager into Coordinator

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1740: - Fix Version/s: 0.9.0 > Merge Offset manager into Coordinator > ---

[jira] [Created] (KAFKA-1740) Merge Offset manager into Coordinator

2014-10-30 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-1740: Summary: Merge Offset manager into Coordinator Key: KAFKA-1740 URL: https://issues.apache.org/jira/browse/KAFKA-1740 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-1330) Implement subscribe(TopicPartition...partitions) and pool(timeout) in the consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1330: - Description: This involves adding basic fetch functionality (equivalent to SimpleConsumer) to the

[jira] [Updated] (KAFKA-1330) Implement subscribe(TopicPartition...partitions) and pool(timeout) in the consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1330: - Summary: Implement subscribe(TopicPartition...partitions) and pool(timeout) in the consumer (was:

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1647: Attachment: KAFKA-1647_2014-10-30_15:50:33.patch > Replication offset checkpoints (high water marks)

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190977#comment-14190977 ] Jiangjie Qin commented on KAFKA-1647: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 26373: Patch for KAFKA-1647

2014-10-30 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26373/ --- (Updated Oct. 30, 2014, 10:50 p.m.) Review request for kafka. Bugs: KAFKA-164

[jira] [Updated] (KAFKA-1335) Add rebalancing logic to the coordinator / consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1335: - Description: This implements the group management protocol. This will be a tricky and potentially

[jira] [Updated] (KAFKA-1335) Add rebalancing logic to the coordinator / consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1335: - Summary: Add rebalancing logic to the coordinator / consumer (was: Add rebalancing logic to the s

Re: Review Request 26373: Patch for KAFKA-1647

2014-10-30 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26373/ --- (Updated Oct. 30, 2014, 10:38 p.m.) Review request for kafka. Bugs: KAFKA-164

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190963#comment-14190963 ] Jiangjie Qin commented on KAFKA-1647: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1647: Attachment: KAFKA-1647_2014-10-30_15:38:02.patch > Replication offset checkpoints (high water marks)

[jira] [Updated] (KAFKA-1334) Add failure detection capability to the coordinator / consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1334: - Description: 1) Add coordinator discovery and failure detection to the consumer. 2) Add failure de

[jira] [Updated] (KAFKA-1334) Add failure detection capability to the coordinator / consumer

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1334: - Summary: Add failure detection capability to the coordinator / consumer (was: Add failure detecti

[jira] [Created] (KAFKA-1739) Remove testComplexCompressDecompress in MessageCompressionTest

2014-10-30 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-1739: Summary: Remove testComplexCompressDecompress in MessageCompressionTest Key: KAFKA-1739 URL: https://issues.apache.org/jira/browse/KAFKA-1739 Project: Kafka

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1647: Attachment: KAFKA-1647_2014-10-30_15:10:22.patch > Replication offset checkpoints (high water marks)

Re: Review Request 26373: Patch for KAFKA-1647

2014-10-30 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26373/ --- (Updated Oct. 30, 2014, 10:10 p.m.) Review request for kafka. Bugs: KAFKA-164

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190936#comment-14190936 ] Jiangjie Qin commented on KAFKA-1647: - Updated reviewboard https://reviews.apache.org/

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190931#comment-14190931 ] Jiangjie Qin commented on KAFKA-1647: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1647: Attachment: KAFKA-1647_2014-10-30_15:07:09.patch > Replication offset checkpoints (high water marks)

Re: Review Request 26373: Patch for KAFKA-1647

2014-10-30 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26373/ --- (Updated Oct. 30, 2014, 10:07 p.m.) Review request for kafka. Bugs: KAFKA-164

Re: [DISCUSSION] Nested compression in Kafka?

2014-10-30 Thread Guozhang Wang
Thanks Jun and Neha. I will go ahead and file the JIRA to remove this test case then.

[jira] [Updated] (KAFKA-1733) Producer.send will block indeterminately when broker is unavailable.

2014-10-30 Thread Marc Chung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Chung updated KAFKA-1733: -- Attachment: kafka-1733-add-connectTimeoutMs.patch In this patch, I've set the connectTimeoutMs to the sa

Re: [DISCUSSION] Nested compression in Kafka?

2014-10-30 Thread Neha Narkhede
+1 on not having more than one level of compression. On Thu, Oct 30, 2014 at 1:30 PM, Jun Rao wrote: > Yes, I agree that we don't need to support more than 1 level of nesting for > the compressed messages. > > Thanks, > > Jun > > On Mon, Oct 27, 2014 at 4:27 PM, Guozhang Wang wrote: > > > Hello

[jira] [Commented] (KAFKA-1733) Producer.send will block indeterminately when broker is unavailable.

2014-10-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190809#comment-14190809 ] Jun Rao commented on KAFKA-1733: Since this doesn't seem to be an 0.8.2 blocker, let's pat

[jira] [Commented] (KAFKA-1733) Producer.send will block indeterminately when broker is unavailable.

2014-10-30 Thread R. Tyler Croy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190803#comment-14190803 ] R. Tyler Croy commented on KAFKA-1733: -- [~junrao], the commit referenced is on top of

Re: Setting up Kafka and Intellij

2014-10-30 Thread Jun Rao
./gradelw idea works for me. You can also try importing Kafka as a gradle project directly in Intellji. Tried this with Intellij 3 and it worked. Thanks, Jun On Tue, Oct 28, 2014 at 9:46 AM, Renato MarroquĂ­n Mogrovejo < renatoj.marroq...@gmail.com> wrote: > Hi guys, > > I have struggling throug

Re: Setting up Kafka and Intellij

2014-10-30 Thread Jun Rao
Actually, it should be intellji 13, instead of intellij 3. Thanks, Jun On Thu, Oct 30, 2014 at 1:33 PM, Jun Rao wrote: > ./gradelw idea works for me. You can also try importing Kafka as a gradle > project directly in Intellji. Tried this with Intellij 3 and it worked. > > Thanks, > > Jun > > O

Re: [DISCUSSION] Nested compression in Kafka?

2014-10-30 Thread Jun Rao
Yes, I agree that we don't need to support more than 1 level of nesting for the compressed messages. Thanks, Jun On Mon, Oct 27, 2014 at 4:27 PM, Guozhang Wang wrote: > Hello folks, > > I came across this "testComplexCompressDecompress" in > kafka.message.MessageCompressionTest while I'm worki

[jira] [Commented] (KAFKA-1733) Producer.send will block indeterminately when broker is unavailable.

2014-10-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190765#comment-14190765 ] Jun Rao commented on KAFKA-1733: Thanks for the patch. Could you generate a patch and atta

[jira] [Updated] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Vladimir Tretyakov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Tretyakov updated KAFKA-1481: -- Attachment: KAFKA-1481_2014-10-30_21-35-43.patch > Stop using dashes AND underscores as

[jira] [Updated] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Vladimir Tretyakov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Tretyakov updated KAFKA-1481: -- Attachment: (was: KAFKA-1481_2014-10-30_21-35-43.patch.patch.patch) > Stop using das

[jira] [Updated] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1634: - Attachment: KAFKA-1634.patch > Improve semantics of timestamp in OffsetCommitRequests and update

[jira] [Updated] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Vladimir Tretyakov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Tretyakov updated KAFKA-1481: -- Attachment: (was: KAFKA-1481_2014-10-24_14-14-35.patch.patch) > Stop using dashes AN

[jira] [Updated] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1634: - Status: Patch Available (was: Reopened) > Improve semantics of timestamp in OffsetCommitRequests

[jira] [Commented] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2014-10-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190587#comment-14190587 ] Guozhang Wang commented on KAFKA-1634: -- Created reviewboard https://reviews.apache.or

Review Request 27391: Fix KAFKA-1634: Add the global retentionTime (in milis) and disable the per-offset timestamp

2014-10-30 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27391/ --- Review request for kafka. Bugs: KAFKA-1634 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Vladimir Tretyakov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190581#comment-14190581 ] Vladimir Tretyakov commented on KAFKA-1481: --- Hi Jun, added new patch, removed Ta

[jira] [Updated] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2014-10-30 Thread Vladimir Tretyakov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Tretyakov updated KAFKA-1481: -- Attachment: KAFKA-1481_2014-10-30_21-35-43.patch.patch.patch > Stop using dashes AND und

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2014-10-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190568#comment-14190568 ] Jun Rao commented on KAFKA-1695: If we upgrade zkclient, we also need to pick up the chang

[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation

2014-10-30 Thread Sachin Goyal (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190556#comment-14190556 ] Sachin Goyal commented on KAFKA-1275: - [~joestein], [~jkreps], We are using Kafka 0.8.

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-10-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190490#comment-14190490 ] Joel Koshy commented on KAFKA-1647: --- Patch is good, but could you rebase? > Replication

[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown

2014-10-30 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190370#comment-14190370 ] Neha Narkhede commented on KAFKA-1738: -- [~pradeepbadiger] Do you also try to send dat

Re: Compile failure going from kafka 0.8.1.1 to 0.8.2

2014-10-30 Thread Jun Rao
Yes, we can change this to two methods in 0.8.2 final. Thanks, Jun On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps wrote: > But Jun, > > That change breaks peoples code who weren't calling in that way. I don't > think we should be making breaking changes in a point release like this. > > I think we

Re: Compile failure going from kafka 0.8.1.1 to 0.8.2

2014-10-30 Thread Colin
+1 on two methods -- Colin Clark +1-320-221-9531 > On Oct 30, 2014, at 11:24 AM, Jonathan Weeks wrote: > > +1 on the two methods suggestion > > -JW > >> On Oct 30, 2014, at 9:20 AM, Jay Kreps wrote: >> >> But Jun, >> >> That change breaks peoples code who weren't calling in that way. I

Re: Compile failure going from kafka 0.8.1.1 to 0.8.2

2014-10-30 Thread Jonathan Weeks
+1 on the two methods suggestion -JW On Oct 30, 2014, at 9:20 AM, Jay Kreps wrote: > But Jun, > > That change breaks peoples code who weren't calling in that way. I don't > think we should be making breaking changes in a point release like this. > > I think we should treat this like a bug for

Re: Compile failure going from kafka 0.8.1.1 to 0.8.2

2014-10-30 Thread Jay Kreps
But Jun, That change breaks peoples code who weren't calling in that way. I don't think we should be making breaking changes in a point release like this. I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which s

Re: Compile failure going from kafka 0.8.1.1 to 0.8.2

2014-10-30 Thread Jun Rao
Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, th

A more scalable Kafka to Hadoop InputFormat

2014-10-30 Thread Casey Green
Hi Folks, I'm open sourcing a scalable Kafka InputFormat. As far as I know or am aware of, my version is unique compared to other Kafka InputFormats out there, in that input splits are mapped to Kafka log files, rather than entire Kafka partitions. Mapping Kafka log files to input splits scal

[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep updated KAFKA-1738: --- Description: We are using Kafka Topic APIs to create the topic. But in some cases, the topic gets created but

[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep updated KAFKA-1738: --- Summary: Partitions for topic not created after restart from forced shutdown (was: Partition files not create

[jira] [Updated] (KAFKA-1738) Partition files not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep updated KAFKA-1738: --- Affects Version/s: 0.8.2 > Partition files not created after restart from forced shutdown > --

[jira] [Updated] (KAFKA-1738) Partition files not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep updated KAFKA-1738: --- Description: We are using Kafka Topic APIs to create the topic. But in some cases, the topic gets created but

[jira] [Created] (KAFKA-1738) Partition files not created after restart from forced shutdown

2014-10-30 Thread Pradeep (JIRA)
Pradeep created KAFKA-1738: -- Summary: Partition files not created after restart from forced shutdown Key: KAFKA-1738 URL: https://issues.apache.org/jira/browse/KAFKA-1738 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2014-10-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189727#comment-14189727 ] Gwen Shapira commented on KAFKA-1695: - New pull request for ZKClient: https://github.c