[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747829#comment-15747829 ] Robert Schumann commented on KAFKA-4497: [~becket_qin] [~junrao]: Many thanks for

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Apurva, Thank you for the answers. Just one follow-on. 15. Let me rephrase my original question. If all control messages (messages to transaction logs and markers on user logs) were acknowledged only after flushing the log segment, will transactions become durable in the traditional sense (i.e

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

That's a good idea. Authenticating every request with delegation token will be useful for impersonation use-cases. But as of now, we are thinking delegation token as just another way to authenticate the users. We haven't think through all the use cases related to impersonation or using delegation t

[jira] [Created] (KAFKA-4535) http://kafka.apache.org/quickstart Step 8 missing argument

Xin created KAFKA-4535: -- Summary: http://kafka.apache.org/quickstart Step 8 missing argument Key: KAFKA-4535 URL: https://issues.apache.org/jira/browse/KAFKA-4535 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-4535) http://kafka.apache.org/quickstart Step 8 missing argument

[ https://issues.apache.org/jira/browse/KAFKA-4535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin updated KAFKA-4535: --- Labels: documentation (was: ) Fix Version/s: 0.10.1.1 Status: Patch Available (was: Open) kafk

[jira] [Updated] (KAFKA-4473) RecordCollector should handle retriable exceptions more strictly

[ https://issues.apache.org/jira/browse/KAFKA-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4473: -- Status: Patch Available (was: In Progress) > RecordCollector should handle retriable exceptions more st

[GitHub] kafka pull request #2253: KAFKA-4534: StreamPartitionAssignor only ever upda...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2253 KAFKA-4534: StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics on first assignment. `partitionsByHostState` and `metadataWithInternalTopics` need to be u

[jira] [Commented] (KAFKA-4534) StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics once.

[ https://issues.apache.org/jira/browse/KAFKA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747970#comment-15747970 ] ASF GitHub Bot commented on KAFKA-4534: --- GitHub user dguy opened a pull request:

[jira] [Updated] (KAFKA-4534) StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics once.

[ https://issues.apache.org/jira/browse/KAFKA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4534: -- Status: Patch Available (was: Open) > StreamPartitionAssignor only ever updates the partitionsByHostSta

[jira] [Created] (KAFKA-4536) Kafka clients throw NullPointerException on poll when delete the relative topic

mayi_hetu created KAFKA-4536: Summary: Kafka clients throw NullPointerException on poll when delete the relative topic Key: KAFKA-4536 URL: https://issues.apache.org/jira/browse/KAFKA-4536 Project: Kafka

[jira] [Created] (KAFKA-4537) StreamPartitionAssignor incorrectly adds standby partitions to the partitionsByHostState map

Damian Guy created KAFKA-4537: - Summary: StreamPartitionAssignor incorrectly adds standby partitions to the partitionsByHostState map Key: KAFKA-4537 URL: https://issues.apache.org/jira/browse/KAFKA-4537

[jira] [Commented] (KAFKA-4536) Kafka clients throw NullPointerException on poll when delete the relative topic

[ https://issues.apache.org/jira/browse/KAFKA-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748056#comment-15748056 ] mayi_hetu commented on KAFKA-4536: -- As I followed the code, I found there is a MetadataL

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Thanks Neha. It’s a fair point. We don’t need two anymore. There isn’t much to be gained from changing the existing nomenclature, unless people feel strongly about it the word epoch. LeaderEpoch is currently fairly ingrained (spans Zookeeper, the controller, the metadata requests etc). I sugge

kafka_2.11-0.9.0.1 crash with java coredump

Hi All, I am using kafka_2.11-0.9.0.1 with java version "1.7.0_51". On random days kafka process stops (crashes) with a java coredump file as below. (gdb) bt #0 0x7f33059f70d5 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f33059fa83b in abort () from /lib/x86_64-linux-gnu/libc.s

[jira] [Created] (KAFKA-4538) Version

Samuel Durand created KAFKA-4538: Summary: Version Key: KAFKA-4538 URL: https://issues.apache.org/jira/browse/KAFKA-4538 Project: Kafka Issue Type: Bug Components: clients Affe

Re: [DISCUSS] KIP-81: Max in-flight fetches

Thanks Rajini, Before Kafka-4137, we could avoid coordinator starvation without making a special case for a special connection, but rather simply, in applying the buffer.memory check only to 'large' responses (e.g. size > 1k, possibly introducing a new config entry) in NetworkReceive.readFrom

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Thanks Onur. That’s correct, we no longer nee that extra mapping. I’ll update the KIP. B Ben Stopford Confluent, http://www.confluent.io > On 11 Dec 2016, at 23:54, Onur Karaman wrote: > > Pretty happy to see a KIP tackling this problem! One comment below. > > Th

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

To clarify slightly, the case described in the KIP doesn’t necessitate an extra mapping to the CZXID. But there is an issue filed against the controller, which would also affect the LeaderGeneration correctness. The suggested fix for this includes such a mapping, according to Jun’s reasoning in

[GitHub] kafka pull request #2254: KAFKA-4537: StreamPartitionAssignor incorrectly ad...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2254 KAFKA-4537: StreamPartitionAssignor incorrectly adds standby partitions to the partitionsByHostState map If a KafkaStreams app is using Standby Tasks then `StreamPartitionAssignor` will add the standb

[jira] [Commented] (KAFKA-4537) StreamPartitionAssignor incorrectly adds standby partitions to the partitionsByHostState map

[ https://issues.apache.org/jira/browse/KAFKA-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748147#comment-15748147 ] ASF GitHub Bot commented on KAFKA-4537: --- GitHub user dguy opened a pull request:

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

It would clearly be very useful to enable clients to send requests on behalf of multiple users. A separate KIP makes sense, but it may be worth thinking through some of the implications now, especially if the main interest in delegation tokens comes from its potential to enable impersonation. I un

[jira] [Commented] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

[ https://issues.apache.org/jira/browse/KAFKA-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748169#comment-15748169 ] Ben Stopford commented on KAFKA-4522: - It'd be good to have some empirical evidence th

[jira] [Created] (KAFKA-4539) NPE in StreamThread when creating StandbyTasks

Damian Guy created KAFKA-4539: - Summary: NPE in StreamThread when creating StandbyTasks Key: KAFKA-4539 URL: https://issues.apache.org/jira/browse/KAFKA-4539 Project: Kafka Issue Type: Bug

[jira] [Work started] (KAFKA-4539) NPE in StreamThread when creating StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4539 started by Damian Guy. - > NPE in StreamThread when creating StandbyTasks >

Re: [DISCUSS] KIP-81: Max in-flight fetches

Edo, I wouldn't introduce a new config entry, especially since you don't need it after KAFKA-4137. As a temporary measure that would work for consumers. But you probably don't want to do the same for brokers - will be worth checking with Radai since the implementation will be based on KIP-72. To d

[GitHub] kafka pull request #2255: KAFKA-4539: NPE in StreamThread when creating Stan...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2255 KAFKA-4539: NPE in StreamThread when creating StandbyTasks Tasks that don't have any `StateStore`s wont have a `StandbyTask`, so `createStandbyTask` can return `null`. We need to check for this in `St

[jira] [Commented] (KAFKA-4539) NPE in StreamThread when creating StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748306#comment-15748306 ] ASF GitHub Bot commented on KAFKA-4539: --- GitHub user dguy opened a pull request:

[jira] [Updated] (KAFKA-4539) NPE in StreamThread when creating StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4539: -- Status: Patch Available (was: In Progress) > NPE in StreamThread when creating StandbyTasks > -

[jira] [Commented] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748324#comment-15748324 ] Rajini Sivaram commented on KAFKA-4538: --- If you are using KafkaConsumer, this could

[jira] [Commented] (KAFKA-4536) Kafka clients throw NullPointerException on poll when delete the relative topic

[ https://issues.apache.org/jira/browse/KAFKA-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748359#comment-15748359 ] Rajini Sivaram commented on KAFKA-4536: --- This is fixed in 0.10.1.0. > Kafka clients

[GitHub] kafka pull request #2253: KAFKA-4534: StreamPartitionAssignor only ever upda...

Github user dguy closed the pull request at: https://github.com/apache/kafka/pull/2253 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[jira] [Commented] (KAFKA-4534) StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics once.

[ https://issues.apache.org/jira/browse/KAFKA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748376#comment-15748376 ] ASF GitHub Bot commented on KAFKA-4534: --- Github user dguy closed the pull request at

[jira] [Commented] (KAFKA-4534) StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics once.

[ https://issues.apache.org/jira/browse/KAFKA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748380#comment-15748380 ] ASF GitHub Bot commented on KAFKA-4534: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #2256: KAFKA-4534: StreamPartitionAssignor only ever upda...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2256 KAFKA-4534: StreamPartitionAssignor only ever updates the partitionsByHostState and metadataWithInternalTopics on first assignment partitionsByHostState and metadataWithInternalTopics need to be update

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

Hi Rajini, I think it would definitely be valuable to have a KIP for impersonation. Ismael On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram wrote: > It would clearly be very useful to enable clients to send requests on > behalf of multiple users. A separate KIP makes sense, but it may be worth

Re: [DISCUSS] KIP-83 - Allow multiple SASL PLAIN authenticated Java clients in a single JVM process

Hi Edoardo, Yes, you're right, this can be just a JIRA now as there are no publicly facing changes. Thanks, Ismael On Tue, Dec 13, 2016 at 9:07 AM, Edoardo Comar wrote: > Thanks for your review, Ismael. > > First, I am no longer sure KIP-83 is worth keeping as KIP, I created it > just before R

[jira] [Commented] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748422#comment-15748422 ] Samuel Durand commented on KAFKA-4538: -- [~rsivaram] it might , if I don't find the ti

[jira] [Comment Edited] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748422#comment-15748422 ] Samuel Durand edited comment on KAFKA-4538 at 12/14/16 2:13 PM:

[jira] [Resolved] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4538. Resolution: Duplicate Duplicate of KAFKA-4431, please reopen if not. > Version > > >

[jira] [Updated] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4431: --- Fix Version/s: 0.10.1.1 > HeartbeatThread should be a daemon thread >

[jira] [Commented] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748569#comment-15748569 ] Ismael Juma commented on KAFKA-4431: I cherry-picked this to the 0.10.1 branch as it's

[jira] [Commented] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748574#comment-15748574 ] Ismael Juma commented on KAFKA-4538: I cherry-picked KAFKA-4431 into the 0.10.1 branch

[jira] [Commented] (KAFKA-4474) Poor kafka-streams throughput

[ https://issues.apache.org/jira/browse/KAFKA-4474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748575#comment-15748575 ] Juan Chorro commented on KAFKA-4474: Hi again! We have been doing more performance te

[jira] [Comment Edited] (KAFKA-4474) Poor kafka-streams throughput

[ https://issues.apache.org/jira/browse/KAFKA-4474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748575#comment-15748575 ] Juan Chorro edited comment on KAFKA-4474 at 12/14/16 3:17 PM: --

Re: [DISCUSS] KIP-86: Configurable SASL callback handlers

Have added sample callback handlers for PLAIN and SCRAM. On Tue, Dec 13, 2016 at 4:10 PM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > Ismael, > > Thank you for the review. I will add an example. > > On Tue, Dec 13, 2016 at 1:07 PM, Ismael Juma wrote: > >> Hi Rajini, >> >> Thanks for

log.retention attribute not working

Hi there, Any idea why log.retention attribute is not working? We kept log.retention.hours=6 in server.properties but we see old data are not getting deleted. We see Dec 9th data/log files are still there. We are running this in production boxes and if it does not delete the old files our stor

Re: [DISCUSS] KIP-86: Configurable SASL callback handlers

Thanks Rajini, that helps. A few comments: 1. The `AuthCallbackHandler` interface already exists and we are making breaking changes (removing a parameter from `configure` and adding additional methods). Is the reasoning that it was not a public interface before? It would be good to clearly separat

[jira] [Updated] (KAFKA-4539) StreamThread is not correctly creating StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4539: -- Summary: StreamThread is not correctly creating StandbyTasks (was: NPE in StreamThread when creating S

[jira] [Updated] (KAFKA-4539) StreamThread is not correctly creating StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4539: -- Description: Fails because {{createStandbyTask(..)}} can return null fi the topology for the {{TaskId}}

New question / request on kafka consumer

Hello, We are trying to make a consumer for kafka, (client code alone and camel integrated) and we ended in a blocking point. Can you please give us an advice, or any other idea? Our problem: - We create a kafka consumer that connects to a wrong server (wrong ip/port), and the consume

[jira] [Commented] (KAFKA-4474) Poor kafka-streams throughput

[ https://issues.apache.org/jira/browse/KAFKA-4474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748741#comment-15748741 ] Eno Thereska commented on KAFKA-4474: - [~jjchorrobe] can I confirm again that this is

[jira] [Commented] (KAFKA-4538) Version

[ https://issues.apache.org/jira/browse/KAFKA-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748743#comment-15748743 ] Samuel Durand commented on KAFKA-4538: -- thanks for the info [~ijuma] will follow that

[jira] [Commented] (KAFKA-4180) Shared authentication with multiple active Kafka producers/consumers

[ https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748758#comment-15748758 ] Edoardo Comar commented on KAFKA-4180: -- I have updated the PR based on the discussion

Re: [DISCUSS] KIP-83 - Allow multiple SASL PLAIN authenticated Java clients in a single JVM process

Hi Ismael, I have retired the KIP and updated the PR. Can't wait for https://issues.apache.org/jira/browse/KAFKA-4259 to be merged :-) -- Edoardo Comar IBM MessageHub eco...@uk.ibm.com IBM UK Ltd, Hursley Park, SO21 2JN IBM United Kingdom Limited R

[jira] [Commented] (KAFKA-4487) Tests should be run in Jenkins with INFO or DEBUG level

[ https://issues.apache.org/jira/browse/KAFKA-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748788#comment-15748788 ] Colin P. McCabe commented on KAFKA-4487: Good idea. +1 for Jenkins running tests

Re: Kafka on Ubuntu Core and Snap store

Hi Michael, "snapcraft" sounds like an interesting idea. I'm curious what you see as the advantage over Docker images. Does the snap file contain the Java runtime? Kafka, like most JDK projects, already bundles its library dependencies internally, so I don't think snapcraft is adding quite as m

[jira] [Created] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

Damian Guy created KAFKA-4540: - Summary: Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created Key: KAFKA-4540 URL: https://issues.apache.org/jira/

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

@Gwen @Mani Not sure why we want to authenticate at every request. Even if the token exchange is cheap it still a few calls that need to go through round trip. Impersonation doesn't require authentication for every request. "So a centralized app can create few producers, do the metadata request

Build failed in Jenkins: kafka-0.10.1-jdk7 #107

See Changes: [ismael] KAFKA-4431; Make consumer heartbeat thread a daemon thread -- [...truncated 14289 lines...] org.apache.kafka.streams.state.internals.CompositeReadOnlyKeyValueStoreTest > s

Re: [DISCUSS] KIP-82 - Add Record Headers

arent control messages getting pushed as their own top level protocol change (and a fairly massive one) for the transactions KIP ? On Tue, Dec 13, 2016 at 5:54 PM, Matthias J. Sax wrote: > Hi, > > I want to add a completely new angle to this discussion. For this, I > want to propose an extension

[jira] [Commented] (KAFKA-4529) tombstone may be removed earlier than it should

[ https://issues.apache.org/jira/browse/KAFKA-4529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748966#comment-15748966 ] Jun Rao commented on KAFKA-4529: Committed [~becket_qin]'s patch to 0.10.1 branch. Leaving

Re: [DISCUSS] KIP-82 - Add Record Headers

Yes and no. I did overload the term "control message". EOS control messages are for client-broker communication and thus never exposed to any application. And I think this is a good design because broker needs to understand those control messages. Thus, this should be a protocol change. The type

[jira] [Assigned] (KAFKA-3714) Allow users greater access to register custom streams metrics

[ https://issues.apache.org/jira/browse/KAFKA-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3714: --- Assignee: Eno Thereska (was: Guozhang Wang) > Allow users greater access to register custom

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749077#comment-15749077 ] Tom DeVoe commented on KAFKA-4477: -- Regarding 2 - I restarted the host which shrank all o

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Hi, Onur, The reason for keeping track of the CZXID of the broker registration path is the following. There is one corner case bug (KAFKA-1120) that Ben mentioned where the controller could miss a ZK watcher event if the broker deregisters and registers quickly. Always triggering a leader election

[jira] [Assigned] (KAFKA-4531) Rationalise client configuration validation

[ https://issues.apache.org/jira/browse/KAFKA-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-4531: -- Assignee: Vahid Hashemian > Rationalise client configuration validation >

[jira] [Commented] (KAFKA-4531) Rationalise client configuration validation

[ https://issues.apache.org/jira/browse/KAFKA-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749120#comment-15749120 ] Vahid Hashemian commented on KAFKA-4531: [~ecomar] It would be nice to centralize

Re: kafka_2.11-0.9.0.1 crash with java coredump

I would suggest creating a JIRA and describing in detail what was going on in the cluster when this happened, and posting the associated broker / state change / controller logs. Thanks, Apurva On Wed, Dec 14, 2016 at 3:28 AM, Mazhar Shaikh wrote: > Hi All, > > I am using kafka_2.11-0.9.0.1 with

Re: kafka_2.11-0.9.0.1 crash with java coredump

Hi Mazhar, That looks like a G1 GC bug. Lots of bug in G1 have been fixed since JDK 7 update 51. I highly recommend that you upgrade to a more recent JDK (JDK 8 recommended). Alternatively you could try the CMS GC. Ismael On Wed, Dec 14, 2016 at 3:28 AM, Mazhar Shaikh wrote: > Hi All, > > I am

[jira] [Commented] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749174#comment-15749174 ] David Judd commented on KAFKA-4431: --- Awesome, thanks! > HeartbeatThread should be a dae

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Ben, You are are right on both counts: Writing apps to do consume-process-produce batching will be tricky to program using this API directly. The expectation is that 99% of the users would use the streams API to leverage this functionality, and that API will take care of the details. This seem

Jenkins build is back to normal : kafka-0.10.1-jdk7 #108

See

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Rajini, I think my original response to your point 15 was not accurate. The regular definition of durability is that data once committed would never be lost. So it is not enough for only the control messages to be flushed before being acknowledged -- all the messages (and offset commits) which

[DISCUSS] Control Messages - [Was: KIP-82 - Add Record Headers]

I'm renaming this thread in case we start deep diving. I'm in favor of so called "control messages", at least the notion of those. However, I'm not sure about the design. What I understood from the original mail: A. Provide a message that does not get returned by poll() B. Provide a way for app

Re: kafka_2.11-0.9.0.1 crash with java coredump

I would suggest trying a recent java version first, if I read about this one: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-December/011534.html Cheers Robert -- Robert Schumann | Lead DevOps Engineer | mobile.de GmbH T: + 49. 30. 8109. 7219 M: +49.151. 5504. 8246 F: +49. 30. 8109.

Brokers cashing with OOME Map failed

Recently, we have seen our brokers crash with below errors, any idea what might be wrong here? The brokers have been running for long with the same hosts/configs without this issue before. Is this something to do with new version 0.10.0.1 (which we upgraded recently) or could it be a h/w issue?

[GitHub] kafka pull request #2257: MINOR: Reenable streams smoke test

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/2257 MINOR: Reenable streams smoke test I ran it 3 times and it works again. You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka min

[jira] [Commented] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

[ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749465#comment-15749465 ] Guozhang Wang commented on KAFKA-4540: -- Is this already resolved as part of https://

[GitHub] kafka pull request #2257: MINOR: Reenable streams smoke test

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2257 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4528) Failure in kafka.producer.ProducerTest.testAsyncSendCanCorrectlyFailWithTimeout

[ https://issues.apache.org/jira/browse/KAFKA-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749479#comment-15749479 ] Guozhang Wang commented on KAFKA-4528: -- Another observed failure: https://builds.apa

[jira] [Work stopped] (KAFKA-4437) Incremental Batch Processing for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4437 stopped by Matthias J. Sax. -- > Incremental Batch Processing for Kafka Streams > --

[jira] [Commented] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

[ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749549#comment-15749549 ] Damian Guy commented on KAFKA-4540: --- This issue isn't resolved. The problem here is that

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

With regard to a), just using `ConnectRecord` with `newRecord` as a new abstract method would be a fine choice. In prototyping, both options end up looking pretty similar (in terms of how transformations are implemented and the runtime initializes and uses them) and I'm starting to lean towards not

[jira] [Commented] (KAFKA-1521) Producer Graceful Shutdown issue in Container (Kafka version 0.8.x.x)

[ https://issues.apache.org/jira/browse/KAFKA-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749639#comment-15749639 ] Anish Khanzode commented on KAFKA-1521: --- This is really a problem when kafka consume

[jira] [Commented] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

[ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749636#comment-15749636 ] Matthias J. Sax commented on KAFKA-4540: This makes sense. The problem is that {{

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749722#comment-15749722 ] Apurva Mehta commented on KAFKA-4477: - Hi [~tdevoe] (and other as well), Would it be

[jira] [Updated] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens

[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-1696: -- Issue Type: New Feature (was: Sub-task) Parent: (was: KAFKA-1682) > Kafka should be

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749750#comment-15749750 ] Ismael Juma commented on KAFKA-4477: One clarification: if you use the `log4j.properti

[jira] [Created] (KAFKA-4541) Add capability to create delegation token

Ashish K Singh created KAFKA-4541: - Summary: Add capability to create delegation token Key: KAFKA-4541 URL: https://issues.apache.org/jira/browse/KAFKA-4541 Project: Kafka Issue Type: Sub-tas

Build failed in Jenkins: kafka-trunk-jdk8 #1103

See Changes: [wangguoz] MINOR: Reenable streams smoke test -- [...truncated 17361 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldNotGetAllTasksWithStoreWhenNotRunning PASSED org.a

[jira] [Created] (KAFKA-4544) Add capability to renew/expire delegation tokens.

Ashish K Singh created KAFKA-4544: - Summary: Add capability to renew/expire delegation tokens. Key: KAFKA-4544 URL: https://issues.apache.org/jira/browse/KAFKA-4544 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-4543) Add capability to renew/expire delegation tokens.

Ashish K Singh created KAFKA-4543: - Summary: Add capability to renew/expire delegation tokens. Key: KAFKA-4543 URL: https://issues.apache.org/jira/browse/KAFKA-4543 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-4544) Add system tests for delegation token based authentication

[ https://issues.apache.org/jira/browse/KAFKA-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-4544: -- Summary: Add system tests for delegation token based authentication (was: Add capability to ren

[jira] [Updated] (KAFKA-4544) Add system tests for delegation token based authentication

[ https://issues.apache.org/jira/browse/KAFKA-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-4544: -- Description: Add system tests for delegation token based authentication. (was: Add capability t

[jira] [Assigned] (KAFKA-4541) Add capability to create delegation token

[ https://issues.apache.org/jira/browse/KAFKA-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh reassigned KAFKA-4541: - Assignee: Ashish K Singh > Add capability to create delegation token > --

[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens

[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749769#comment-15749769 ] Ashish K Singh commented on KAFKA-1696: --- [~omkreddy] I had to convert this JIRA from

[jira] [Created] (KAFKA-4542) Add authentication based on delegation token.

Ashish K Singh created KAFKA-4542: - Summary: Add authentication based on delegation token. Key: KAFKA-4542 URL: https://issues.apache.org/jira/browse/KAFKA-4542 Project: Kafka Issue Type: Sub

[GitHub] kafka pull request #2258: MINOR: update KStream JavaDocs

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2258 MINOR: update KStream JavaDocs You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka minorKStreamJavaDoc Alternatively you can review and a

[jira] [Commented] (KAFKA-3856) Move inner classes accessible only functions in TopologyBuilder out of public APIs

[ https://issues.apache.org/jira/browse/KAFKA-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15750030#comment-15750030 ] Matthias J. Sax commented on KAFKA-3856: [~jeyhunkarimov] Are you still working on

[jira] [Commented] (KAFKA-3135) Unexpected delay before fetch response transmission

[ https://issues.apache.org/jira/browse/KAFKA-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15750082#comment-15750082 ] Jeff Widman commented on KAFKA-3135: Can the tags on this issue be updated to note tha

  1   2   >