[GitHub] kafka pull request #2559: KAFKA-4775. Fix findbugs warnings in kafka-tools

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2559 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4775) Fix findbugs warnings in kafka-tools

[ https://issues.apache.org/jira/browse/KAFKA-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871548#comment-15871548 ] ASF GitHub Bot commented on KAFKA-4775: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4775) Fix findbugs warnings in kafka-tools

[ https://issues.apache.org/jira/browse/KAFKA-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4775. Resolution: Fixed Fix Version/s: 0.10.3.0 Issue resolved by pull request 2559 [https://github

[jira] [Commented] (KAFKA-4676) Kafka consumers gets stuck for some partitions

[ https://issues.apache.org/jira/browse/KAFKA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871619#comment-15871619 ] Narendra Kumar commented on KAFKA-4676: --- We are also facing the same issue. [~onur

[GitHub] kafka pull request #2544: KFKA-4340: follow up patch for KAFAK-4340

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2544 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4753) KafkaConsumer susceptible to FetchResponse starvation

[ https://issues.apache.org/jira/browse/KAFKA-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871679#comment-15871679 ] Ismael Juma commented on KAFKA-4753: [~onurkaraman], are you intending to submit a PR

Re: [DISCUSS] KIP-82 - Add Record Headers

On the point re: headers in the message protocol being a byte array and not a count of elements followed by the elements. Again this was discussed/argued previously. It was agreed on for a few reasons some of which you have obviously picked up on: Broker is able to pass it through opaquely The

Re: [DISCUSS] KIP-82 - Add Record Headers

On the point of varInts Would you be proposing in KIP-98 to convert the other message int’s (key length, value length) also to varint to keep it uniform. Also I assume there will be a static or helper method made to write/read these in the client and server. Cheers Mike On 17/02/2017, 11:22,

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

+0 I think need some unified agreement on the VarInts. Would this also change in all other area’s of the protocol, e.g. value and key length in message protocol, to keep this uniform across all protocols going forwards? On 17/02/2017, 00:23, "Apurva Mehta" wrote: Hi Jun, Thanks fo

[jira] [Commented] (KAFKA-4767) KafkaProducer is not joining its IO thread properly

[ https://issues.apache.org/jira/browse/KAFKA-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871783#comment-15871783 ] Ismael Juma commented on KAFKA-4767: [~bgedik], `this.sender.forceClose();` is "some c

Kafka Connect / Access to OffsetStorageReader from SourceConnector

Hi Kafka Team, I'm developping a connector which need to monitor the progress of its tasks in order to be able to request a tasks reconfiguration in some situations. Our connector is pretty simple. It's used to stream a thousands of files into Kafka. The connector scans directories then schedules

Build failed in Jenkins: kafka-trunk-jdk8 #1281

See Changes: [ismael] KAFKA-4775; Fix findbugs warnings in kafka-tools -- [...truncated 33781 lines...] kafka.security.auth.SimpleAclAuthorizerTest > testHighConcurrencyModificationOfResourceAc

Build failed in Jenkins: kafka-trunk-jdk7 #1947

See Changes: [ismael] KAFKA-4340; Follow-up fixing system test failures and handling non -- [...truncated 15968 lines...] org.apache.kafka.clients.producer.internals.RecordAccumulatorTest > test

Build failed in Jenkins: kafka-trunk-jdk8 #1282

See Changes: [ismael] KAFKA-4340; Follow-up fixing system test failures and handling non -- [...truncated 4136 lines...] kafka.security.auth.SimpleAclAuthorizerTest > testLoadCache STARTED kaf

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Hi, according to the feedback, I've updated the KIP: - We have added and ordered the scenarios, scopes and executions of the Reset Offset tool. - Consider it as an extension to the current `ConsumerGroupCommand` tool - Execution will be possible without generating JSON files. https://cwiki.apach

Re: documentation feedback

Hi Brian, Have you created a pull request for this? best, Colin On Thu, Feb 9, 2017, at 15:21, Brian Cornally wrote: > @apachekafka suggested addition to > http://kafka.apache.org/documentation/#security - section 5 - Examples > using console-producer and console-consumer: > > echo "ssl" | /u

[jira] [Commented] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872116#comment-15872116 ] Colin P. McCabe commented on KAFKA-4754: bq. Its worth noting, it was also possibl

Would kafka support delay queue in the future

Hello, Some MQ products could support sending the message with ttl property; after ttl timeout, MQ could publish the message to the specified queue. I want to know whether kafka would support the feature in the future. Best Regards, Steven.zhang vipshop 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨

[jira] [Comment Edited] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872116#comment-15872116 ] Colin P. McCabe edited comment on KAFKA-4754 at 2/17/17 5:02 PM: ---

[DISCUSS] KIP-124: Request rate quotas

Hi all, I have just created KIP-124 to introduce request rate quotas to Kafka: https://cwiki.apache.org/confluence/display/KAFKA/KIP-124+-+Request+rate+quotas The proposal is for a simple percentage request handling time quota that can be allocated to **, ** or **. There are a few other suggesti

[jira] [Commented] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872127#comment-15872127 ] Grant Henke commented on KAFKA-4754: {quote} Hmm. It is not a good practice to pass pa

[jira] [Commented] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872139#comment-15872139 ] Grant Henke commented on KAFKA-4754: {quote} This could expose the password to anyone

[jira] [Updated] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4757: --- Summary: NetworkClient should log request details at trace level when a request is cancelled b

[jira] [Assigned] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe reassigned KAFKA-4757: -- Assignee: Colin P. McCabe > NetworkClient should log request details at trace level whe

[jira] [Commented] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872176#comment-15872176 ] Colin P. McCabe commented on KAFKA-4757: bq. 1. The [request cancelled] log at one

Re: [DISCUSS] KIP-124: Request rate quotas

Hey Rajini, Thanks for the KIP. I have some questions: - I am wondering why throttling based on request rate is listed as a rejected alternative. Can you provide more specific reason why it is difficult for administrators to decide request rates to allocate? It seems to me that determination of r

[jira] [Commented] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872194#comment-15872194 ] Onur Karaman commented on KAFKA-4757: - Thanks [~cmccabe]. Regarding the second issue,

Re: [DISCUSS] KIP-124: Request rate quotas

To correct the typo above: It seems to me that determination of request rate is not any more difficult than determination of *byte* rate as both metrics are commonly used to measure performance and provide guarantee to user. On Fri, Feb 17, 2017 at 9:40 AM, Dong Lin wrote: > Hey Rajini, > > Than

[jira] [Commented] (KAFKA-4676) Kafka consumers gets stuck for some partitions

[ https://issues.apache.org/jira/browse/KAFKA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872199#comment-15872199 ] Onur Karaman commented on KAFKA-4676: - [~Narendra Kumar] if you're repeatedly seeing t

[jira] [Commented] (KAFKA-4676) Kafka consumers gets stuck for some partitions

[ https://issues.apache.org/jira/browse/KAFKA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872213#comment-15872213 ] Onur Karaman commented on KAFKA-4676: - This ticket never really found a root cause so

Re: [DISCUSS] KIP-82 - Add Record Headers

> > Would you be proposing in KIP-98 to convert the other message int’s (key > length, value length) also to varint to keep it uniform. > Also I assume there will be a static or helper method made to write/read > these in the client and server. Yes, that is what we are proposing, so using varints

[jira] [Commented] (KAFKA-4715) Consumer/Producer config does not work with related enums

[ https://issues.apache.org/jira/browse/KAFKA-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872219#comment-15872219 ] Colin P. McCabe commented on KAFKA-4715: Hmm. It seems better to override the {{n

Re: get function

Hi Samy, If I understand your question correctly, you're looking for a way to retrieve a value based on a corresponding key. That would require you to build an index. Kafka can look up records based on the record time, but by default will not build any kind of index based on the record key. In

Re: [DISCUSS] KIP-124: Request rate quotas

Overall a big fan of the KIP. I'd have to agree with Dong. I'm not sure about the decision of using the percentage over the window as opposed to request rate. It's pretty hard to reason about. I just spoke to one of our SRE's and he agrees. Also I may have missed it, but I couldn't find informati

[jira] [Commented] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872228#comment-15872228 ] Colin P. McCabe commented on KAFKA-4754: Sorry, I guess I was unclear... I definit

[jira] [Comment Edited] (KAFKA-4754) Correctly parse '=' characters in command line overrides

[ https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872228#comment-15872228 ] Colin P. McCabe edited comment on KAFKA-4754 at 2/17/17 6:15 PM: ---

Re: [DISCUSS] KIP-82 - Add Record Headers

Re headers available on the record va interceptors only One of the key use cases requires access on consume at per event/message level at the point that message is being processed, as such the batch interceptors and batch consume api isn't suitable. It needs to be at the record level. This any

Request Wiki edit permission for cshannon

Hi, I would like to contribute a KIP. Can someone give me access to the wiki to create pages? Thanks, Chris

[jira] [Created] (KAFKA-4777) Kafka client Heartbeat thread use all the cpu.

Allen Xiang created KAFKA-4777: -- Summary: Kafka client Heartbeat thread use all the cpu. Key: KAFKA-4777 URL: https://issues.apache.org/jira/browse/KAFKA-4777 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-82 - Add Record Headers

> > Re headers as byte array and future use by broker. This doesn't take away > from that at all. Nor makes it difficult at all in my opinion. Yeah, I didn't say it was difficult, only awkward. You wouldn't write the schema that way if you were planning to use it on the brokers from the beginning

Re: [DISCUSS] KIP-82 - Add Record Headers

Sorry, my mistake. The consumer interceptor is per batch, though I'm not sure that's an actual limitation since you still have access to the individual records. -Jason On Fri, Feb 17, 2017 at 10:39 AM, Jason Gustafson wrote: > Re headers as byte array and future use by broker. This doesn't take

[jira] [Commented] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872284#comment-15872284 ] Onur Karaman commented on KAFKA-4757: - [~cmccabe] I double checked and the 2nd logging

Re: Request to add to contributor list

Done, cheers. Guozhang On Thu, Feb 16, 2017 at 11:01 PM, Shun Takebayashi wrote: > Hi dev team, > > I want to be added as a contributor in Kafka JIRA. > My JIRA ID is takebayashi > > Thanks, > > Shun > -- -- Guozhang

Re: [DISCUSS] KIP-82 - Add Record Headers

Big +1 on VarInts. CPUs are fast, memory is slow. I agree with Jason that we'll want to continue verifying messages, including their headers, so while I appreciate the idea of the opaque header blob it won't be useful in practice. /Magnus 2017-02-17 10:41 GMT-08:00 Jason Gustafson : > Sorry, my

[GitHub] kafka pull request #2563: Kafka 4757: NetworkClient should log request detai...

GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/2563 Kafka 4757: NetworkClient should log request details at trace level when a request is cancelled because of disconnection You can merge this pull request into a Git repository by running: $ gi

[jira] [Commented] (KAFKA-4757) NetworkClient should log request details at trace level when a request is cancelled because of disconnection

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872390#comment-15872390 ] Onur Karaman commented on KAFKA-4757: - Ah. Might've found it: Here's what ApiVersionsR

Re: Request Wiki edit permission for cshannon

Done. Guozhang On Fri, Feb 17, 2017 at 10:20 AM, Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > Hi, > > I would like to contribute a KIP. Can someone give me access to the wiki > to create pages? > > Thanks, > Chris > -- -- Guozhang

[jira] [Updated] (KAFKA-4757) Improve NetworkClient trace logging of request details

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4757: --- Summary: Improve NetworkClient trace logging of request details (was: NetworkClient should lo

Re: [DISCUSS] KIP-82 - Add Record Headers

Yes exactly we have access to the records thus why the header should be accessible via it and not hidden for only interceptors to access. Sent using OWA for iPhone From: Magnus Edenhill Sent: Friday, February 17, 2017 7:34:49 PM To: dev@kafka.apache.org S

Re: Request Wiki edit permission for cshannon

Awesome, thanks. On Fri, Feb 17, 2017 at 2:30 PM, Guozhang Wang wrote: > Done. > > Guozhang > > On Fri, Feb 17, 2017 at 10:20 AM, Christopher Shannon < > christopher.l.shan...@gmail.com> wrote: > > > Hi, > > > > I would like to contribute a KIP. Can someone give me access to the wiki > > to cre

[jira] [Commented] (KAFKA-4757) Improve NetworkClient trace logging of request details

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872406#comment-15872406 ] Colin P. McCabe commented on KAFKA-4757: Thanks. That's a good find. I just post

Re: [DISCUSS] KIP-82 - Add Record Headers

I am happy to move the definition of the header into the message body, but would cause us not to lazy initialise/parse the headers, as obviously, we would have to traverse these reading the message. This was actually one of Jay’s requests: “2. I think we should think about creating the Map

[GitHub] kafka pull request #2564: KAFKA-4777 fix client heartbeat non-stop retry iss...

GitHub user allenxiang opened a pull request: https://github.com/apache/kafka/pull/2564 KAFKA-4777 fix client heartbeat non-stop retry issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/allenxiang/kafka client-heartbeat-fix

[jira] [Commented] (KAFKA-4777) Kafka client Heartbeat thread use all the cpu.

[ https://issues.apache.org/jira/browse/KAFKA-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872435#comment-15872435 ] ASF GitHub Bot commented on KAFKA-4777: --- GitHub user allenxiang opened a pull reques

Re: [DISCUSS] KIP-82 - Add Record Headers

The point about creation of maps seems orthogonal. We can still represent the headers as a slice of bytes until the time it is accessed. > Yes exactly we have access to the records thus why the header should be > accessible via it and not hidden for only interceptors to access. As explained abo

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

@Manikumar, Can you give an example how you are planning to use PrincipalBuilder? @Jun Yes, that is right. To give a brief overview, we just extract the cert and hand it over to a third party library for creating a Principal. So we cannot create a Principal from just a string. The main motive beh

Re: [DISCUSS] KIP-82 - Add Record Headers

What’s the issue with exposing a method getHeaders on the producer/consumer record? It doesn’t break anything. We don’t need any special version. Current batch consumer model and consumer interceptors don’t work where headers need to be acted on at per message level at time of processing, very c

[jira] [Commented] (KAFKA-4715) Consumer/Producer config does not work with related enums

[ https://issues.apache.org/jira/browse/KAFKA-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872493#comment-15872493 ] Mathias Kub commented on KAFKA-4715: Overriding any method coming from Enum seems bad

Re: [DISCUSS] KIP-82 - Add Record Headers

Re: “The point about creation of maps seems orthogonal. We can still represent the headers as a slice of bytes until the time it is accessed.” That’s exactly what we’re doing the headers are a slice of bytes, which then gets parsed later if needed, or can be parsed right away, the header

Re: [DISCUSS] KIP-82 - Add Record Headers

> > That’s exactly what we’re doing the headers are a slice of bytes, which > then gets parsed later if needed, or can be parsed right away, the headers > is part of the protocol, so can still be validated if wanted. > If you had a header count then you would have to go through each header > key an

Re: [DISCUSS] KIP-82 - Add Record Headers

For APM single event tracing, need access to the header at the point of processing on the processing thread. As such interceptors will not work/be suitable for these, due to the fact they act on the ConsumerRecords as a batch, before the handling thread can split out and process per message whi

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi, Mani, Good point on using PrincipalBuilder for SASL. It seems that PrincipalBuilder already has access to Authenticator. So, we could just enable that in SaslChannelBuilder. We probably could do that in a separate KIP? Hi, Mayuresh, If you don't think there is a concrete use case for using P

[jira] [Updated] (KAFKA-4759) Add support for subnet masks in SimpleACLAuthorizer

[ https://issues.apache.org/jira/browse/KAFKA-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shun Takebayashi updated KAFKA-4759: Assignee: Shun Takebayashi Status: Patch Available (was: Open) > Add support for subn

[jira] [Commented] (KAFKA-4757) Improve NetworkClient trace logging of request details

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872616#comment-15872616 ] Onur Karaman commented on KAFKA-4757: - Left some comments on the PR. Speaking of whic

[GitHub] kafka pull request #2565: MINOR: Use scale of 3 in streams benchmark by defa...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/2565 MINOR: Use scale of 3 in streams benchmark by default [DO NOT MERGE] You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka KM

[jira] [Commented] (KAFKA-4757) Improve NetworkClient trace logging of request details

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872643#comment-15872643 ] Colin P. McCabe commented on KAFKA-4757: https://github.com/apache/kafka/pull/2563

[jira] [Commented] (KAFKA-4385) producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false

[ https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872645#comment-15872645 ] Stevo Slavic commented on KAFKA-4385: - [~Jun Yao] [~ewencp] it would open up more opti

[jira] [Commented] (KAFKA-4757) Improve NetworkClient trace logging of request details

[ https://issues.apache.org/jira/browse/KAFKA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872650#comment-15872650 ] Colin P. McCabe commented on KAFKA-4757: Thanks, [~onurkaraman]. I usually do nam

Re: Should InvalidTopicException be called IllegalTopicException

Hmm. I'm not sure that "illegal" is that different from "invalid" semantically. If an argument is illegal for a given function, you could also say that argument is invalid for that function. Also, changing exception names could be frustrating to users from a compatibility perspective. Perhaps s

[GitHub] kafka pull request #2513: MINOR: Remove Struct from Request/Response classes

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2513 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4715) Consumer/Producer config does not work with related enums

[ https://issues.apache.org/jira/browse/KAFKA-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872664#comment-15872664 ] Colin P. McCabe commented on KAFKA-4715: bq. Beside that the name() method is fina

[jira] [Commented] (KAFKA-4317) RocksDB checkpoint files lost on kill -9

[ https://issues.apache.org/jira/browse/KAFKA-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872676#comment-15872676 ] ASF GitHub Bot commented on KAFKA-4317: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2471: KAFKA-4317: Regularly checkpoint StateStore change...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2471 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-4317) RocksDB checkpoint files lost on kill -9

[ https://issues.apache.org/jira/browse/KAFKA-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4317: - Resolution: Fixed Fix Version/s: 0.10.3.0 Status: Resolved (was: Patch Available

[jira] [Created] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

Dave Thomas created KAFKA-4778: -- Summary: OOM on kafka-streams instances with high numbers of unreaped Record classes Key: KAFKA-4778 URL: https://issues.apache.org/jira/browse/KAFKA-4778 Project: Kafka

[jira] [Updated] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

[ https://issues.apache.org/jira/browse/KAFKA-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Thomas updated KAFKA-4778: --- Description: We have a stream processing app with ~8 source/sink stages operating roughly at the rate

[jira] [Updated] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

[ https://issues.apache.org/jira/browse/KAFKA-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Thomas updated KAFKA-4778: --- Attachment: oom-killer.txt OOM-killer kernel message is attached. > OOM on kafka-streams instances wi

[GitHub] kafka pull request #2566: HOTFIX: ClassCastException in requrest logging

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2566 HOTFIX: ClassCastException in requrest logging You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka hotfix-request-logging Alterna

[GitHub] kafka pull request #2566: HOTFIX: ClassCastException in request logging

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2566 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Jun, Sure sounds good to me. Thanks, Mayuresh On Fri, Feb 17, 2017 at 1:54 PM, Jun Rao wrote: > Hi, Mani, > > Good point on using PrincipalBuilder for SASL. It seems that > PrincipalBuilder already has access to Authenticator. So, we could just > enable that in SaslChannelBuilder. We proba

Jenkins build is back to normal : kafka-trunk-jdk7 #1949

See

[GitHub] kafka pull request #2567: MINOR: Increase consumer init timeout in throttlin...

GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/2567 MINOR: Increase consumer init timeout in throttling test The throttling system test sometimes fail because it takes longer than the current 10 second time out for partitions to get assigned to the c

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Jun, I have updated the KIP. Would you mind taking another look? Thanks, Mayuresh On Fri, Feb 17, 2017 at 4:42 PM, Mayuresh Gharat wrote: > Hi Jun, > > Sure sounds good to me. > > Thanks, > > Mayuresh > > On Fri, Feb 17, 2017 at 1:54 PM, Jun Rao wrote: > >> Hi, Mani, >> >> Good point on u

[jira] [Created] (KAFKA-4779) Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py

Apurva Mehta created KAFKA-4779: --- Summary: Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py Key: KAFKA-4779 URL: https://issues.apache.org/jira/browse/KAFKA-4779 Project: Kaf

[jira] [Commented] (KAFKA-4779) Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py

[ https://issues.apache.org/jira/browse/KAFKA-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872862#comment-15872862 ] Apurva Mehta commented on KAFKA-4779: - [~rsivaram] do you have an idea about what coul

Build failed in Jenkins: kafka-trunk-jdk8 #1284

See Changes: [jason] HOTFIX: ClassCastException in request logging -- [...truncated 8436 lines...] kafka.server.DynamicConfigChangeTest > testDefaultUserQuotaConfigChange STARTED kafka.server.

[jira] [Commented] (KAFKA-2319) After controlled shutdown: IllegalStateException: Kafka scheduler has not been started

[ https://issues.apache.org/jira/browse/KAFKA-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872967#comment-15872967 ] Michał Woś commented on KAFKA-2319: --- I've got the same problem after SIGTERM. Moreover:

[jira] [Comment Edited] (KAFKA-2319) After controlled shutdown: IllegalStateException: Kafka scheduler has not been started

[ https://issues.apache.org/jira/browse/KAFKA-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872967#comment-15872967 ] Michał Woś edited comment on KAFKA-2319 at 2/18/17 3:40 AM: I'

[jira] [Comment Edited] (KAFKA-2319) After controlled shutdown: IllegalStateException: Kafka scheduler has not been started

[ https://issues.apache.org/jira/browse/KAFKA-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872967#comment-15872967 ] Michał Woś edited comment on KAFKA-2319 at 2/18/17 3:41 AM: I'

Re: Kafka Connect / Access to OffsetStorageReader from SourceConnector

Hey Florian, Can you explain a bit more how having access to the offset storage from the connector helps in your use case? I guess you are planning to use offsets to be able to tell when a task has finished a file? Thanks, Jason On Fri, Feb 17, 2017 at 4:45 AM, Florian Hussonnois wrote: > Hi K