Re: [DISCUSS] KIP-939: Support Participation in 2PC

t; with a single producer. > > -Artem > > On Wed, Aug 23, 2023 at 4:50 PM Roger Hoover > wrote: > > > Thanks. I like that you're moving Kafka toward supporting this > dual-write > > pattern. Each use case needs to consider the tradeoffs. You already > &

Re: [DISCUSS] KIP-939: Support Participation in 2PC

assessment. Not necessarily exactly single-threaded per > application, but a single producer per thread model (i.e. an application > could have a pool of threads + producers to increase concurrency). > > -Artem > > On Tue, Aug 22, 2023 at 7:22 PM Roger Hoover > wrote: > > &g

Re: [DISCUSS] KIP-939: Support Participation in 2PC

ansaction timeout interval. > Or > > are we saying that when used in this 2PC context, we should configure > these > > transaction timeouts to very large durations? > > > > Thanks in advance! > > > > Best, > > Arjun > > > > > > On Mo

Re: [DISCUSS] KIP-939: Support Participation in 2PC

Hi Artem, Thanks for writing this KIP. Can you clarify the requirements a bit more for managing transaction state? It looks like the application must have stable transactional ids over time? What is the granularity of those ids and producers? Say the application is a multi-threaded Java web s

Re: [VOTE] KIP-602: Change default value for client.dns.lookup

+1 (non-binding) Thanks, Badai. On Fri, May 22, 2020 at 10:05 AM Ismael Juma wrote: > Thanks for the KIP, +1 (binding). > > Ismael > > On Fri, May 22, 2020 at 1:40 AM Badai Aqrandista > wrote: > > > Hi All, > > > > I would like to start the vote on KIP-602: Change default value for > > client.

Re: [DISCUSS] KIP-196: Add metrics to Kafka Connect framework

s that we feel would be super useful for debugging? > > On Tue, Sep 12, 2017 at 6:08 PM, Roger Hoover > wrote: > > > Thanks, Ewen. > > > > I agree with you on the overhead of measuring time for SMTs and > > converters. I'd still argue for keeping other me

Re: [DISCUSS] KIP-196: Add metrics to Kafka Connect framework

ly provide a bit more detail > in some ways (e.g. join & sync vs the entire rebalance). Not sure if we > want to effectively duplicate some info so it can all be located under > Connect names or rely on the existing metrics for some of these. > > -Ewen > > On Tue, Sep 12,

Re: [DISCUSS] KIP-196: Add metrics to Kafka Connect framework

ate them from different workers. Removing them > probably > > > > also > > > > > reduces the risk of them being misinterpretted. > > > > > > > > > > > > > > >> * You have status for connectors but not for tasks. Any reason

Re: [DISCUSS] KIP-196: Add metrics to Kafka Connect framework

ld find this useful or not. Probably not for alerts, but > > might > > >> be useful to be able to check it via your metrics dashboard. > > >> > > > > > > Seems like it might be useful, at least in terms of tracking the number > > of

Re: [DISCUSS] KIP-196: Add metrics to Kafka Connect framework

Randall, Thank you for the KIP. This should improve visibility greatly. I had a few questions/ideas for more metrics. 1. What's the relationship between the worker state and the connector status? Does the 'paused' status at the Connector level include the time that worker is 'rebalan

Re: [DISCUSS] KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy

Edoardo, thanks for the KIP. I think it's a good idea overall. +1 especially for including Session/Principal in the API. (#2 mentioned by Ismael) Also, the AlterPolicy should get the same info as create + delete (#4). Cheers, Roger On Thu, Sep 7, 2017 at 8:43 AM, Ismael Juma wrote: > Hi To

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

smaller impact > on reducing replication latency. But it would definitely be interesting to > get those numbers and see whether the theory bears out in practice. > > Regards, > Apurva > > On Thu, Aug 31, 2017 at 8:56 AM, Roger Hoover > wrote: > > > Sorry, my math was sloppy.

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

5:18 PM, Roger Hoover wrote: > Sorry if this is a bit out of left field but can't help wondering... > > One way to improve producer performance while still having good guarantees > would be to allow a setting between acks=1 and acks=all. We could > introduce "acks

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

Sorry if this is a bit out of left field but can't help wondering... One way to improve producer performance while still having good guarantees would be to allow a setting between acks=1 and acks=all. We could introduce "acks=minIsr". This is already the guarantee you get when the ISR set shrink

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

know if you have any > concerns about the new field. > > Thank you... > > Regards, > > Rajini > > > > On Tue, Aug 29, 2017 at 8:44 PM, Roger Hoover > wrote: > > > Hi Rajini, > > > > One more thought. Maybe we should also add an error_message

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

personation not allowed", or "You account has been locked, please contact cluster admin". Thanks, Roger On Tue, Aug 29, 2017 at 12:41 PM, Roger Hoover wrote: > Hi Rajini, > > The metrics in KIP-188 will provide counts across all users but the log > could potentially

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

really need info-level logging for > authentication? At the moment, there don't seem to be any per-connection > informational messages at info-level, but if you think it is useful, we > could do this in a separate JIRA. Let me know what you think. > > On Tue, Aug 29, 2017 at 1:09 PM

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

Just re-read the KIP and was wondering if you think INFO would be ok for logging successful authentications? They should be relatively infrequent. On Tue, Aug 29, 2017 at 9:54 AM, Roger Hoover wrote: > +1 (non-binding). Thanks, Rajini > > On Tue, Aug 29, 2017 at 2:10 AM, Ismael Ju

Re: [DISCUSS] KIP-188 - Add new metrics to support health checks

r > > > > > > > this, > > > > > > > > > we > > > > > > > > > > could easily add a long expiration period to be > > conservative, > > > > > but I > > > > > > > am > > > >

Re: [VOTE] KIP-152 - Improve diagnostics for SASL authentication failures

+1 (non-binding). Thanks, Rajini On Tue, Aug 29, 2017 at 2:10 AM, Ismael Juma wrote: > Thanks for the KIP, +1 (binding) from me. > > Ismael > > On Thu, Aug 24, 2017 at 6:29 PM, Rajini Sivaram > wrote: > > > Hi all, > > > > I would like to start vote on KIP-152 to improve diagnostics of > > aut

Re: [DISCUSS] KIP-188 - Add new metrics to support health checks

Rajini, The table is super helpful. Thank you. On Thu, Aug 17, 2017 at 2:16 AM, Rajini Sivaram wrote: > Hi Roger, > > Thank you for the review. I have added a table with the scope of errors > counted for each request. > > Regards, > > Rajini > > On Thu, Aug 17,

Re: [DISCUSS] KIP-187 - Add cumulative count metric for all Kafka rate metrics

Rajini, Thank you. This is very useful. Grouping by metric by prefixing the name instead of making them MBeans is not quite as nice but seems like an good compromise for backward compatibility. Cheers, Roger On Wed, Aug 16, 2017 at 5:35 AM, Rajini Sivaram wrote: > Sorry, pressed send by mis

Re: [DISCUSS] KIP-188 - Add new metrics to support health checks

t requests? Cheers, Roger On Wed, Aug 16, 2017 at 4:02 PM, Roger Hoover wrote: > Rajini, > > Thank you for the KIP. These are very helpful additions. One question on > the error code metrics: > > Will the total error counting happen at the the level of topic partition

Re: [DISCUSS] KIP-188 - Add new metrics to support health checks

Rajini, Thank you for the KIP. These are very helpful additions. One question on the error code metrics: Will the total error counting happen at the the level of topic partition? For example, if a single ProduceRequest contains messages to append to 3 partitions and say all 3 appends are succes

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Well deserved, indeed! Congrats, Ismael. On Wed, Jul 5, 2017 at 3:24 PM, Damian Guy wrote: > Congratulations Ismael! Very well deserved. > Cheers, > Damian > On Wed, 5 Jul 2017 at 22:54, Dong Lin wrote: > > > Congratulations Ismael! > > > > On Wed, Jul 5, 2017 at 1:55 PM, Jun Rao wrote: > > >

Re: [VOTE] KIP-153 (separating replication traffic from BytesOutPerSec metric)

+1 Sent from my iPhone > On May 8, 2017, at 5:00 AM, Edoardo Comar wrote: > > +1 > Many thanks Jun > -- > Edoardo Comar > IBM MessageHub > eco...@uk.ibm.com > IBM UK Ltd, Hursley Park, SO21 2JN > > IBM United Kingdom Limited Registered in Englan

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

Very helpful. Thank you, Jun. On Fri, May 5, 2017 at 4:42 PM, Guozhang Wang wrote: > Jun, > > Thanks for the KIP, LGTM. > > Guozhang > > On Fri, May 5, 2017 at 3:38 PM, Ismael Juma wrote: > > > Thanks Jun, looks good to me. > > > > Ismael > > > > On Fri, May 5, 2017 at 11:35 PM, Jun Rao wrote

[jira] [Commented] (KAFKA-3795) Transient system test failure upgrade_test.TestUpgrade

[ https://issues.apache.org/jira/browse/KAFKA-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953759#comment-15953759 ] Roger Hoover commented on KAFKA-3795: - Happened again: http://confluent-systes

Re: [DISCUSS] KIP-136: Add Listener name and Security Protocol name to SelectorMetrics tags

Edo, Thanks for the proposal. This looks great to me. Cheers, Roger On Thu, Mar 30, 2017 at 8:51 AM, Edoardo Comar wrote: > Hi all, > > We created KIP-136: Add Listener name and Security Protocol name to > SelectorMetrics tags > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 136%

[jira] [Updated] (KAFKA-4755) SimpleBenchmark consume test fails for streams

[ https://issues.apache.org/jira/browse/KAFKA-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-4755: Priority: Blocker (was: Major) > SimpleBenchmark consume test fails for stre

[jira] [Reopened] (KAFKA-4755) SimpleBenchmark consume test fails for streams

[ https://issues.apache.org/jira/browse/KAFKA-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover reopened KAFKA-4755: - Happened again. Logs here: http://confluent-kafka-0-10-2-system-test-results.s3-us-west-2

[jira] [Commented] (KAFKA-4689) OffsetValidationTest fails validation with "Current position greater than the total number of consumed records"

[ https://issues.apache.org/jira/browse/KAFKA-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945543#comment-15945543 ] Roger Hoover commented on KAFKA-4689: - Happened again. Logs here http://confl

Re: [VOTE] KIP-124: Request rate quotas

Rajini, This is great. Thank you. +1 (non-binding) Roger On Tue, Mar 21, 2017 at 8:55 AM, Ismael Juma wrote: > Rajini, > > Thanks for the proposal and for addressing the (sometimes contradictory) > feedback. :) +1 (binding) from me. > > Ismael > > On Mon, Mar 20, 2017 at 2:47 PM, Rajini Siva

Re: [DISCUSS] KIP-124: Request rate quotas

Great to see this KIP and the excellent discussion. To me, Jun's suggestion makes sense. If my application is allocated 1 request handler unit, then it's as if I have a Kafka broker with a single request handler thread dedicated to me. That's the most I can use, at least. That allocation doesn'

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

.840.1.101.3.4.2.2 [RFC4055] > > "sha-512" 2.16.840.1.101.3.4.2.3 [RFC4055] > > > https://www.iana.org/assignments/hash-function- > text-names/hash-function-text-names.xhtml > > As you see, bcrypt is not an option for the current spec. The naming of the >

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

d for impersonation related discussion. > > Thanks, > Manikumar > > On Wed, Feb 8, 2017 at 8:37 AM, Roger Hoover > wrote: > > > Hi Jun, > > > > How does it allow impersonation at the connection level? Looking at the > > KIP, the DelegationTokenReq

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

t; connection level. Are you talking about impersonation at the request level? > > Thanks, > > Jun > > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover > wrote: > > > Just wondering...how difficult would be it be to later add impersonation > ( > > https://issues.apache.or

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Just wondering...how difficult would be it be to later add impersonation ( https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be a Kafka admin UI that would take action on the cluster on behalf different users.I suppose we could later add an "effectiveUserId" (in Unix termin

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

This is great. Thanks, Ismael. On Fri, Feb 3, 2017 at 7:35 AM, Grant Henke wrote: > Looks good to me. Thanks for handling the KIP. > > On Fri, Feb 3, 2017 at 8:49 AM, Damian Guy wrote: > > > Thanks Ismael. Makes sense to me. > > > > On Fri, 3 Feb 2017 at 10:39 Ismael Juma wrote: > > > > > Hi

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

M-SHA-512 (not > SCRAM-SHA-1). > > Ismael > > On Mon, Jan 23, 2017 at 10:49 PM, Roger Hoover > wrote: > > > Sorry for the late question but is there a reason to choose SHA-1 and > > SHA-256 instead of bcrypt? > > > > https://codahale.com/how-to-safely-stor

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Sorry for the late question but is there a reason to choose SHA-1 and SHA-256 instead of bcrypt? https://codahale.com/how-to-safely-store-a-password/ On Fri, Nov 11, 2016 at 5:30 AM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > I think all the comments and suggestions on this thread h

[jira] [Closed] (KAFKA-4670) Kafka Consumer should validate FetchResponse

[ https://issues.apache.org/jira/browse/KAFKA-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover closed KAFKA-4670. --- > Kafka Consumer should validate FetchRespo

[jira] [Commented] (KAFKA-4670) Kafka Consumer should validate FetchResponse

[ https://issues.apache.org/jira/browse/KAFKA-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830670#comment-15830670 ] Roger Hoover commented on KAFKA-4670: - Ah, yes, thanks, [~ijuma]. > Kafka C

[jira] [Created] (KAFKA-4670) Kafka Consumer should validate FetchResponse

Roger Hoover created KAFKA-4670: --- Summary: Kafka Consumer should validate FetchResponse Key: KAFKA-4670 URL: https://issues.apache.org/jira/browse/KAFKA-4670 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-108: Create Topic Policy

+1 (non-binding) On Mon, Jan 9, 2017 at 2:15 AM, Edoardo Comar wrote: > Ismael, > thanks for the KIP I see it as quite useful in a managed cloud > environment. > > +1 (non-binding) > -- > Edoardo Comar > IBM MessageHub > eco...@uk.ibm.com > IBM UK

Re: [DISCUSS] KIP-108: Create Topic Policy

his explicitly in the KIP. > > Ismael > > On Mon, Jan 9, 2017 at 6:04 PM, Roger Hoover > wrote: > > > This is great. Thanks, Ismael. > > > > One question. When TopicDetails are passed to the policy implementation, > > would the server defaults already hav

Re: [DISCUSS] KIP-108: Create Topic Policy

This is great. Thanks, Ismael. One question. When TopicDetails are passed to the policy implementation, would the server defaults already have been merged? If not, I think the policy also needs access to the server defaults. Cheers, Roger On Fri, Jan 6, 2017 at 9:26 AM, Ismael Juma wrote:

Re: KIP-103: Separation of Internal and External traffic

l > >> On Fri, Jan 6, 2017 at 9:42 PM, Roger Hoover wrote: >> >> Ismael, >> >> Listener id would also convey uniqueness but I'm ok with listener key as >> well since it fits with the use of the term "map" in other properties. >> &

Re: KIP-103: Separation of Internal and External traffic

r > listener, so I prefer it a little over `listener_name`. Since the existing > config is called `listeners` instead of `protocol.listeners`, maybe we > don't need the protocol prefix? > > Ismael > >> On Fri, Jan 6, 2017 at 7:48 PM, Roger Hoover wrote: >> >

Re: KIP-103: Separation of Internal and External traffic

Maybe it's clearer to to say protocol_listener_name? The proposed config allows you to name each listener and refer to their names in various places. On Wed, Jan 4, 2017 at 4:34 AM, Ismael Juma wrote: > Hi Colin, > > Thanks for the feedback. It's a good question regarding the name `protocol >

Re: [VOTE] KIP-103: Separation of Internal and External traffic

+1 (non-binding) On Fri, Jan 6, 2017 at 11:16 AM, Tom Crayford wrote: > +1 (non-binding) > > On Fri, Jan 6, 2017 at 6:58 PM, Colin McCabe wrote: > > > Looks good. +1 (non-binding). > > > > What do you think about changing "protocol label" to "listener key"? > > > > best, > > Colin > > > > > >

[jira] [Commented] (KAFKA-4527) Transient failure of ConnectDistributedTest.test_pause_and_resume_sink where paused connector produces messages

[ https://issues.apache.org/jira/browse/KAFKA-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15759269#comment-15759269 ] Roger Hoover commented on KAFKA-4527: - Happened again: http://confluent-systes

[jira] [Commented] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

[ https://issues.apache.org/jira/browse/KAFKA-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15759266#comment-15759266 ] Roger Hoover commented on KAFKA-3808: - Happened again: http://confluent-systes

[jira] [Commented] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15759264#comment-15759264 ] Roger Hoover commented on KAFKA-4166: - Similar failure: http://confluent-systes

[jira] [Commented] (KAFKA-4526) Transient failure in ThrottlingTest.test_throttled_reassignment

[ https://issues.apache.org/jira/browse/KAFKA-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15759260#comment-15759260 ] Roger Hoover commented on KAFKA-4526: - Thanks, [~apurva] > Transient fai

Re: [DISCUSS] KIP-82 - Add Record Headers

Matthias, Thanks for your input. I'm +1 on control messages as they seem to be the simplest way to implement watermarks ( https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102), a feature that would add a lot of value to Kafka Streams IMHO. Your argument that the control-message ind

[jira] [Commented] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

[ https://issues.apache.org/jira/browse/KAFKA-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15758169#comment-15758169 ] Roger Hoover commented on KAFKA-3808: - Happened again: {

[jira] [Resolved] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover resolved KAFKA-4554. - Resolution: Duplicate Duplicate of https://issues.apache.org/jira/browse/KAFKA-3808

[jira] [Created] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

Roger Hoover created KAFKA-4554: --- Summary: ReplicaVerificationToolTest.test_replica_lags system test failure Key: KAFKA-4554 URL: https://issues.apache.org/jira/browse/KAFKA-4554 Project: Kafka

[jira] [Commented] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15758158#comment-15758158 ] Roger Hoover commented on KAFKA-4166: - Failed again: http://confluent-systes

[jira] [Commented] (KAFKA-4526) Transient failure in ThrottlingTest.test_throttled_reassignment

[ https://issues.apache.org/jira/browse/KAFKA-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15758155#comment-15758155 ] Roger Hoover commented on KAFKA-4526: - Failed again {

[jira] [Created] (KAFKA-4551) StreamsSmokeTest.test_streams intermittent failure

Roger Hoover created KAFKA-4551: --- Summary: StreamsSmokeTest.test_streams intermittent failure Key: KAFKA-4551 URL: https://issues.apache.org/jira/browse/KAFKA-4551 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754984#comment-15754984 ] Roger Hoover commented on KAFKA-4166: - Happened again here: http://confl

[jira] [Commented] (KAFKA-4527) Transient failure of ConnectDistributedTest.test_pause_and_resume_sink where paused connector produces messages

[ https://issues.apache.org/jira/browse/KAFKA-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754980#comment-15754980 ] Roger Hoover commented on KAFKA-4527: - Happened again: http://confluent-systes

[jira] [Commented] (KAFKA-4526) Transient failure in ThrottlingTest.test_throttled_reassignment

[ https://issues.apache.org/jira/browse/KAFKA-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754982#comment-15754982 ] Roger Hoover commented on KAFKA-4526: - Happened again here: http://confl

[jira] [Comment Edited] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747454#comment-15747454 ] Roger Hoover edited comment on KAFKA-4166 at 12/14/16 6:3

[jira] [Comment Edited] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747454#comment-15747454 ] Roger Hoover edited comment on KAFKA-4166 at 12/14/16 6:3

[jira] [Commented] (KAFKA-4166) TestMirrorMakerService.test_bounce transient system test failure

[ https://issues.apache.org/jira/browse/KAFKA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747454#comment-15747454 ] Roger Hoover commented on KAFKA-4166: - It happened twice more on these test: {

[jira] [Commented] (KAFKA-4526) Transient failure in ThrottlingTest.test_throttled_reassignment

[ https://issues.apache.org/jira/browse/KAFKA-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747308#comment-15747308 ] Roger Hoover commented on KAFKA-4526: - This happened again on the Dec 13 nightly

Re: [DISCUSS] KIP-82 - Add Record Headers

the same thing - > youre just namespacing an int64 and giving people while 2^32 ranges at a > time. the part i like about this is letting people have a large swath of > numbers with one registration so they dont have to come back for every > single plugin/header they want to "r

Re: [DISCUSS] KIP-82 - Add Record Headers

;>>> down to int16(2bytes) it saves on space and as headers > we > > > >> wouldn't > > > >> >> >>>>>> expect > > > >> >> >>>>>>>> the number of headers being used concurrently being that > > > high.

Re: [DISCUSS] KIP-82 - Add Record Headers

he > > > > ecosystem, having to have this also customizable makes it a bit > harder. > > > > Think about making the whole message record custom serializable, this > > > would > > > > make it fairly tricky (though it would not be impossible) to have > ma

Re: [DISCUSS] KIP-82 - Add Record Headers

rate from Apache Kafka > or it could be part of Apache Kafka. The people that would like to use > headers can use that client, and the people that think it's an overhead can > use the one without. > > > Nacho > > > On Mon, Nov 7, 2016 at 2:54 PM, Roger Hoover > w

Re: [DISCUSS] KIP-82 - Add Record Headers

;1/20" someone who uses (contrived example) Map Double> wouldnt be able to re-use my code. > > not the end of a the world within an organization, but problematic if you > want to enable an ecosystem > > On Thu, Nov 3, 2016 at 2:04 PM, Roger Hoover > wrote: > > > As

Re: [DISCUSS] KIP-82 - Add Record Headers

dermines > > the > > board usefulness of the feature (any point along the path must be > able > > to > > read the header keys. the values may be whatever and require more > > intimate > > knowledge of the code that produced specific headers

Re: [DISCUSS] KIP-82 - Add Record Headers

e a > large message splitter/combiner - if i rely on key "largeMessage" and > values of the form "1/20" someone who uses (contrived example) > Map Double> wouldnt be able to re-use my code. > > not the end of a the world within an organizati

Re: [DISCUSS] KIP-82 - Add Record Headers

As others have laid out, I see strong reasons for a common message metadata structure for the Kafka ecosystem. In particular, I've seen that even within a single organization, infrastructure teams often own the message metadata while application teams own the application-level data format. Allow

[jira] [Updated] (KAFKA-4361) Streams does not respect user configs for "default" params

[ https://issues.apache.org/jira/browse/KAFKA-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-4361: Description: For the config params in CONSUMER_DEFAULT_OVERRIDES (https://github.com/apache/kafka

[jira] [Updated] (KAFKA-4361) Streams does not respect user configs for "default" params

[ https://issues.apache.org/jira/browse/KAFKA-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-4361: Description: For the config params in CONSUMER_DEFAULT_OVERRIDES (https://github.com/apache/kafka

[jira] [Updated] (KAFKA-4361) Streams does not respect user configs for "default" params

[ https://issues.apache.org/jira/browse/KAFKA-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-4361: Description: For the config params in CONSUMER_DEFAULT_OVERRIDES (https://github.com/apache/kafka

[jira] [Created] (KAFKA-4361) Streams does not respect user configs for "default" params

Roger Hoover created KAFKA-4361: --- Summary: Streams does not respect user configs for "default" params Key: KAFKA-4361 URL: https://issues.apache.org/jira/browse/KAFKA-4361 Project: Kafka

Re: [VOTE] Add REST Server to Apache Kafka

-1 I think a good set of HTTP APIs can simplify client implementations and allow for easier integration with Kafka as well as simpler ops/admin tools. Elasticsearch is a great example. However, I think it's too early to put this into the core. The existing Kafka REST proxies (that I know of) ar

[jira] [Created] (KAFKA-4331) Kafka Streams resetter is slow because it joins the same group for each topic

Roger Hoover created KAFKA-4331: --- Summary: Kafka Streams resetter is slow because it joins the same group for each topic Key: KAFKA-4331 URL: https://issues.apache.org/jira/browse/KAFKA-4331 Project

[jira] [Commented] (KAFKA-3993) Console producer drops data

[ https://issues.apache.org/jira/browse/KAFKA-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450054#comment-15450054 ] Roger Hoover commented on KAFKA-3993: - Thanks, [~cotedm]. I tried to set acks

[jira] [Closed] (KAFKA-4063) Add support for infinite endpoints for range queries in Kafka Streams KV stores

[ https://issues.apache.org/jira/browse/KAFKA-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover closed KAFKA-4063. --- The JIRA UI was unresponsive so I accidentally submitted the form twice. > Add support for infin

[jira] [Created] (KAFKA-4063) Add support for infinite endpoints for range queries in Kafka Streams KV stores

Roger Hoover created KAFKA-4063: --- Summary: Add support for infinite endpoints for range queries in Kafka Streams KV stores Key: KAFKA-4063 URL: https://issues.apache.org/jira/browse/KAFKA-4063 Project

[jira] [Created] (KAFKA-4064) Add support for infinite endpoints for range queries in Kafka Streams KV stores

Roger Hoover created KAFKA-4064: --- Summary: Add support for infinite endpoints for range queries in Kafka Streams KV stores Key: KAFKA-4064 URL: https://issues.apache.org/jira/browse/KAFKA-4064 Project

[jira] [Commented] (KAFKA-3993) Console producer drops data

[ https://issues.apache.org/jira/browse/KAFKA-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415811#comment-15415811 ] Roger Hoover commented on KAFKA-3993: - Thanks, [~vahid]. Yeah, it looks the

[jira] [Updated] (KAFKA-3993) Console producer drops data

[ https://issues.apache.org/jira/browse/KAFKA-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-3993: Description: The console producer drops data when if the process exits too quickly. I suspect

[jira] [Commented] (KAFKA-3752) Provide a way for KStreams to recover from unclean shutdown

[ https://issues.apache.org/jira/browse/KAFKA-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404439#comment-15404439 ] Roger Hoover commented on KAFKA-3752: - [~guozhang] You're assessment seem

[jira] [Created] (KAFKA-3993) Console producer drops data

Roger Hoover created KAFKA-3993: --- Summary: Console producer drops data Key: KAFKA-3993 URL: https://issues.apache.org/jira/browse/KAFKA-3993 Project: Kafka Issue Type: Bug Components

Re: [DISCUSS] Client Side Auto Topic Creation

My comments go a bit beyond just topic creation but I'd like to see Kafka make it easier for application developers to specify their requirements declaratively in a single place. Today, for example, if your application requires strong guarantees against data loss, you must set a mix of topic-level

[DISCUSS] KAFKA-3761: Controller has RunningAsBroker instead of RunningAsController state

Hi all, Does anyone have an issue with removing the broker state called "RunningAsController"? The reasons to remove it are: 1. It's currently broken. The purpose of the JIRA was to report that the RunningAsController state gets overwritten back

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Thanks, Eno. On Tue, Jun 21, 2016 at 2:22 AM, Eno Thereska wrote: > Hi Roger, > > I realised I never got back to you on this one, sorry. Some answers inline: > > > On 3 Jun 2016, at 22:48, Roger Hoover wrote: > > > > Hi Eno, > > > > Does this mean t

[jira] [Commented] (KAFKA-3740) Add configs for RocksDBStores

[ https://issues.apache.org/jira/browse/KAFKA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334812#comment-15334812 ] Roger Hoover commented on KAFKA-3740: - [~h...@pinterest.com], any update on

[jira] [Created] (KAFKA-3858) Add functions to print stream topologies

Roger Hoover created KAFKA-3858: --- Summary: Add functions to print stream topologies Key: KAFKA-3858 URL: https://issues.apache.org/jira/browse/KAFKA-3858 Project: Kafka Issue Type: New Feature

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Eno, Does this mean that Kafka Streams will disable the RocksDB write buffer? Is it currently safe to recover a Kafka Streams application after SIGKILL on the same machine? If not, will this make it safe to do so? If RocksDB is not flushed before offsets are commited in Kafka and is killed w

[jira] [Work started] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3761 started by Roger Hoover. --- > Controller has RunningAsBroker instead of RunningAsController st

[jira] [Issue Comment Deleted] (KAFKA-3760) Set broker state as running after publishing to ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Hoover updated KAFKA-3760: Comment: was deleted (was: Create PR: https://github.com/apache/kafka/pull/1436) > Set bro

[jira] [Commented] (KAFKA-3760) Set broker state as running after publishing to ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303141#comment-15303141 ] Roger Hoover commented on KAFKA-3760: - Create PR: https://github.com/apache/k

  1   2   >