[jira] [Issue Comment Deleted] (KAFKA-3450) Producer blocks on send to topic that doesn't exist if auto create is disabled

[ https://issues.apache.org/jira/browse/KAFKA-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hangzhao updated KAFKA-3450: Comment: was deleted (was: 44) > Producer blocks on send to topic that doesn't exist if auto create is disa

[jira] [Commented] (KAFKA-3450) Producer blocks on send to topic that doesn't exist if auto create is disabled

[ https://issues.apache.org/jira/browse/KAFKA-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15650041#comment-15650041 ] hangzhao commented on KAFKA-3450: - 44 > Producer blocks on send to topic that doesn't exi

Re: [DISCUSS] KIP-82 - Add Record Headers

While you can do whatever you want with a namespace and your code, what I'd expect is for each app to namespaces configurable... So if I accidentally used 666 for my HR department, and still want to run RadaiApp, I can config "namespace=42" for RadaiApp and everything will look normal. This means

Re: [DISCUSS] KIP-82 - Add Record Headers

and we can start with {namespace, id} and no re-mapping support and always add it later on if/when collisions actually happen (i dont think they'd be a problem). every interested party (so orgs or individuals) could then register a prefix (0 = reserved, 1 = confluent ... 666 = me :-) ) and do what

Re: [DISCUSS] KIP-82 - Add Record Headers

Yes, you got it right. With MirrorMaker's event handlers (which we'll need to update to support headers) you can make sure the header namespaces will change when you replicate between clusters so 0x01 will become 0x57 when Mirrored. I'm not a huge fan, but definitely an option. If you choose not

Re: [DISCUSS] KIP-82 - Add Record Headers

> On Nov 8, 2016, at 5:54 PM, Gwen Shapira wrote: > > Thank you so much for this clear and fair summary of the arguments. > > I'm in favor of ints. Not a deal-breaker, but in favor. > > Even more in favor of Magnus's decentralized suggestion with Roger's > tweak: add a namespace for headers.

Re: [DISCUSS] KIP-82 - Add Record Headers

A local namespace mapping from namespace ids to ints would definitely solve the problem of having a global namespace and would make the int header keys potentially more readable for logging and debugging purposes. But this means another (potentially very large) set of configuration parameters that

[GitHub] kafka pull request #2118: MINOR: improve exception message for incompatible ...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2118 MINOR: improve exception message for incompatible Serdes to actual key/value data types You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kaf

Re: [DISCUSS] KIP-82 - Add Record Headers

"...whether the broker would ever want/need to act on non-standard headers..." core broker code? no. never (which is why i suggested lazily parsing the headers on 1st call from a bytebuffer slice() somewhere earlier in the discussion). broker-side plugins however ... so here's something off the

[jira] [Updated] (KAFKA-4393) Improve invalid/negative TS handling

[ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4393: --- Description: Currently, Kafka Streams does not handle invalid/negative timestamps returned fr

Re: [DISCUSS] KIP-82 - Add Record Headers

+1 for {namespace, plugin}, would definitely cut down on registration overhead. On Tue, Nov 8, 2016 at 5:54 PM, Gwen Shapira wrote: > Thank you so much for this clear and fair summary of the arguments. > > I'm in favor of ints. Not a deal-breaker, but in favor. > > Even more in favor of Magnus's

[jira] [Commented] (KAFKA-4393) Improve invalid/negative TS handling

[ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649605#comment-15649605 ] ASF GitHub Bot commented on KAFKA-4393: --- GitHub user mjsax opened a pull request:

[GitHub] kafka pull request #2117: KAFKA-4393: Improve invalid/negative TS handling

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2117 KAFKA-4393: Improve invalid/negative TS handling You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka kafka-4393-improveInvalidTsHandling

[jira] [Updated] (KAFKA-4393) Improve invalid/negative TS handling

[ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4393: --- Description: Currently, Kafka Streams does not handle invalid/negative timestamps returned fr

Re: [DISCUSS] KIP-82 - Add Record Headers

Thank you so much for this clear and fair summary of the arguments. I'm in favor of ints. Not a deal-breaker, but in favor. Even more in favor of Magnus's decentralized suggestion with Roger's tweak: add a namespace for headers. This will allow each app to just use whatever IDs it wants internall

[jira] [Commented] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649493#comment-15649493 ] ASF GitHub Bot commented on KAFKA-4362: --- GitHub user MayureshGharat opened a pull re

[GitHub] kafka pull request #2116: KAFKA-4362 : Consumer can fail after reassignment ...

GitHub user MayureshGharat opened a pull request: https://github.com/apache/kafka/pull/2116 KAFKA-4362 : Consumer can fail after reassignment of the offsets topic partition You can merge this pull request into a Git repository by running: $ git pull https://github.com/Mayures

Re: [DISCUSS] KIP-82 - Add Record Headers

Sorry I didn't read the KIP carefully enough and thought there was more difference between 5a and 5c. I now see that in 5a, the headers section is already defined as a sub-protocol that (I assume) does not need to be parsed at the broker. The main difference, as pointed out, is whether the broker

[jira] [Commented] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649485#comment-15649485 ] Mayuresh Gharat commented on KAFKA-4362: I did some more testing while reproducing

Re: [ANNOUNCE] New committer: Jiangjie (Becket) Qin

congratulations! ) ( ) ( (^)(^)(^)(^) _i__i__i__i_ () ||>o<|###| () On Mon, Nov 7, 2016 at 6:03 AM, Michael Noll wrote: > Congratulations, Becket! > > Best wishes, > Michael > > On Thu, Nov 3, 2016 at 5:13 PM, Efe Gencer wrote: > > > Congr

Jenkins build is back to normal : kafka-trunk-jdk7 #1678

See

Re: [VOTE] KIP-72 - Allow putting a bound on memory consumed by Incoming requests

+1 (binding) On Tue, Nov 8, 2016 at 10:26 AM, radai wrote: > I've updated the KIP page to specify the new config would co-exist with > "queued.max.request" to minimize the impact on compatibility. > > On Tue, Nov 8, 2016 at 7:02 AM, radai wrote: > >> My personal opinion on this is that control o

Re: [DISCUSS] KIP-82 - Add Record Headers

+1 for sean's document. it covers pretty much all the trade-offs and provides concrete figures to argue about :-) (nit-picking - used the same xkcd twice, also trove has been superceded for purposes of high performance collections: look at https://github.com/leventov/Koloboke) so to sum up the str

[jira] [Commented] (KAFKA-4381) Add per partition lag metric to KafkaConsumer.

[ https://issues.apache.org/jira/browse/KAFKA-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649280#comment-15649280 ] Ewen Cheslack-Postava commented on KAFKA-4381: -- Yeah, I don't want things to

[jira] [Comment Edited] (KAFKA-3901) KStreamTransformValues$KStreamTransformValuesProcessor#process() forwards null values

[ https://issues.apache.org/jira/browse/KAFKA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649270#comment-15649270 ] Dmitry Minkovsky edited comment on KAFKA-3901 at 11/9/16 12:08 AM: -

[jira] [Commented] (KAFKA-3901) KStreamTransformValues$KStreamTransformValuesProcessor#process() forwards null values

[ https://issues.apache.org/jira/browse/KAFKA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649270#comment-15649270 ] Dmitry Minkovsky commented on KAFKA-3901: - Also affects 0.10.1.0 <3 > KStreamTran

Build failed in Jenkins: kafka-trunk-jdk8 #1028

See Changes: [me] MINOR: some trace logging for streams debugging -- [...truncated 3875 lines...] kafka.integration.SaslPlaintextTopicMetadataTest > testIsrAfterBrokerShutDownAndJoinsBack STAR

Re: [DISCUSS] KIP-82 - Add Record Headers

I think it's well known I've been pushing for ints (and I could switch to 16 bit shorts if pressed). - efficient (space) - efficient (processing) - easily partitionable However, if the only thing that is keeping us from adopting headers is the use of strings vs ints as keys, then I would cave in

[jira] [Commented] (KAFKA-4381) Add per partition lag metric to KafkaConsumer.

[ https://issues.apache.org/jira/browse/KAFKA-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649233#comment-15649233 ] Guozhang Wang commented on KAFKA-4381: -- To me, KIP is designed to solicit broader fee

[jira] [Issue Comment Deleted] (KAFKA-4364) Sink tasks expose secrets in DEBUG logging

[ https://issues.apache.org/jira/browse/KAFKA-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan P updated KAFKA-4364: -- Comment: was deleted (was: https://github.com/apache/kafka/pull/2115) > Sink tasks expose secrets in DEBUG logg

[jira] [Commented] (KAFKA-4364) Sink tasks expose secrets in DEBUG logging

[ https://issues.apache.org/jira/browse/KAFKA-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649205#comment-15649205 ] Ryan P commented on KAFKA-4364: --- https://github.com/apache/kafka/pull/2115 > Sink tasks exp

[jira] [Commented] (KAFKA-4364) Sink tasks expose secrets in DEBUG logging

[ https://issues.apache.org/jira/browse/KAFKA-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649203#comment-15649203 ] ASF GitHub Bot commented on KAFKA-4364: --- GitHub user rnpridgeon opened a pull reques

[GitHub] kafka pull request #2115: KAFKA-4364: Remove secrets from DEBUG logging

GitHub user rnpridgeon opened a pull request: https://github.com/apache/kafka/pull/2115 KAFKA-4364: Remove secrets from DEBUG logging leverage fix from KAFKA-2690 to remove secrets from task logging You can merge this pull request into a Git repository by running: $ git pull h

[jira] [Created] (KAFKA-4393) Improve invalid/negative TS handling

Matthias J. Sax created KAFKA-4393: -- Summary: Improve invalid/negative TS handling Key: KAFKA-4393 URL: https://issues.apache.org/jira/browse/KAFKA-4393 Project: Kafka Issue Type: Improvemen

Re: [DISCUSS] KIP-82 - Add Record Headers

Forgot to mention: Thank you for quantifying the trade-off - it is helpful and important regardless of what we end up deciding. On Tue, Nov 8, 2016 at 3:12 PM, Sean McCauliff wrote: > On Tue, Nov 8, 2016 at 2:15 PM, Gwen Shapira wrote: > >> Since Kafka specifically targets high-throughput, low-l

[jira] [Comment Edited] (KAFKA-4391) On Windows, Kafka server stops with uncaught exception after coming back from sleep

[ https://issues.apache.org/jira/browse/KAFKA-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649179#comment-15649179 ] huxi edited comment on KAFKA-4391 at 11/8/16 11:31 PM: --- It might be

[jira] [Commented] (KAFKA-4391) On Windows, Kafka server stops with uncaught exception after coming back from sleep

[ https://issues.apache.org/jira/browse/KAFKA-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649179#comment-15649179 ] huxi commented on KAFKA-4391: - It might because of the fact that network and disk will be deac

Re: [DISCUSS] KIP-82 - Add Record Headers

The benefit of having "high throughput, low latency" as a design goal is that when you are trying to choose between two designs (int key vs string key), we make sure we trade-off performance to get a benefit somewhere else. I know it is obvious but when you said: "The overall amount of time spent p

Re: [DISCUSS] KIP-82 - Add Record Headers

On Tue, Nov 8, 2016 at 2:15 PM, Gwen Shapira wrote: > Since Kafka specifically targets high-throughput, low-latency > use-cases, I don't think we should trade them off that easily. > I find these kind of design goals not to be really helpful unless it's quantified in someway. Because it's alway

[jira] [Updated] (KAFKA-4364) Sink tasks expose secrets in DEBUG logging

[ https://issues.apache.org/jira/browse/KAFKA-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan P updated KAFKA-4364: -- Assignee: Ryan P (was: Ewen Cheslack-Postava) > Sink tasks expose secrets in DEBUG logging > --

[GitHub] kafka pull request #1882: MINOR: some trace logging for streams debugging

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1882 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2114: MINOR: add upgrade guide for Kafka Streams API

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2114 MINOR: add upgrade guide for Kafka Streams API You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka updateDocUpgradeSection Alternatively

[jira] [Created] (KAFKA-4392) Failed to lock the state directory due to an unexpected exception

Ara Ebrahimi created KAFKA-4392: --- Summary: Failed to lock the state directory due to an unexpected exception Key: KAFKA-4392 URL: https://issues.apache.org/jira/browse/KAFKA-4392 Project: Kafka

[GitHub] kafka pull request #2113: KAFKA-4376: Cross compile to Scala 2.12.0

GitHub user leachbj opened a pull request: https://github.com/apache/kafka/pull/2113 KAFKA-4376: Cross compile to Scala 2.12.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/leachbj/kafka 2.12.0-trunk-build Alternatively you ca

[jira] [Commented] (KAFKA-4376) Add scala 2.12 support

[ https://issues.apache.org/jira/browse/KAFKA-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649005#comment-15649005 ] ASF GitHub Bot commented on KAFKA-4376: --- GitHub user leachbj opened a pull request:

Re: [DISCUSS] KIP-82 - Add Record Headers

Since Kafka specifically targets high-throughput, low-latency use-cases, I don't think we should trade them off that easily. I love strings as much as the next guy (we had them in Flume), but I was convinced by Magnus/Michael/Radai that strings don't actually have strong benefits as opposed to int

Build failed in Jenkins: kafka-trunk-jdk7 #1677

See Changes: [wangguoz] MINOR: remove commented out code and System.out.println -- [...truncated 3888 lines...] kafka.producer.ProducerTest > testAsyncSendCanCorrectlyFailWithTimeout STARTED k

Re: [DISCUSS] KAFKA-4345: Run decktape test for each pull request

On Mon, Nov 7, 2016 at 3:20 PM, Ewen Cheslack-Postava wrote: > On Mon, Nov 7, 2016 at 10:30 AM Raghav Kumar Gautam > wrote: > > > Hi Ewen, > > > > Thanks for the feedback. Answers are inlined. > > > > On Sun, Nov 6, 2016 at 8:46 PM, Ewen Cheslack-Postava > > > wrote: > > > > > Yeah, I'm all for

Re: [DISCUSS] KIP-82 - Add Record Headers

+1 for String keys. I've been doing some bechmarking and it seems like the speedup for using integer keys is about 2-5 depending on the length of the strings and what collections are being used. The overall amount of time spent parsing a set of header key, value pairs probably does not matter unl

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Jun, Have added a sub-section on delegation token support to the KIP. Thank you, Rajini On Tue, Nov 8, 2016 at 8:07 PM, Jun Rao wrote: > Hi, Rajini, > > That makes sense. Could you document this potential future extension in the > KIP? > > Jun > > On Tue, Nov 8, 2016 at 11:17 AM, Rajini Sivar

[jira] [Updated] (KAFKA-4391) On Windows, Kafka server stops with uncaught exception after coming back from sleep

[ https://issues.apache.org/jira/browse/KAFKA-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiquan Zhou updated KAFKA-4391: --- Description: Steps to reproduce: 1. start the zookeeper $ bin\windows\zookeeper-server-start.bat confi

[jira] [Created] (KAFKA-4391) On Windows, Kafka server stops with uncaught exception after coming back from sleep

Yiquan Zhou created KAFKA-4391: -- Summary: On Windows, Kafka server stops with uncaught exception after coming back from sleep Key: KAFKA-4391 URL: https://issues.apache.org/jira/browse/KAFKA-4391 Project

Build failed in Jenkins: kafka-trunk-jdk7 #1676

See Changes: [me] KAFKA-4284: Make Partitioner a Closeable and close it when closing the -- [...truncated 14180 lines...] org.apache.kafka.streams.kstream.KStreamBuilderTest > testMerge PASSED

Build failed in Jenkins: kafka-trunk-jdk8 #1027

See Changes: [wangguoz] MINOR: remove commented out code and System.out.println -- [...truncated 3875 lines...] kafka.utils.CommandLineUtilsTest > testParseSingleArg STARTED kafka.utils.Comman

[GitHub] kafka pull request #2092: MINOR: remove commented out code and System.out.pr...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2092 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hi, Rajini, That makes sense. Could you document this potential future extension in the KIP? Jun On Tue, Nov 8, 2016 at 11:17 AM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > Jun, > > 11. SCRAM messages have an optional extensions field which is a list of > key=value pairs. We can ad

Jenkins build is back to normal : kafka-trunk-jdk8 #1026

See

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Jun, 11. SCRAM messages have an optional extensions field which is a list of key=value pairs. We can add an extension key to the first client message to indicate delegation token. Broker can then obtain credentials and principal using a different code path for delegation tokens. On Tue, Nov 8, 20

[GitHub] kafka pull request #2112: MINOR: fix typos and incorrect docs

GitHub user xvrl opened a pull request: https://github.com/apache/kafka/pull/2112 MINOR: fix typos and incorrect docs You can merge this pull request into a Git repository by running: $ git pull https://github.com/xvrl/kafka minor-doc-fixes Alternatively you can review and ap

Re: [DISCUSS] KIP-82 - Add Record Headers

>From Roger's description: 5a- separate metadata field, built in serialization 5c- separate metadata field, custom serialization 5b- custom serialization (inside V) 5d- built in serialization (inside V) I added 5d for completeness. >From this perspective I would choose 5a > 5c > 5d > 5b In all

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Magnus, Thanks for the input. If you don't feel strongly the need to bump up the version of SaslHandshake, we can leave the version unchanged. Rajini, 11. Yes, we could send the HMAC as the SCRAM password for the delegation token. Do we need something to indicate that this SCRAM token is special

Re: [VOTE] KIP-72 - Allow putting a bound on memory consumed by Incoming requests

I've updated the KIP page to specify the new config would co-exist with "queued.max.request" to minimize the impact on compatibility. On Tue, Nov 8, 2016 at 7:02 AM, radai wrote: > My personal opinion on this is that control of memory was always the > intent behind queued.max.requests and so thi

[GitHub] kafka-site issue #29: Update the website repo link in code.html to point to ...

Github user becketqin commented on the issue: https://github.com/apache/kafka-site/pull/29 @ijuma Does the updated page look good to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[jira] [Commented] (KAFKA-4284) Partitioner never closed by producer

[ https://issues.apache.org/jira/browse/KAFKA-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648262#comment-15648262 ] ASF GitHub Bot commented on KAFKA-4284: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-4284) Partitioner never closed by producer

[ https://issues.apache.org/jira/browse/KAFKA-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4284: - Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was:

[GitHub] kafka pull request #2000: KAFKA-4284: Make Partitioner a Closeable and close...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2000 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-82 - Add Record Headers

both 5a and 5c would involve a wire format change, so any arguments about needing an upgrade path bumping protocol version etc apply equally to both. so the "cost" (in terms of impact of a wire format change) is the same. 5c, to me, means doing all the work (more exactly incurring all the cost) bu

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

I think the migration can be done in 2 stages : 1) In first stage the broker should understand the attribute flag as well as Null for the value for log compaction. 2) In second stage we move on to supporting only the attribute flag for log compaction. I agree with Becket that for older clients (c

[jira] [Created] (KAFKA-4390) Replace MessageSet usage with client-side equivalents

Jason Gustafson created KAFKA-4390: -- Summary: Replace MessageSet usage with client-side equivalents Key: KAFKA-4390 URL: https://issues.apache.org/jira/browse/KAFKA-4390 Project: Kafka Issue

[jira] [Commented] (KAFKA-4353) Add semantic types to Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647999#comment-15647999 ] Randall Hauch commented on KAFKA-4353: -- Logical types and semantic types are not the

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hey, I'm probably going to argue against bumping the SaslHandshake version, it is redundant to the existing SaslHandshakeResponse.enabled_mechanisms field. With SaslHandshake version bump: * Client performs ApiVersionRequest * If SaslHandshake >= v1: use SCRAM * If SaslHandshake < v1: use

[jira] [Commented] (KAFKA-4322) StateRestoreCallback begin and end indication

[ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647817#comment-15647817 ] ASF GitHub Bot commented on KAFKA-4322: --- GitHub user markcshelton reopened a pull re

[jira] [Commented] (KAFKA-4322) StateRestoreCallback begin and end indication

[ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647816#comment-15647816 ] ASF GitHub Bot commented on KAFKA-4322: --- Github user markcshelton closed the pull re

[GitHub] kafka pull request #2105: KAFKA-4322 StateRestoreCallback begin and end indi...

GitHub user markcshelton reopened a pull request: https://github.com/apache/kafka/pull/2105 KAFKA-4322 StateRestoreCallback begin and end indication This adds a begin and end callback to StateRestoreCallback. The contribution is my original work and I license the work to Apa

[GitHub] kafka pull request #2105: KAFKA-4322 StateRestoreCallback begin and end indi...

Github user markcshelton closed the pull request at: https://github.com/apache/kafka/pull/2105 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature i

Re: [VOTE] KIP-72 - Allow putting a bound on memory consumed by Incoming requests

My personal opinion on this is that control of memory was always the intent behind queued.max.requests and so this KIP could completely obsolete it. For now its probably safest to leave it as-is (making memory-bound "opt-in") and revisit this at a later date On Mon, Nov 7, 2016 at 2:32 PM, Gwen Sh

[jira] [Commented] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647603#comment-15647603 ] Ismael Juma commented on KAFKA-4362: This looks like a duplicate of KAFKA-4268. Since

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hi Jun, 10. *s=* and *i=* come from the SCRAM standard (they are transferred during SCRAM auth). Scram messages look like (for example) *r=,s=,i=*. StoredKey and ServerKey and not transferred in SCRAM messages, so I picked two keys that are unused in SCRAM. 11. SCRAM (like DIGEST-MD5 or PLAIN) us

RE: Kafka Connect key.converter and value.converter properties for Avro encoding

Hi Ewen, Thanks for the additional insight. Because I have no Connect schema (only an Avro schema) am I safe to just use the byte[] <-> (Avro Schema, SpecificRecord) conversion? This seems to work with the, admittedly limited, testing I've done so far. Without converting my Avro schema into

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

Also we can add further guidance: To avoid the below caveat to organisations by promoting of upgrading all consumers first before relying on producing tombstone messages with data Sent using OWA for iPhone From: Michael Pearce Sent: Tuesday, November 8,

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

Thanks Jun on the feedback, I think I understand the issue/point now. We def can add that on older client version if tombstone marker make the value null to preserve behaviour. There is one caveats to this: * we have to be clear that data is lost if reading via old client/message format - I do