Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer

2017-11-30 Thread Ted Yu
+1 On Thu, Nov 30, 2017 at 12:56 AM, Manikumar wrote: > +1 (non-binding) > > On Thu, Nov 30, 2017 at 8:35 AM, Hu Xi wrote: > > > Hi all, > > > > As I didn't see any further discussion around this KIP, I'd like to start > > voting. > > > > KIP documentation: > > > > https://cwiki.apache.org/conf

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

2017-12-01 Thread Ted Yu
at 12:52, Rajini Sivaram wrote: > >>> > > > Hi Tom, > >>> > > > > >>> > > > No, I am not proposing this as a way to configure replication > >>> quotas. > >>> > > > When > >>> > > > you describe broker

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

2017-12-01 Thread Ted Yu
d* contains the ephemeral > metadata registered by broker *.* For broker version, we don't want the > data to outlive the broker. Hence adding it to */brokers/ids/id.* > > On Fri, Dec 1, 2017 at 5:38 PM, Ted Yu wrote: > > > Thanks for the update. > > > > bq. To en

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

2017-12-02 Thread Ted Yu
bq. It the controller_epoch of the incoming MetadataResponse, or if the controller_epoch is the same but the controller_metadata_epoch Can you update the above sentence so that the intention is clearer ? Thanks On Fri, Dec 1, 2017 at 6:33 PM, Dong Lin wrote: > Hi all, > > I have created KIP-23

Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore

2017-12-04 Thread Ted Yu
Can you describe how sourceName is inferred based on the new API ? Please fill out JIRA number. BTW here is the URL for the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+StreamsBuilder%23addGlobalStore On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon < panuwat.

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-04 Thread Ted Yu
Can you add some code snippet showing how ConfigResource delivers topic defaults with the proposed change ? Cheers On Mon, Dec 4, 2017 at 9:25 AM, dan wrote: > I would like to start a discussion about KIP-234 > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 234%3A+add+support+for+get

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-04 Thread Ted Yu
In the example, resources is derived from ConfigResource. It would be clearer if you show the derivation. Cheers On Mon, Dec 4, 2017 at 9:51 AM, dan wrote: > I added sample usage as well as a WIP commit to the KIP. > > dan > > On Mon, Dec 4, 2017 at 9:36 AM, Ted Yu wrote: &g

Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore

2017-12-04 Thread Ted Yu
> On Mon, 4 Dec 2560 at 23:10 Ted Yu wrote: > > > Can you describe how sourceName is inferred based on the new API ? > > > > Please fill out JIRA number. > > > > BTW here is the URL for the KIP: > > > > https://cwiki.apache.org/confluence/display/KAFK

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

2017-12-04 Thread Ted Yu
; On Sat, Dec 2, 2017 at 3:05 PM, Ted Yu wrote: > > > bq. It the controller_epoch of the incoming MetadataResponse, or if the > > controller_epoch is the same but the controller_metadata_epoch > > > > Can you update the above sentence so that the intention is clearer ? &g

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

2017-12-04 Thread Ted Yu
I agree with Dong on maintaining the semantics. The user should know which group(s) he / she is allowed to describe. Cheers On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin wrote: > Hey Vahid, > > Thanks for the KIP. If I understand the you correctly, you want client to > be able to list all the group

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

2017-12-04 Thread Ted Yu
he user should know > which group(s) he / she is allowed to describe." in relationship to the > KIP. Perhaps you can clarify? > > Thanks. > --Vahid > > > > > From: Ted Yu > To: dev@kafka.apache.org > Date: 12/04/2017 02:01 PM > Subject:Re: [

Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore

2017-12-05 Thread Ted Yu
> > Just one nit: `does not follow provide a good` -> spelling: remove > > `follow` ? > > > > Otherwise, looks good to me. > > > > > > -Matthias > > > > > > > &

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-05 Thread Ted Yu
In KeepTimestampOnInvalidTimestamp, there should be check that timestamp is < 0. This would protect against future change to onInvalidTimestamp() callback. Wednesday, December 31, 1969 11:59:59 PM UTC was in the past. Can you enrich Motivation section on why the proposal is made (writing data gene

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-05 Thread Ted Yu
bq. We also have a tunable for total number of cache slots. We never cache more than this number of incremental fetch sessions. Is it possible to manage the cache based on heap consumption instead of number of slots ? It seems heap estimation can be done by counting PartitionData (along with overh

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-05 Thread Ted Yu
hanges to the KIP. > > > On Dec 5, 2017, at 11:59 AM, Ted Yu wrote: > > > > In KeepTimestampOnInvalidTimestamp, there should be check that > timestamp is > > < 0. > > This would protect against future change to onInvalidTimestamp() > callback. > > Not qui

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-05 Thread Ted Yu
approximate heap consumption, that should be good enough, IMHO. Evicting only one of the N small fetch sessions may not release enough memory since the total partition count would increase a lot. Cheers On Tue, Dec 5, 2017 at 1:44 PM, Colin McCabe wrote: > On Tue, Dec 5, 2017, at 11:24, Ted

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-05 Thread Ted Yu
What if the negative timestamp is stored this way ? Long.MIN_VALUE + delta (where delta is positvie) and calculated this way when used: 1/1/1970 - delta This approach avoids the ambiguity of -1 timestamp since -1 would be stored as Long.MIN_VALUE+1 Log retention can handle such format with min

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-05 Thread Ted Yu
bq. you can add X to the timestamp before you produce Kafka message This assumes the earliest timestamp (for user application) is known beforehand. However, what if this earliest timestamp shifts even earlier (e.g. due to some discovery) ? Cheers On Tue, Dec 5, 2017 at 6:36 PM, Dong Lin wrote:

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-06 Thread Ted Yu
bq. Using deltas will probably cause lots of headaches The downside of my initial proposal for using delta is that it didn't keep the natural ordering among negative timestamps. Here is variation of using delta : -1 is still kept as how it is treated today. The delta is calculated based on: 1/

Re: [DISCUSS] KIP-237: More Controller Health Metrics

2017-12-06 Thread Ted Yu
Thanks for the KIP. Can you add description for EventRateAndQueueTimeMs ? Cheers On Wed, Dec 6, 2017 at 6:21 PM, Dong Lin wrote: > Hi all, > > I have created KIP-237: More Controller Health Metrics > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 237%3A+More+Controller+Health+Metrics

Re: [VOTE] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-08 Thread Ted Yu
+1 On Fri, Dec 8, 2017 at 3:49 AM, Steven Aerts wrote: > Hello everybody, > > > I think KIP-218 is crystallized enough to start voting. > > KIP documentation: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 218%3A+Make+KafkaFuture.Function+java+8+lambda+compatible > > > Thanks, > >

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-09 Thread Ted Yu
For the znodes created under /admin/reassignments/, would it be better if the partition znodes for the same topic are put under subtree headed by topic name ? In Tom's example, instead of: /admin/reassignments/mytopic-42 the znode would be: /admin/reassignments/mytopic/42 Cheers On Fri, Dec 8,

Re: [VOTE] KIP-86: Configurable SASL callback handlers

2017-12-11 Thread Ted Yu
+1 Original message From: Tom Bentley Date: 12/11/17 6:06 AM (GMT-08:00) To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-86: Configurable SASL callback handlers +1 (non-binding) On 5 May 2017 at 11:57, Mickael Maison wrote: > Thanks for the KIP Rajini, this will significa

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ted Yu
Looks good overall. Currently lookupKafkaClusterId() is called synchronously. Have you considered making the call asynchronous (normally the GET / request comes sometime after worker start) ? Thanks On Mon, Dec 11, 2017 at 3:40 PM, Ewen Cheslack-Postava wrote: > I'd like to start discussion on

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ted Yu
nce you wouldn't be able to load the configs or > status topics, etc). > > -Ewen > > On Mon, Dec 11, 2017 at 4:35 PM, Ted Yu wrote: > > > Looks good overall. > > > > Currently lookupKafkaClusterId() is called synchronously. Have you > > considered making t

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2017-12-12 Thread Ted Yu
+1 On Tue, Dec 12, 2017 at 5:44 AM, Rajini Sivaram wrote: > Since there are no more outstanding comments, I would like to start vote > for KIP-226: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 226+-+Dynamic+Broker+Configuration > > > The KIP enables dynamic update of commonly upd

[DISCUSS] KIP-239 Add queryableStoreName() to GlobalKTable

2017-12-12 Thread Ted Yu
Hi, Please kindly provide feedback on the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-239+Add+queryableStoreName%28%29+to+GlobalKTable Thanks

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-13 Thread Ted Yu
Tom: bq. create a znode /admin/reassignments/$topic-$partition Looks like the tree structure above should be: /admin/reassignments/$topic/$partition bq. The controller removes /admin/reassignment/$topic/$partition Note the lack of 's' for reassignment. It would be good to make zookeeper paths c

Re: Reg. beginner issues for a new contributor

2017-12-14 Thread Ted Yu
Have you seen this ? http://search-hadoop.com/m/Kafka/uyzND14PPsT5THhN1?subj=Re+Contributing+to+Kafka On Thu, Dec 14, 2017 at 8:55 AM, Abhinav Koppula wrote: > Hi Team, > I wanted to get started with contributing to Kafka and learn more about its > internals. Can anyone please suggest me some "

Re: Queries on Kafka Capacity

2017-12-14 Thread Ted Yu
It seems this should help (coming in 1.1 release): https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories On Wed, Dec 13, 2017 at 11:37 PM, ajay chaudhary < juniora...@yahoo.com.invalid> wrote: > Hi Team, > This is Ajay working with Yodlee

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-15 Thread Ted Yu
Please create corresponding JIRA. For class Reassignment, it seems you forgot to include set of brokers. For class DescribeReassignmentsResult: public KafkaFuture reassignments(); the return value should be a Collection. On Fri, Dec 15, 2017 at 10:16 AM, Tom Bentley wrote: > Hi, > > KIP-2

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-15 Thread Ted Yu
+1 On Fri, Dec 15, 2017 at 10:49 AM, Ewen Cheslack-Postava wrote: > Discussion seems to have tapered off, so I'd like to start the vote on > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API > > Obviously +1 (binding) from me :) > > -Ewen

[VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

2017-12-15 Thread Ted Yu
Hi, Here is the discussion thread: http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj=Re+DISCUSS+KIP+239+Add+queryableStoreName+to+GlobalKTable Please vote on this KIP. Thanks

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

2017-12-18 Thread Ted Yu
+1 nit: via "copy and past" an 'e' is missing at the end. On Mon, Dec 18, 2017 at 2:38 PM, Matthias J. Sax wrote: > Hi, > > I want to propose the following KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 243%3A+Make+ProducerConfig+and+ConsumerConfig+constructors+public > > > Thi

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

2017-12-20 Thread Ted Yu
Ping for more (binding) votes. The pull request is ready. On Fri, Dec 15, 2017 at 12:57 PM, Guozhang Wang wrote: > +1 (binding), thanks! > > On Fri, Dec 15, 2017 at 11:56 AM, Ted Yu wrote: > > > Hi, > > Here is the discussion thread: > > > > http://search-

Re: Vote for KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor

2017-12-26 Thread Ted Yu
Normally a DISCUSS thread precedes VOTE thread so that people have ample time examining the proposal. Original message From: Boyang Chen Date: 12/26/17 1:22 AM (GMT-07:00) To: dev@kafka.apache.org Subject: Vote for KIP-245: Use Properties instead of StreamsConfig in KafkaStre

Re: Vote for KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor

2017-12-26 Thread Ted Yu
; > > -Matthias > > > @Boyang: it's recommended to use this format for the subject > > "[VOTE] KIP-245: ..." > > Same for DISCUSS threads. People are used to those headlines and they > pay more attention than. For this KIP, just leave it as it though. For >

Re: [VOTE] KIP-233: Simplify StreamsBuilder#addGlobalStore

2017-12-27 Thread Ted Yu
+1 On Wed, Dec 27, 2017 at 12:15 PM, Bill Bejeck wrote: > +1 > > On Wed, Dec 27, 2017 at 3:07 PM, Matthias J. Sax > wrote: > > > +1 > > > > On 12/26/17 9:00 PM, Panuwat Anawatmongkhon wrote: > > > Hi all, > > > I would like to start the vote thread. > > > This is link for the kip. > > > > > > h

Re: Unable to start Sink connectors[Confluent Kafka]

2017-12-28 Thread Ted Yu
Looking at https://docs.confluent.io/3.0.0/control-center/docs/clients.html#adding-the-interceptor-to-your-kafka-consumer , it seems that the class name in interceptor.classes had a typo (extra 'e'). Can you double check ? Cheers On Tue, Dec 19, 2017 at 11:00 PM, Somasundaram Sekar < somasundar.

Re: [DISCUSS] KIP-242: Mask password fields in Kafka Connect REST response

2017-12-28 Thread Ted Yu
For the last point you raised, can you come up with a unit test that shows what you observed ? Cheers On Mon, Dec 18, 2017 at 11:14 AM, Vincent Meng wrote: > Hi all, > > I've created KIP-242, a proposal to secure credentials in kafka connect > rest endpoint. > > https://cwiki.apache.org/conflue

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

2018-01-01 Thread Ted Yu
Gentle reminder: one more binding vote is needed for the KIP to pass. Cheers On Thu, Dec 21, 2017 at 4:13 AM, Damian Guy wrote: > +1 > > On Wed, 20 Dec 2017 at 21:09 Ted Yu wrote: > > > Ping for more (binding) votes. > > > > The pull request is ready. > >

Re: Pull Request emails are not being sent to the mailing list

2018-01-02 Thread Ted Yu
pull request emails, at least the initial one, are recorded on the underlying JIRA. For the time being, you can get to pull request(s) of interest thru JIRA. FYI On Tue, Jan 2, 2018 at 1:30 PM, James Cheng wrote: > Since Dec 21st, it seems that email notifications of Github pull requests > are

Re: [DISCUSS] KIP-189: Improve principal builder interface and add support for SASL

2017-08-28 Thread Ted Yu
bq. change the check in equals() to be this.getClass().equals(other. getClass()) I happened to have Effective Java on hand. Please take a look at related discussion on page 39. Josh later on mentioned Liskov substitution principle and a workaround (favoring composition). FYI On Mon, Aug 28, 2

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

2017-08-30 Thread Ted Yu
For ProduceRequest v4, would int32 or int16 be enough for idempotenceLevel ? Cheers On Wed, Aug 30, 2017 at 3:47 PM, Apurva Mehta wrote: > Thanks Ismael and Jason, I filed a separate KIP to solve the problems > identified through this discussion. I also incorporated Jason's comments in > that d

Re: Dry-run option for new admin commands

2017-09-01 Thread Ted Yu
State change on broker side may be subject to various factors (network, disk, etc). How would the dry-run report state change which is close to what actually happens in non-dry-run mode ? Thanks On Fri, Sep 1, 2017 at 7:48 AM, Viktor Somogyi wrote: > Hi all, > > I've been working on creating a

Turing on idempotence by default

2017-09-02 Thread Ted Yu
Hi, I was reading https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/ bq. this feature add negligible performance overhead over the non-idempotent producer. Should idempotence be turned on by default ? Cheers

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-09-05 Thread Ted Yu
bq. What about startPartitionAssignment() ? Make sense. startPartitionReassignment() seems to be better since the API deals with reassignment. Cheers On Tue, Sep 5, 2017 at 9:39 AM, Tom Bentley wrote: > I've revised this KIP again: > > * Change the alterPartitionCounts() API to support passing

Re: [DISCUSS] KIP-192 - Provide cleaner semantics when idempotence is enabled

2017-09-05 Thread Ted Yu
For enable.idempotence=safe, it seems giving user impression that idempotence would be safe. However, since it really means best effort, the 'safety' is debatable. Why not just call the new mode besteffort ? Cheers On Tue, Sep 5, 2017 at 11:24 AM, Ismael Juma wrote: > If we add the message fo

Re: [DISCUSS] KIP-192 - Provide cleaner semantics when idempotence is enabled

2017-09-05 Thread Ted Yu
requested, required, off ("safe" was in > an earlier proposal). I think these convey the meaning more clearly IMO. > > Ismael > > On 5 Sep 2017 9:22 pm, "Ted Yu" wrote: > > > For enable.idempotence=safe, it seems giving user impression that > > idempotenc

integration between pull request and JIRA

2017-09-05 Thread Ted Yu
Hi, Currently the conversations on pull request are not posted back to JIRA. Is there technical hurdle preventing this from being done ? Other Apache projects, such as Flink, establish automatic post from pull request to JIRA. Cheers

Re: integration between pull request and JIRA

2017-09-05 Thread Ted Yu
o the PR to read the discussion if interested. > > Or what do you think the value would be? > > > -Matthias > > > On 9/5/17 6:16 PM, Ted Yu wrote: > > Hi, > > Currently the conversations on pull request are not posted back to JIRA. > > > > Is there tec

Re: integration between pull request and JIRA

2017-09-05 Thread Ted Yu
can just "not watch" Kafka's Github repo. I don't watch it either > and thus I get emails for only those PRs I did get tagged or I did > comment on etc. > > Would this not work for you? > > > -Matthias > > On 9/5/17 7:31 PM, Ted Yu wrote: > > If

Re: Fw: [VOTE] KIP-176 : Remove deprecated new-consumer option for tools

2017-09-06 Thread Ted Yu
Looks good to me. bq. *specifying the --zookeeper for the farmer * *farmer -> former* On Wed, Sep 6, 2017 at 1:42 AM, Paolo Patierno wrote: > Hi devs, > > > I haven't seen any votes for this since last month. > > Is there something that should be addressed in the KIP (it didn't have any > comm

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-07 Thread Ted Yu
Tom: Looks good overall. bq. for the topic from the AlterPartitionCountsResult Please align the name of Result with current proposal. Please also fill in JIRA number when you have it. On Thu, Sep 7, 2017 at 9:38 AM, Tom Bentley wrote: > As suggested by Ismael, I've factored the increasePartit

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Ted Yu
I think increasePartitions is better name, implying there are already partitions. bq. use cases for non-consecutive partition ids. +1 on not supporting non-consecutive partition ids Cheers On Fri, Sep 8, 2017 at 5:34 AM, Ismael Juma wrote: > Hmm, maybe it should be createPartitions for symmet

request for permission to create KIP

2017-09-08 Thread Ted Yu
Hi, My wiki username is tedyu Can someone grant me permission to create KIP ? Thanks

[DISCUSS] KIP-197: Include Connector type in Connector REST API

2017-09-08 Thread Ted Yu
Hi, Please take a look at: https://cwiki.apache.org/confluence/display/KAFKA/KIP-197+Connect+REST+API+should+include+the+connector+type+when+describing+a+connector Thanks

Re: [DISCUSS] KIP-197: Include Connector type in Connector REST API

2017-09-08 Thread Ted Yu
> > Best regards, > > Randall > > On Fri, Sep 8, 2017 at 12:00 PM, Ted Yu wrote: > > > Hi, > > Please take a look at: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 197+Connect+REST+API+should+include+the+connector+type+ > > when+describing+a+connector > > > > Thanks > > >

Re: [VOTE] KIP-198: Remove ZK dependency from Streams Reset Tool

2017-09-08 Thread Ted Yu
bq. parameter `--zookeper` that will be deprecated. Can you clarify in which release the parameter will be deprecated and in which release it will be removed ? On Fri, Sep 8, 2017 at 1:15 PM, Matthias J. Sax wrote: > Hi, > > I want to propose KIP-198 for 1.0.0 release. It's about removing ZK >

Re: [DISCUSS] KIP-197: Include Connector type in Connector REST API

2017-09-08 Thread Ted Yu
(just like you did to the > `ConnectorInfo` class). WDYT? > > Best regards, > > Randall > > On Fri, Sep 8, 2017 at 1:12 PM, Ted Yu wrote: > > > Thanks for the reminder, Randall. > > > > I have modified the KIP to include these two endpoints. > > > &

Re: [DISCUSS] KIP-197: Include Connector type in Connector REST API

2017-09-08 Thread Ted Yu
Sink" and > "Source"). Thoughts? > > On Fri, Sep 8, 2017 at 6:02 PM, Ted Yu wrote: > > > Updated the KIP accordingly. > > > > Cheers > > > > On Fri, Sep 8, 2017 at 3:57 PM, Randall Hauch wrote: > > > > > Hi, Ted. Thanks for the qui

[VOTE] KIP-197: Include Connector type in Connector REST API

2017-09-08 Thread Ted Yu
Hi, Please take a look at the following and cast your vote: https://cwiki.apache.org/confluence/display/KAFKA/KIP- 197+Connect+REST+API+should+include+the+connector+type+ when+describing+a+connector Thanks

Re: [VOTE] KIP-91 Provide Intuitive User Timeouts in The Producer

2017-09-09 Thread Ted Yu
+1 for the KIP. For delivery.timeout.ms , since it should be >= linger.ms + request.timeout.ms + retry.backoff.ms , it seems the default value should be max(120 seconds, linger.ms + request.timeout.ms + retry.backoff.ms). Cheers On Fri, Sep 8, 2017 at 2:04 AM, Ismael Juma wrote: > Thanks for t

Re: [DISCUSS] KIP-199: Add Kafka Connect offset reset tool

2017-09-10 Thread Ted Yu
bq. connector restart and the next message The last part of the sentence seems to be incomplete. bq. command line tool called kafka-connect-source-offset-reset.sh >From the description, the tool does more than resetting (e.g. deleting). How about calling it kafka-connect-source-offset-tool.sh b

Re: [DISCUSS] KIP-199: Add Kafka Connect offset reset tool

2017-09-10 Thread Ted Yu
; > Regards, > > Randall > > On Sun, Sep 10, 2017 at 2:42 PM, Ted Yu wrote: > > > bq. connector restart and the next message > > > > The last part of the sentence seems to be incomplete. > > > > Fixed. > > > > bq. command line tool called

Re: [VOTE] KIP-199: Add Kafka Connect Offset Tool

2017-09-11 Thread Ted Yu
+1 On Mon, Sep 11, 2017 at 7:43 AM, Randall Hauch wrote: > I'd like to start the vote on KIP-199 to add a command line tool that will > allow Connect operators to read, modify, and update source connector > offsets. Details are here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 1

Re: [VOTE] KIP-91 Provide Intuitive User Timeouts in The Producer

2017-09-11 Thread Ted Yu
, Sumant, > > > > Thanks for the KIP. +1. > > > > Just a minor clarification. The KIP says "Batches expire in order > > when max.in.flight.request.per.connection==1". Is that true? It seems > that > > even with max.in.flight.request.per.connection &

Re: [VOTE] KIP-91 Provide Intuitive User Timeouts in The Producer

2017-09-11 Thread Ted Yu
We throw a ConfigException when user-configured values of linger.ms, > request.timeout.ms, retry.backoff.ms add up to more than > delivery.timeout.ms > . The kip mentions this in the Validation section. > > On 11 September 2017 at 14:31, Ted Yu wrote: > > > bq. lar

Re: [DISCUSS] KIP-197: Include Connector type in Connector REST API

2017-09-11 Thread Ted Yu
to me! Thanks again! > > > > I say go ahead and ask for a vote in a new thread. > > > > Randall > > > > On Fri, Sep 8, 2017 at 6:22 PM, Ted Yu wrote: > > > > > Lowercase constants are generated by the enum. > > > > > > Updated KIP

Re: [VOTE] KIP-197: Include Connector type in Connector REST API

2017-09-12 Thread Ted Yu
> > > +1 > > > > > > On Fri, Sep 8, 2017 at 4:33 PM, Randall Hauch > wrote: > > > > > > > +1 (non-binding) > > > > > > > > Randall > > > > > > > > On Fri, Sep 8, 2017 at 6:32 PM, Ted Yu wrote: > >

Re: [VOTE] KIP-197: Include Connector type in Connector REST API

2017-09-12 Thread Ted Yu
nks for the contribution Ted! Simple addition, but makes the API > > > > significantly more usable. > > > > > > > > -Ewen > > > > > > > > On Fri, Sep 8, 2017 at 7:46 PM, Sriram Subramanian > > > > > wrote: >

Re: Build failed in Jenkins: kafka-trunk-jdk8 #2016

2017-09-13 Thread Ted Yu
I have filed INFRA-15080 for the 'No space left on device' error. On Wed, Sep 13, 2017 at 12:07 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See display/redirect?page=changes> > > Changes: > > [ismael] KAFKA-4501; Fix Easy

Re: [ANNOUCE] Apache Kafka 0.11.0.1 Released

2017-09-13 Thread Ted Yu
Damian: Looks like maven artifacts are not populated yet: https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients Do you know when artifacts would be visible for downstream projects ? Thanks On Wed, Sep 13, 2017 at 4:36 AM, Damian Guy wrote: > The Apache Kafka community is pleased t

Re: [ANNOUCE] Apache Kafka 0.11.0.1 Released

2017-09-13 Thread Ted Yu
Using a downstream project, I verified that artifacts for 0.11.0.1 can be pulled down. Thanks On Wed, Sep 13, 2017 at 2:00 PM, Damian Guy wrote: > AFAICT they are there > http://repo.maven.apache.org/maven2/org/apache/kafka/kafka- > clients/0.11.0.1/ > On Wed, 13 Sep 2017 at 2

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-09-14 Thread Ted Yu
+1 One interface is cleaner. On Thu, Sep 14, 2017 at 7:26 AM, Bill Bejeck wrote: > +1 for me on collapsing the Rich and ValueWithKey interfaces into 1 > interface. > > Thanks, > Bill > > On Wed, Sep 13, 2017 at 11:31 AM, Jeyhun Karimov > wrote: > > > Hi Damian, > > > > Thanks for your

QA bot has stalled

2017-09-14 Thread Ted Yu
Hi, I logged INFRA-15084 . FYI

system test builder

2017-09-14 Thread Ted Yu
Hi, When I put the following in the address bar of Chrome: https://jenkins.confluent.io/job/system-test-kafka-branch-builder I was told: This site can’t be reached Are the tests accessible by the public ? Thanks

Re: KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule

2017-09-18 Thread Ted Yu
Makes sense. Please fill out JIRA name, etc. On Mon, Sep 18, 2017 at 9:30 AM, Manikumar wrote: > Hi all, > > I've created a small KIP to extend the sasl.kerberos.principal.to.local > rule syntax to convert short names to lower case. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 2

Re: unsubscribe

2017-09-18 Thread Ted Yu
Please look at instruction on https://kafka.apache.org/contact for unsubscribing On Mon, Sep 18, 2017 at 10:24 AM, Madhav Ancha (BLOOMBERG/ 919 3RD A) < manc...@bloomberg.net> wrote: > unsubscribe > >

Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API

2017-09-18 Thread Ted Yu
+1 On Mon, Sep 18, 2017 at 9:19 AM, Paolo Patierno wrote: > Hi devs, > > > I'd like to start a discussion around adding the delete records operation, > already available at protocol level and in the "legacy" Admin Client in > Scala, to the "new" Admin Client API in Java. > > > https://cwiki.apac

Limit Jenkins jobs on H0 to H13

2017-09-19 Thread Ted Yu
Hi, See Gavin's comment: https://issues.apache.org/jira/browse/INFRA-15084?page= com.atlassian.jira.plugin.system.issuetabpanels:comment- tabpanel&focusedCommentId=16172575#comment-16172575 Can someone with admin permission modify the Jenkins job(s) ? Thanks

Re: [DISCUSS] : KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance

2020-01-18 Thread Ted Yu
I wonder if the following two methods can be combined: Integer getPartition() // would be null if unset or if "all partitions" boolean getAllLocalPartitions() // true/false if "all partitions" requested into: Integer getPartition() // would be null if unset or -1 if "all partitions" Cheers On

Re: [DISCUSS] : KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance

2020-01-18 Thread Ted Yu
I guess this also means getPartition() should either throw an exception or > return null if the partition is unspecified. > > Thanks, > John > > On Sat, Jan 18, 2020, at 08:43, Ted Yu wrote: > > I wonder if the following two methods can be combined: > > > > Integ

Re: [VOTE] KIP-553: Disable all SSL protocols except TLSV1.2 by default.

2020-01-21 Thread Ted Yu
+1 On Tue, Jan 21, 2020 at 8:24 AM Rajini Sivaram wrote: > +1 (binding) > > Thanks for the KIP! > > Regards, > > Rajini > > > On Tue, Jan 21, 2020 at 3:43 PM Николай Ижиков > wrote: > > > Hello. > > > > I would like to start vote for KIP-553: Disable all SSL protocols except > > TLSV1.2 by defa

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-28 Thread Ted Yu
+1 On Tue, Jan 28, 2020 at 10:52 AM Rajini Sivaram wrote: > +1 (binding) > > Thanks for the KIP, Brian! > > Regards, > > Rajini > > On Thu, Jan 23, 2020 at 7:34 PM Jason Gustafson > wrote: > > > Sounds good. +1 from me. > > > > On Thu, Jan 23, 2020 at 9:00 AM Brian Byrne wrote: > > > > > Thank

Re: [KAFKA-557] Add emit on change support for Kafka Streams

2020-02-02 Thread Ted Yu
w.r.t. new metric, there is already droppedRecordsSensor which logs: "Skipping record due to null key. topic=[{}] partition=[{}] offset=[{}]", It seems we should introduce another metric which records the skipped (duplicate) values. This way, it is easier to observe the effect when this feature

Re: [KAFKA-557] Add emit on change support for Kafka Streams

2020-02-05 Thread Ted Yu
Thanks for the comments, Matthias. w.r.t. requirement of an `equals()` implementation, each template type would have an equals() method. We can use the following code to know whether it is provided by JVM or provided by user. boolean customEquals = false; try { Class cls = value.getClass().ge

Re: [DISCUSS] Compatability and KIPs

2015-01-09 Thread Ted Yu
Jay: https://www.python.org/dev/peps/pep-0257 gives me 404. Can you double check ? Cheers

Re: [DISCUSS] Compatability and KIPs

2015-01-09 Thread Ted Yu
Jaikiran > On Saturday 10 January 2015 07:16 AM, Ted Yu wrote: > >> Jay: >> https://www.python.org/dev/peps/pep-0257 >> <https://www.python.org/dev/peps/pep-0257/)> gives me 404. >> >> Can you double check ? >> >> Cheers >> >> >

Re: [DISCUSS] Compatability and KIPs

2015-01-11 Thread Ted Yu
For projects written in Java, there is http://ispras.linuxbase.org/index.php/Java_API_Compliance_Checker I searched for similar tool for Scala but haven't found one yet. Cheers On Sat, Jan 10, 2015 at 10:40 AM, Ashish Singh wrote: > Jay, > > I totally agree with paying more attention to compat

[jira] [Resolved] (KAFKA-6734) TopicMetadataTest is flaky

2018-05-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6734. --- Resolution: Cannot Reproduce > TopicMetadataTest is fl

[jira] [Resolved] (KAFKA-6736) ReassignPartitionsClusterTest#shouldMoveSubsetOfPartitions is flaky

2018-05-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6736. --- Resolution: Cannot Reproduce > ReassignPartitionsClusterTest#shouldMoveSubsetOfPartitions is fl

[jira] [Created] (KAFKA-6875) EosIntegrationTest#shouldNotViolateEosIfOneTaskFails is flaky

2018-05-07 Thread Ted Yu (JIRA)
Ted Yu created KAFKA-6875: - Summary: EosIntegrationTest#shouldNotViolateEosIfOneTaskFails is flaky Key: KAFKA-6875 URL: https://issues.apache.org/jira/browse/KAFKA-6875 Project: Kafka Issue Type

[jira] [Created] (KAFKA-6904) DynamicBrokerReconfigurationTest#testAdvertisedListenerUpdate is flaky

2018-05-14 Thread Ted Yu (JIRA)
Ted Yu created KAFKA-6904: - Summary: DynamicBrokerReconfigurationTest#testAdvertisedListenerUpdate is flaky Key: KAFKA-6904 URL: https://issues.apache.org/jira/browse/KAFKA-6904 Project: Kafka

[jira] [Resolved] (KAFKA-6875) EosIntegrationTest#shouldNotViolateEosIfOneTaskFails is flaky

2018-05-22 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6875. --- Resolution: Cannot Reproduce > EosIntegrationTest#shouldNotViolateEosIfOneTaskFails is fl

[jira] [Resolved] (KAFKA-6698) ConsumerBounceTest#testClose sometimes fails

2018-05-28 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6698. --- Resolution: Cannot Reproduce > ConsumerBounceTest#testClose sometimes fa

[jira] [Created] (KAFKA-7049) InternalTopicIntegrationTest sometimes fails

2018-06-12 Thread Ted Yu (JIRA)
Ted Yu created KAFKA-7049: - Summary: InternalTopicIntegrationTest sometimes fails Key: KAFKA-7049 URL: https://issues.apache.org/jira/browse/KAFKA-7049 Project: Kafka Issue Type: Test

[jira] [Created] (KAFKA-7124) Number of AnyLogDir should match the length of the replicas list

2018-07-01 Thread Ted Yu (JIRA)
Ted Yu created KAFKA-7124: - Summary: Number of AnyLogDir should match the length of the replicas list Key: KAFKA-7124 URL: https://issues.apache.org/jira/browse/KAFKA-7124 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-7124) Number of AnyLogDir should match the length of the replicas list

2018-07-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-7124. --- Resolution: Not A Problem > Number of AnyLogDir should match the length of the replicas l

[jira] [Resolved] (KAFKA-6335) SimpleAclAuthorizerTest#testHighConcurrencyModificationOfResourceAcls fails intermittently

2018-07-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6335. --- Resolution: Cannot Reproduce > SimpleAclAuthorizerTest#testHighConcurrencyModificationOfResourceAcls fa

<    1   2   3   4   5   >