Re: [VOTE] 2.1.1 RC2

2019-02-11 Thread Colin McCabe
gt; > > > > +1 (non-binding). I built it from source and run my tests. Everything > > > > seems > > > > to be fine. > > > > > > > > On Sat, Feb 9, 2019 at 12:10 AM Magnus Edenhill > > > > wrote: > > > > > > > >

Re: [ANNOUNCE] New Committer: Bill Bejeck

2019-02-14 Thread Colin McCabe
Congratulations, Bill! best, Colin On Thu, Feb 14, 2019, at 09:17, Viktor Somogyi-Vass wrote: > Congrats Bill! :) > > On Thu, Feb 14, 2019 at 4:50 PM Dongjin Lee wrote: > > > Congrats Bill, and thank you again for your great book and contributions on > > Kafka Streams! > > > > Best, > > Dongji

Re: [kafka-clients] [VOTE] 2.1.1 RC2

2019-02-15 Thread Colin McCabe
me. >> >> Thanks Colin! >> >> On Thu, Feb 14, 2019 at 3:32 PM Jun Rao wrote: >> >> > Hi, Colin, >> > >> > Thanks for running the release. Verified the quickstart for 2.12 binary. +1 >> > from me. >> > >> > Jun

Re: [kafka-clients] [VOTE] 2.1.1 RC2

2019-02-15 Thread Colin McCabe
P.S. I have added KAFKA-7897 to the release notes. Good catch, Jason. best, Colin On Fri, Feb 15, 2019, at 00:49, Colin McCabe wrote: > Hi all, > > With 7 non-binding +1 votes, 3 binding +1 votes, no +0 votes, and no -1 > votes, the vote passes. > > Thanks, all! &g

[ANNOUNCE] Apache Kafka 2.1.1

2019-02-19 Thread Colin McCabe
Dagostino, slim, Stig Rohde Døssing, Vahid Hashemian, and Xi Yang. We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/ Thank you! Regards, Colin McCabe

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

2019-02-22 Thread Colin McCabe
Hi Rajini, Thanks for the KIP! The KIP specifies that "Authorized operations will be returned as [an] INT8 consistent with [the] AclOperation used in ACL requests and responses." But there may be more than one AclOperation that is applied to a given resource. For example, a principal may hav

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

2019-02-22 Thread Colin McCabe
Hi Yaodong, KIP-422 says that it would be good if "applications [could] leverage the unified KafkaAdminClient to manage their user/client configurations, instead of the direct dependency on Zookeeper." But the KIP doesn't talk about any changes to KafkaAdminClient. Instead, the only changes p

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

2019-02-22 Thread Colin McCabe
software is too old to understand is useful (you may choose not to modify that ACL, since you don't know what it does, for example.) What do you think? cheers, Colin > > Thank you, > > Rajini > > > > On Fri, Feb 22, 2019 at 6:46 PM Colin McCabe wrote: > >

Re: [DISCUSS] KIP-409: Allow creating under-replicated topics and partitions

2019-02-26 Thread Colin McCabe
Hi Mickael, I don't think adding CREATED_UNDER_REPLICATED as an error code makes sense. It is not an error condition, as described here. > Updates to the Decommissioning brokers section in the documentation > will mention that if a broker id is never to be reused then its corresponding > node

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

2019-02-26 Thread Colin McCabe
which suggests that quotas are involved somehow What relationship does this have with KIP-257? best, Colin On Fri, Feb 22, 2019, at 15:11, Yaodong Yang wrote: > Hi Colin, > > CIL, > > Thanks! > Yaodong > > > On Fri, Feb 22, 2019 at 10:56 AM Colin McCabe wrote:

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-02-26 Thread Colin McCabe
Hi Harsha, I think Ismael and Gwen here bring up a good point. The version of the request is a technical detail that isn't really related to authorization. There are a lot of other technical details like this like the size of the request, the protocol it came in on, etc. None of them are pas

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

2019-02-26 Thread Colin McCabe
client version. Does that match your expectation? > > Thank you, > > Rajini > > > On Sat, Feb 23, 2019 at 1:03 AM Colin McCabe wrote: > > > Hi Rajini, > > > > Thanks for the explanations. > > > > On Fri, Feb 22, 2019, at 11:59, Rajini Sivar

Re: Speeding up integration tests

2019-02-27 Thread Colin McCabe
On Wed, Feb 27, 2019, at 10:02, Ron Dagostino wrote: > Hi everyone. Maybe providing the option to run it both ways -- start your > own cluster vs. using one that is pre-started -- might be useful? Don't > know how it would work or if it would be useful, but it is something to > think about. > >

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

2019-02-27 Thread Colin McCabe
Hi Li, > The mechanism simplifies deployment because the same configuration can be > used across all brokers, however, in a large system where disk failure is > a norm, the meta file could often get lost, causing a new broker id being > allocated. This is problematic because new broker id h

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-02-27 Thread Colin McCabe
on on a topic shouldn't have to change anything. So I think the Authorizer interface should not be aware of individual RPC types or message versions. best, Colin > > > Thanks, > Harsha > > > n Tue, Feb 26, 2019, at 10:04 AM, Colin McCabe wrote: > > Hi Harsha

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-03-01 Thread Colin McCabe
be part of the dynamic config . +1, sounds like a good idea. Colin > > Thanks, > Harsha > > On Wed, Feb 27, 2019, at 12:17 PM, Colin McCabe wrote: > > On Tue, Feb 26, 2019, at 16:33, Harsha wrote: > > > Hi Colin, > > > > > > "> I think

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

2019-03-01 Thread Colin McCabe
-creating nodes that failed. That's up to kubernetes, or ansible, or whatever cluster provisioning framework you have in place. This feels like the same kind of thing: managing how the cluster is provisioned. best, Colin > > Thanks, > Harsha > On Wed, Feb 27, 2019, at 11:53

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

2019-03-01 Thread Colin McCabe
+1, looks like a good improvement! Best, Colin On Mon, Feb 18, 2019, at 14:00, Mateusz Zakarczemny wrote: > Hi all, > > I have created a KIP to support additional message fields in console > consumer: > KIP-431 - Support of printing additional ConsumerRecord fields in > DefaultMessageFormatter >

Re: [VOTE] KIP-436 Add a metric indicating start time

2019-03-07 Thread Colin McCabe
+1 (binding). Thanks, Stanislav. best, Colin On Tue, Mar 5, 2019, at 05:23, Stanislav Kozlovski wrote: > Hey everybody, > > I'd like to start a vote thread about the lightweight KIP-436 > KIP: KIP-436 >

Re: [DISCUSSION] KIP-421: Support resolving externalized secrets in AbstractConfig

2019-03-14 Thread Colin McCabe
ng the thread for further discussion. Please let me know your > thoughts on it. > > Thanks, > Tejal > > On 2019/01/25 19:11:07, "Colin McCabe" wrote: > > On Fri, Jan 25, 2019, at 09:12, Andy Coates wrote:> > > > > Further, if we're worried about

Re: Guava version upgrade

2019-03-15 Thread Colin McCabe
Hi JIAHAO, Kafka does not use Guava. Some of the packages Kafka Connect depend on use Guava. Perhaps the right thing to do is track down those projects and see how they are using Guava (if they are vulnerable to the CVE). best, Colin On Mon, Mar 4, 2019, at 15:52, JIAHAO ZHOU wrote: > Hello

Re: Inconsistencies in the Kafka protocol guide

2019-03-18 Thread Colin McCabe
Hi Fredrik, There is no need to parse the documentation-- the Kafka protocol now has a protocol specification here: https://github.com/apache/kafka/tree/trunk/clients/src/main/resources/common/message One caveat is that there are no protocol specifications for the message format yet. best, Co

Re: [DISCUSS] KIP-398: Support reading trust store from classpath

2019-03-20 Thread Colin McCabe
Hi Noa, I think Jun makes a good point that this would be better as a generic mechanism, not specific to SSL. With KIP-421: Support resolving externalized secrets in AbstractConfig, we should be able to do something like that. We could have a ClasspathConfigProvider. Then this could be used

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2019-03-20 Thread Colin McCabe
Hi George, One big problem here is that administrative APIs should be done through RPCs, not through ZooKeeper. KIP-4 (Command line and centralized administrative operations) describes the rationale for this. We want public and stable APIs that don't depend on the internal representation of s

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2019-03-23 Thread Colin McCabe
elieve there is a KIP for that conversion. Reassigning partitions is probably the biggest KIP-4 gap we have right now. best, Colin > > Thanks, > George > > > On Wednesday, March 20, 2019, 5:28:59 PM PDT, Colin McCabe > wrote: > > Hi George, > > One b

Re: [Vote] KIP-421: Support resolving externalized secrets in AbstractConfig

2019-03-23 Thread Colin McCabe
I'm not sure if this is ready for a vote yet. In particular, I don't understand how it will work in the broker. Having external secrets in the broker is something that a lot of people have been asking for -- it seems like a big omission to not talk about it at all in this KIP. I also don't un

Re: [VOTE] KIP-349 Priorities for Source Topics

2019-03-24 Thread Colin McCabe
On Sat, Mar 23, 2019, at 18:41, nathank...@gmail.com wrote: > > > On 2019/01/28 02:26:31, n...@afshartous.com wrote: > > Hi Sönke, > > > > Thanks for taking the time to review. I’ve put KIP-349 into hibernation. > > > > Thanks also to everyone who participated in the discussion. > > > > Be

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-03-27 Thread Colin McCabe
e review the new KIP. Thank you! > > On Fri, Mar 1, 2019 at 11:06 AM Colin McCabe wrote: > > > On Wed, Feb 27, 2019, at 15:53, Harsha wrote: > > > HI Colin, > > > Overlooked the IDEMPOTENT_WRITE ACL. This along with > > > client.min.version s

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-03-29 Thread Colin McCabe
st, Colin On Thu, Mar 28, 2019, at 17:36, Ying Zheng wrote: > @Colin McCabe > > I did think about that option. Yes, for most users, it's much easier to > understand Kafka version, rather than API version. However, the existing > Kafka clients only report API versions in the Kaf

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

2019-04-03 Thread Colin McCabe
rsion first, look up the proper ACL schema for that > >> version and then re-parse the ACL string with that schema - should not > >> be a real issue if the initial parsing is robust, but strictly > >> speaking we are parsing something that we don't know the schema for > >

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

2019-04-04 Thread Colin McCabe
Let's just create a JSON structure for ACLs in ZK, similar to the other JSON structures we have there. Put it under a new ZK root if we must. Call what exists today version 0, and make it so that you can't use version 1 (the first JSON version) until upgrading the IBP. What do you think? be

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

2019-04-07 Thread Colin McCabe
Hi George, As Jason was saying, it seems like there are two directions we could go here: an external system handling batching, and the controller handling batching. I think the controller handling batching would be better, since the controller has more information about the state of the system

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

2019-04-08 Thread Colin McCabe
e to just do it once. > > However, if we think there is scope for experimenting with different > > approaches, then an API that tools can use makes a lot of sense. They can > > experiment and innovate. Eventually, we can integrate something into Kafka > > if it makes sense. > &

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

2019-04-10 Thread Colin McCabe
re so that it can co-exist with the old method, > > On Mon, Oct 1, 2018 at 8:35 PM Colin McCabe wrote: > > > Hi all, > > > > With 3 binding +1s from myself, Ismael, and Gwen, the vote passes. > > > > Thanks, all. > > Colin > > > > > >

[DISCUSS] 2.3.0 release

2019-04-12 Thread Colin McCabe
Hi all, I'd like to volunteer to be the release manager for the our next feature release, the 2.3 release. If that sounds good, I'll post a release plan on Monday. cheers, Colin

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

2019-04-15 Thread Colin McCabe
lob/b4fa87cc51f0a7d640dc6ae2cc8f89f006aae652/core/src/main/scala/kafka/security/auth/Acl.scala#L37 > [3] > https://github.com/apache/kafka/blob/b4fa87cc51f0a7d640dc6ae2cc8f89f006aae652/core/src/main/scala/kafka/security/auth/Acl.scala#L66 > > > On Fri, Apr 5, 2019 at 8:09 AM Colin

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

2019-04-15 Thread Colin McCabe
AdminClient > >> > listed a few others such as ClientInUser, DefaultClientInUser. > >> > > >> > 14. Could you list any new ACL that might be required? For example, what > >> > types of operations are allowed for the new Resource (User, Client, >

[DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-04-15 Thread Colin McCabe
Hi all, We've been having discussions on a few different KIPs (KIP-236, KIP-435, etc.) about what the Admin Client replica reassignment API should look like. The current API is really hard to extend and maintain, which is a big source of problems. I think it makes sense to have a KIP that est

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-04-15 Thread Colin McCabe
link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment C. On Mon, Apr 15, 2019, at 18:07, Colin McCabe wrote: > Hi all, > > We've been having discussions on a few different KIPs (KIP-236, > KIP-435, etc.) abo

Re: [DISCUSSION] KIP-421: Automatically resolve external configurations.

2019-04-16 Thread Colin McCabe
> > will> > > > >be extended to optionally use the new AbstractConfig constructors?> > > > > > > > > Regards,> > > > > > > > > Rajini> > > > > > > > > > > > > On Mon, Mar 11, 20

Re: [DISCUSSION] KIP-421: Automatically resolve external configurations.

2019-04-17 Thread Colin McCabe
On Wed, Apr 17, 2019, at 07:49, TEJAL ADSUL wrote: > > Hi Colin, > > By default we are enabling this feature only Connect. All the other > components can enable or disable the feature as needed. Hi Tejal, I believe we should enable automatically resolving external configurations in all comp

Re: Cleaning up command line tools argument parsing a little

2019-04-17 Thread Colin McCabe
If we are going to standardize on one argument parsing library, it should certainly be argparse4j, I think. argparse4j is simply a better argument parsing library with support for more features. One example is mutually exclusive options. argparse4j supports this with MutuallyExclusiveGroup.

Re: [DISCUSS] 2.3.0 release

2019-04-17 Thread Colin McCabe
On Fri, Apr 12, 2019, at 17:09, Ismael Juma wrote: > Thanks for volunteering Colin, +1. > > Ismael > > On Sat, Apr 13, 2019 at 2:05 AM Colin McCabe wrote: > > > Hi all, > > > > I'd like to volunteer to be the release manager for the our next feature

Re: [DISCUSSION] KIP-421: Automatically resolve external configurations.

2019-04-18 Thread Colin McCabe
onnect. I have updated the KIP to enable it for all the components. > > > > Thanks, > > Tejal > > > > On 2019/04/17 23:22:50, "Colin McCabe" wrote: > > > On Wed, Apr 17, 2019, at 07:49, TEJAL ADSUL wrote: > > > > > > > > Hi

Re: [VOTE] KIP-421: Automatically resolve external configurations.

2019-04-19 Thread Colin McCabe
+1. Thanks, Tejal. best, Colin On Thu, Apr 18, 2019, at 15:02, TEJAL ADSUL wrote: > Hi All, > > As we have reached a consensus on the design, I would like to start a > vote for KIP-421. Below are the links for this proposal: > > KIP Link: > https://cwiki.apache.org/confluence/pages/viewpage.

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-04-19 Thread Colin McCabe
method to do this, to make it easier to do the conversion. We'll already need that code for the command-line tool. best, Colin > > Thanks, > Bob > > On Mon, Apr 15, 2019 at 9:33 PM Colin McCabe wrote: > > > link: > > > > https://cwiki.a

Re: [ANNOUNCE] New Kafka PMC member: Sriharsh Chintalapan

2019-04-22 Thread Colin McCabe
Congratulations, Harsh! cheers, Colin On Thu, Apr 18, 2019, at 11:46, Jun Rao wrote: > Hi, Everyone, > > Sriharsh Chintalapan has been active in the Kafka community since he became > a Kafka committer in 2015. I am glad to announce that Harsh is now a member > of Kafka PM > > Congratulations,

Re: [VOTE] KIP-421: Automatically resolve external configurations.

2019-04-22 Thread Colin McCabe
; > Hi Folks, > > > > Just a reminder that I will be closing the vote for KIP-421 by today EOD, > > please cast your votes by today. > > > > Thanks Colin, Randall, Gwen for the votes. > > > > Thanks, > > Tejal > > > > > > > > On

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

2019-04-24 Thread Colin McCabe
Hi Jose, Thanks for the KIP, looks valuable. If I use a PreferredLeaderElection RPC to specifically request an unclean leader election, will this take effect even if unclean leader elections are disabled on the topic involved? I assume that the answer is yes, but it would be good to clarify t

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

2019-04-25 Thread Colin McCabe
On Wed, Apr 24, 2019, at 21:04, Jose Armando Garcia Sancio wrote: > Thanks for the reply. Comments below. > > On Wed, Apr 24, 2019 at 6:07 PM Colin McCabe wrote: > > > Hi Jose, > > > > Thanks for the KIP, looks valuable. > > > > If I use a PreferredLea

Re: KAFKA-8238

2019-04-27 Thread Colin McCabe
Hi Sagar, I added you to the "contributors" role, so you should be able to assign JIRAs to yourself now. Take a look at any JIRA labelled "newbie" for something small to get started. There's also some "newbie+" JIRAs too, which are a little bigger. cheers, Colin On Fri, Apr 26, 2019, at 06

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-04-27 Thread Colin McCabe
ks, > Viktor > > On Sat, Apr 20, 2019 at 1:51 AM Colin McCabe wrote: > > > On Wed, Apr 17, 2019, at 17:23, Robert Barrett wrote: > > > Thanks for the KIP, Colin. I have a couple questions: > > > > > > 1. What's the reasoning for requiring cancellat

Re: [VOTE] KIP-460: Admin Leader Election RPC

2019-05-02 Thread Colin McCabe
+1 (binding) thanks, Jose. best, Colin On Wed, May 1, 2019, at 14:44, Jose Armando Garcia Sancio wrote: > Hi all, > > I would like to start the voting for KIP-460: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-460%3A+Admin+Leader+Election+RPC > > The thread discussion is here: > http

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-03 Thread Colin McCabe
inal replicas > is stored, I will modify KIP-236 accordingly for how to cancel/rollback > the reassignments.  > > Thanks, > George  > > > On Monday, April 15, 2019, 6:07:44 PM PDT, Colin McCabe > wrote: > > Hi all, > > We've been having di

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-04 Thread Colin McCabe
the new reassignments & original replicas at the > > topic/partition level. I have some concerns when controller is failing > > over, and the scalability of scanning the active reassignments from ZK > > topic/partition level nodes. Please see my reply to Jason in the KIP-236

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-06 Thread Colin McCabe
On Mon, May 6, 2019, at 07:39, Ismael Juma wrote: > Hi Colin, > > A quick comment. > > On Sat, May 4, 2019 at 11:18 PM Colin McCabe wrote: > > > The big advantage of doing batching on the controller is that the > > controller has more information about what i

Re: Cleaning up command line tools argument parsing a little

2019-05-06 Thread Colin McCabe
t sound useful to everybody? > > Best regards, > Sönke > > > On Thu, Apr 18, 2019 at 1:44 AM Colin McCabe wrote: > > > If we are going to standardize on one argument parsing library, it should > > certainly be argparse4j, I think. > > argparse4j is simp

Kafka 2.3 KIP freeze on May 10th (Friday)

2019-05-06 Thread Colin McCabe
Hi all, As per the release plan for 2.3, we're going to have a KIP freeze this week. Let's have it on Friday (May 10th). For more details about the 2.3 release, check out https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820648 cheers, Colin

Re: Cleaning up command line tools argument parsing a little

2019-05-06 Thread Colin McCabe
possible approach in case you wanted to do a bigger project. best, Colin > > Best regards, > Sönke > > > > On Mon, May 6, 2019 at 7:13 PM Colin McCabe wrote: > > > Hi Sönke, > > > > #2 is a bit tough because people have come to rely on the way th

Re: [DISCUSS] KIP-461 - Improving replica fetcher behavior in case of partition failures

2019-05-07 Thread Colin McCabe
Hi Aishwarya, This looks like a great improvement! Will a fetcher thread exit if all of its partitions have been marked failed? Or will it continue to run? After this KIP is adopted, are there any remaining situations where we would exit a fetcher thread? I guess some errors are not per-part

Re: I want to contribute to Apache Kafka

2019-05-07 Thread Colin McCabe
Hi, Sorry, JIRA doesn't seem to have any user named 15332318...@189.cn best, Colin On Sat, May 4, 2019, at 21:21, 15332318...@189.cn wrote: > Hi, > > I want to contribute to Apache Kafka. > Would you please give me the contributor permission? > My JIRA ID is 15332318...@189.cn. > > > > >

Re: [DISCUSS] KIP-461 - Improving replica fetcher behavior in case of partition failures

2019-05-08 Thread Colin McCabe
obably retry or behave just > as before. > > > On Tue, May 7, 2019 at 9:57 AM Colin McCabe wrote: > > > Hi Aishwarya, > > > > This looks like a great improvement! > > > > Will a fetcher thread exit if all of its partitions have been marked > > fail

[VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-08 Thread Colin McCabe
Hi all, I'd like to start the vote for KIP-455: Create an Administrative API for Replica Reassignment. I think this KIP is important since it will unlock many follow-on improvements to Kafka reassignment (see the "Future work" section, plus a lot of the other discussions we've had recently abo

Re: [VOTE] KIP-461 Improve Replica Fetcher behavior at handling partition failure

2019-05-08 Thread Colin McCabe
+1. Thanks, Aishwarya. Colin On Wed, May 8, 2019, at 17:50, Jason Gustafson wrote: > +1. Thanks! > > On Wed, May 8, 2019 at 4:30 PM Aishwarya Gune > wrote: > > > Hi All! > > > > I would like to call for a vote on KIP-461 that would improve the behavior > > of replica fetcher in case of partit

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-09 Thread Colin McCabe
is exist yet? > > Viktor > > On Thu, May 9, 2019 at 7:23 AM Colin McCabe wrote: > > > Hi all, > > > > I'd like to start the vote for KIP-455: Create an Administrative API for > > Replica Reassignment. I think this KIP is important since it will unlock > &

Re: [VOTE] KIP-458: Connector Client Config Override Policy

2019-05-09 Thread Colin McCabe
Hi Magesh, Thanks for the KIP. It looks good overall. >default boolean useOverrides() { >return true; >} Is this method really needed? As I understand, nobody should have any connector client config overrides set right now, since they don't do anything right now. For example

Re: [VOTE] KIP-464: Defaults for AdminClient#createTopic

2019-05-09 Thread Colin McCabe
+1 (binding). Re: the builder discussion. I don't feel strongly either way-- the builder sketched out in the KIP looks reasonable, but I can also understand Ismael's argument for keeping the KIP minimal. best, Colin On Thu, May 9, 2019, at 08:09, Randall Hauch wrote: > I'm fine with simplify

Re: [VOTE] KIP-458: Connector Client Config Override Policy

2019-05-10 Thread Colin McCabe
t; >> someone > > >> > > having "producer.override.request.timeout.ms" in their connector > > >> config > > >> > > in AK 2.2 or lower is very slim. But the key thing is if in case, > > >> someone > > >> > >

Re: [VOTE] KIP-464: Defaults for AdminClient#createTopic

2019-05-10 Thread Colin McCabe
> > > I see good use cases for the first two - the third (just replicas) > > seems > > > > less necessary but complicates the API a bit (you have to differentiate > > > > NewTopic(int) with NewTopic(short) or something like that). If we're > >

Kafka 2.3 release update

2019-05-10 Thread Colin McCabe
Hi all, Let's extend the KIP freeze deadline until tomorrow (Saturday) to give the current in-progress votes time to finish. This should be the last extension. Thanks to everyone who voted and reviewed! Feature freeze is coming up soon. Since we normally have it a week after KIP freeze, let

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-10 Thread Colin McCabe
ions for the new APIs. Good point. I think alterPartitionAssignments should require ALTER on CLUSTER. MetadataRequest permissions will be unchanged. best, Colin > > Thanks, > Jason > > On Fri, May 10, 2019 at 9:30 AM Gwen Shapira wrote: > > > +1 (binding) > &

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-10 Thread Colin McCabe
On Fri, May 10, 2019, at 17:34, Colin McCabe wrote: > On Fri, May 10, 2019, at 16:43, Jason Gustafson wrote: > > Hi Colin, > > > > I think storing reassignment state at the partition level is the right move > > and I also agree that replicas should understand that the

Re: [VOTE] KIP-461 Improve Replica Fetcher behavior at handling partition failure

2019-05-13 Thread Colin McCabe
On Mon, May 13, 2019, at 11:49, Aishwarya Gune wrote: > Thank you everyone for the discussion and voting for KIP-461 > . > Closing the vote on this KIP which passes with - I'

Re: Kafka 2.3 release update

2019-05-13 Thread Colin McCabe
apache.org/confluence/pages/viewpage.action?pageId=112820648 The next release date is the feature freeze, which is coming up this Friday. regards, Colin On Fri, May 10, 2019, at 09:50, Colin McCabe wrote: > Hi all, > > Let's extend the KIP freeze deadline until tomorrow (Saturday) t

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-16 Thread Colin McCabe
cas for all those topic/partitions in > one AlterPartitionReasisgnmentRequest.  > > > Thanks, > George > > On Friday, May 10, 2019, 8:44:31 PM PDT, Colin McCabe > wrote: > > On Fri, May 10, 2019, at 17:34, Colin McCabe wrote: > > On Fri, May 10, 2019, at 16:43, Jason

2.3 release feature freeze today

2019-05-17 Thread Colin McCabe
Hi all, As discussed earlier, the Kafka 2.3 feature freeze is today. This is the last day for getting in new features or major changes to the release. If your major feature didn't get in, don't feel bad-- there will be another release soon after this one! We will be cutting the 2.3 release b

New release branch 2.3

2019-05-20 Thread Colin McCabe
Hi all, At this point, 2.3 has hit feature freeze. At this point, we expect bugfixes and stabilization efforts to go into 2.3, but not new features. Several things slipped from this release. Just off the top of my head, this list includes KIP-359: Verify epoch in produce requests, KIP-382: M

Re: Unable to download apache kafka.

2019-05-22 Thread Colin McCabe
If this is still a problem, try contacting the Apache Infrastructure mailing list at us...@infra.apache.org. best, Colin On Sat, May 18, 2019, at 01:40, Arpan Roy wrote: > Hi, > > I'm unable to download apache kafka from https://kafka.apache.org/downloads. > The response I'm getting is timed ou

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-05-22 Thread Colin McCabe
    +1 (non-binding) > > What is the estimated timeline to have this implemented?  If any help > is needed in the implementation of cancelling reassignments,  I can > help if there is spare cycle.  > > > Thanks, > George > > > > On Thursday, May 16, 20

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

2019-05-24 Thread Colin McCabe
ne for the feedback and votes. > > Regards, > > Rajini > > > On Tue, Feb 26, 2019 at 6:41 PM Rajini Sivaram > wrote: > > > Thanks Colin, I have updated the KIP to mention that we don't return > > UNKNOWN, ANY or ALL. > > > > Regards,

Re: [ANNOUNCE] Apache Kafka 2.2.1

2019-06-04 Thread Colin McCabe
Thanks, Vahid. best, Colin On Mon, Jun 3, 2019, at 07:23, Vahid Hashemian wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.2.1 > > This is a bugfix release for Kafka 2.2.0. All of the changes in this > release can be found in the release notes: > https:

[VOTE] 2.3.0 RC1

2019-06-04 Thread Colin McCabe
Hi all, This is the first candidate for the release of Apache Kafka 2.3.0. This release includes many new features, including: * Support for incremental cooperative rebalancing * An in-memory session store and window store for Kafka Streams * An API for allowing users to determine what operations

Re: [kafka-clients] [VOTE] 2.3.0 RC1

2019-06-04 Thread Colin McCabe
On Tue, Jun 4, 2019, at 23:17, Colin McCabe wrote: > Hi all, > > This is the first candidate for the release of Apache Kafka 2.3.0. > > This release includes many new features, including: > * Support for incremental cooperative rebalancing > * An in-memory session store

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-05 Thread Colin McCabe
+1 (binding) best, Colin On Wed, Jun 5, 2019, at 03:38, Viktor Somogyi-Vass wrote: > Hi Folks, > > This vote sunk a bit, I'd like to draw some attention to this again in the > hope I get some feedback or votes. > > Thanks, > Viktor > > On Tue, May 7, 2019 at 4:28 PM Harsha wrote: > > > Than

Re: [kafka-clients] [VOTE] 2.3.0 RC1

2019-06-10 Thread Colin McCabe
2019 at 1:29 PM Guozhang Wang wrote: > > > >> Hello Colin, > >> > >> I caught an issue which would affect KIP-345 ( > >> https://issues.apache.org/jira/browse/KAFKA-8487) and hence may need to > >> be > >> considered a blocker for this rel

Re: [VOTE] KIP-476: Add Java AdminClient interface

2019-06-10 Thread Colin McCabe
Hi Andy, This is a big change, and I don't think there has been a lot of discussion about the pros and cons. What specific benefits do we get from transitioning to using an interface rather than an abstract class? If we are serious about doing this, would it be cleaner to just change AdminCli

Re: [VOTE] KIP-476: Add Java AdminClient interface

2019-06-11 Thread Colin McCabe
> > client code could override the methods they needed, like close. > > Again, this would be trivial to implement using dynamic proxies. No need > for us to implement any `DelegatingAdminClient`. If this is an interface we > empower users to do this for themselves. > > b

[VOTE] 2.3.0 RC2

2019-06-12 Thread Colin McCabe
Hi all, We discovered some problems with the first release candidate (RC1) of 2.3.0. Specifically, KAFKA-8484 and KAFKA-8500. I have created a new release candidate that includes fixes for these issues. Check out the release notes for the 2.3.0 release here: https://home.apache.org/~cmccabe/k

Re: [VOTE] 2.3.0 RC2

2019-06-13 Thread Colin McCabe
Hi all, Good news: I have run a junit test build for RC2, and it passed. Check out https://builds.apache.org/job/kafka-2.3-jdk8/51/ Also, the vote will go until Saturday, June 15th (sorry for the typo earlier in the vote end time). best, Colin On Wed, Jun 12, 2019, at 15:55, Colin McCabe

Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
Hi all, I've written up a preliminary blog post about the upcoming Apache Kafka 2.3.0 release. Take a look and let me know what you think. https://blogs.apache.org/roller-ui/authoring/preview/kafka/?previewEntry=what-s-new-in-apache cheers, Colin

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
Hmm. I'm looking to see if there's any way to open up the permissions... :| C. On Tue, Jun 18, 2019, at 13:12, M. Manna wrote: > It’s asking for credentials...? > > On Tue, 18 Jun 2019 at 15:10, Colin McCabe wrote: > > > Hi all, > > > > I've

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
Sorry, I copied the wrong URL at first. Try this URL instead: https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache best, Colin On Tue, Jun 18, 2019, at 13:17, Colin McCabe wrote: > Hmm. I'm looking to see if there's any way to open up the permissions... :|

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
Thanks, Boyang! Fixed. C. On Tue, Jun 18, 2019, at 13:43, Boyang Chen wrote: > One typo: > KIP-428: Add in-mbmory window store > should be > KIP-428: Add in-memory window store > > > ____ > From: Colin McCabe > Sent: Wednesday, June

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
: Add in-mbmory window store > > should be > > KIP-428: Add in-memory window store > > > > > > > > From: Colin McCabe > > Sent: Wednesday, June 19, 2019 4:22 AM > > To: dev@kafka.apache.org > > Subject: Re: Prelimi

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Colin McCabe
:43 Uhr schrieb Boyang Chen < > > bche...@outlook.com > > >: > > > > > One typo: > > > KIP-428: Add in-mbmory window store > > > should be > > > KIP-428: Add in-memory window store > > > > > > > > > ___

Re: [VOTE] 2.3.0 RC2

2019-06-19 Thread Colin McCabe
layer) > was filed yesterday: > > https://issues.apache.org/jira/browse/KAFKA-8564 > > Do you think this qualifies as a blocker? > > Ismael > > On Wed, Jun 12, 2019 at 3:55 PM Colin McCabe wrote: > > > Hi all, > > > > We discovered some problems

[VOTE] 2.3.0 RC3

2019-06-19 Thread Colin McCabe
Hi all, We discovered some problems with the second release candidate (RC2) of 2.3.0. Specifically, KAFKA-8564. I've created a new RC which includes the fix for this issue. Check out the release notes for the 2.3.0 release here: https://home.apache.org/~cmccabe/kafka-2.3.0-rc3/RELEASE_NOTES.h

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-20 Thread Colin McCabe
in both formats -- blog and video -- for every release > > would be helpful as different people have different preferences. > > > > Ron > > > > On Tue, Jun 18, 2019 at 8:20 PM Colin McCabe wrote: > > > >> Thanks, Konstantine. I reworked the wording

Re: [VOTE] 2.3.0 RC3

2019-06-21 Thread Colin McCabe
0 before we announce the release. No new RC is needed, we > > just have to have it on the website. > > > > Ismael > > > > On Wed, Jun 19, 2019 at 3:03 PM Colin McCabe wrote: > > > >> Hi all, > >> > >> We discovered some problems with th

Re: Possible implementation for KAFKA-560

2019-06-21 Thread Colin McCabe
I don't think this requires a change in the protocol. It seems like you should be able to use the high water mark to figure something out here? best, Colin On Fri, Jun 21, 2019, at 04:56, Carlos Manuel Duclos-Vergara wrote: > Hi, > > This is an ancient task, but I feel it is still current tod

<    1   2   3   4   5   6   7   8   9   10   >