+1 to Ewen's suggestions: Deprecation, status and version. Perhaps add the JIRA where the KIP was implemented to the metadata. This will help tie things together.
On Fri, Jan 16, 2015 at 9:35 AM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > I think adding a section about deprecation would be helpful. A good > fraction of the time I would expect the goal of a KIP is to fix or replace > older functionality that needs continued support for compatibility, but > should eventually be phased out. This helps Kafka devs understand how long > they'll end up supporting multiple versions of features and helps users > understand when they're going to have to make updates to their applications. > > Less important but useful -- having a bit of standard metadata like PEPs > do. Two I think are important are status (if someone lands on the KIP page, > can they tell whether this KIP was ever completed?) and/or the version the > KIP was first released in. > > > > On Fri, Jan 16, 2015 at 9:20 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > >> I'm definitely +1 on the KIP concept. As Joe mentioned, we are already >> doing this in one form or the other. However, IMO it is fairly ad hoc >> - i.e., a combination of DISCUSS threads, jira comments, RB and code >> comments, wikis and html documentation. In the past I have had to dig >> into a bunch of these to try and figure out why something was >> implemented a certain way. I think KIPs can help a lot here first by >> providing guidelines on what to think about (compatibility, new APIs, >> etc.) when working through a major feature; and second by becoming a >> crisp source of truth documentation for new releases. E.g., for >> feature X: see relevant KIPs: a, b, c, etc. >> >> On Thu, Jan 15, 2015 at 08:11:20PM -0800, Jay Kreps wrote: >> > Hey Joe, >> > >> > Yeah I guess the question is what is the definition of major? I agree we >> > definitely don't want to generate a bunch of paperwork. We have enough >> > problems just getting all the contributions reviewed and checked in in a >> > timely fashion... >> > >> > So obviously bug fixes would not apply here. >> > >> > I think it is also pretty clear that big features should get reviewed and >> > discussed. To pick on myself, for example, the log compaction work was >> done >> > without enough public discussion about how it worked and why (imho). I >> > hope/claim that enough rigour in thinking about use-cases and operations >> > and so on was done that it turned out well, but the discussion was just >> > between a few people with no real public output. This kind of feature is >> > clearly a big change and something we should discuss. >> > >> > If we limit ourselves to just the public contracts the KIP introduces the >> > discussion would just be on the new configs and monitoring without >> really a >> > discussion of the design and how it works which is obviously closely >> > related. >> > >> > I don't think this should be more work because in practice we are making >> > wiki pages for any big thing anyway. So this would just be a consistent >> way >> > of numbering and structuring these pages. This would also give a clear >> call >> > to action: "hey kafka people, come read my wiki and think this through". >> > >> > I actually thinking the voting aspect is less important. I think it is >> > generally clear when there is agreement on something and not. So from my >> > point of view we could actually just eliminate that part if that is too >> > formal, it just seemed like a good way to formally adopt something. >> > >> > To address some of your comments from the wiki: >> > >> > 1. This doesn't inhibit someone coming along and putting up a patch. It >> is >> > just that when they do if it is a big thing introducing new functionality >> > we would ask for a little discussion on the basic feature/contracts prior >> > to code review. >> > >> > 2. We definitely definitely don't want people generating a lot of these >> > things every time they have an idea that they aren't going to implement. >> So >> > this is only applicable to things you absolutely will check in code for. >> We >> > also don't want to be making proposals before things are thought through, >> > which often requires writing the code. So I think the right time for a >> KIP >> > is when you are far enough along that you know the issues and tradeoffs >> but >> > not so far along that you are going to be totally opposed to any change. >> > Sometimes that is prior to writing any code and sometimes not until you >> are >> > practically done. >> > >> > The key problem I see this fixing is that there is enough new development >> > happening that it is pretty hard for everyone to review every line of >> every >> > iteration of every patch. But all of us should be fully aware of new >> > features, the ramifications, the new public interfaces, etc. If we aren't >> > aware of that we can't really build a holistic system that is beautiful >> and >> > consistent across. So the idea is that if you fully review the KIPs you >> can >> > be sure that even if you don't know every new line of code, you know the >> > major changes coming in. >> > >> > -Jay >> > >> > >> > >> > On Thu, Jan 15, 2015 at 12:18 PM, Joe Stein <joe.st...@stealth.ly> >> wrote: >> > >> > > Thanks Jay for kicking this off! I think the confluence page you wrote >> up >> > > is a great start. >> > > >> > > >> > > The KIP makes sense to me (at a minimum) if there is going to be any >> > > "breaking change". This way Kafka can continue to grow and blossom and >> we >> > > have a process in place if we are going to release a thorn ... and >> when we >> > > do it is *CLEAR* about what and why that is. We can easily document >> which >> > > KIPs where involved with this release (which I think should get >> committed >> > > afterwards somewhere so no chance of edit after release). This >> approach I >> > > had been thinking about also allows changes to occur as they do now as >> long >> > > as they are backwards compatible. Hopefully we never need a KIP but >> when >> > > we do the PMC can vote on it and folks can read the release notes with >> > > *CLEAR* understanding what is going to break their existing setup... at >> > > least that is how I have been thinking about it. >> > > >> > > >> > > Let me know what you think about this base minimum approach... I hadn't >> > > really thought of the KIP for *ANY* "major change" and I have to think >> more >> > > about that. I have some other comments for minor items in the >> confluence >> > > page I will make once I think more about how I feel having a KIP for >> more >> > > than what I was thinking about already. >> > > >> > > >> > > I do think we should have "major changes" go through confluence, >> mailing >> > > list discuss and JIRA but kind of feel we have been doing that already >> ... >> > > if there are cases where that isn't the case we should highlight and >> learn >> > > from them and formalize that more if need be. >> > > >> > > >> > > /******************************************* >> > > Joe Stein >> > > Founder, Principal Consultant >> > > Big Data Open Source Security LLC >> > > http://www.stealth.ly >> > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >> > > ********************************************/ >> > > >> > > On Thu, Jan 15, 2015 at 1:42 PM, Jay Kreps <jay.kr...@gmail.com> >> wrote: >> > > >> > > > The idea of KIPs came up in a previous discussion but there was no >> real >> > > > crisp definition of what they were. Here is an attempt at defining a >> > > > process: >> > > > >> > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals >> > > > >> > > > The trick here is to have something light-weight enough that it >> isn't a >> > > > hassle for small changes, but enough so that changes get the >> eyeballs of >> > > > the committers and heavy users. >> > > > >> > > > Thoughts? >> > > > >> > > > -Jay >> > > > >> > > >> >> > > > -- > Thanks, > Ewen
