Hi Matthias, Thanks for your input and for pointing that out.
It at least is missing from this wiki page, and so is a link to the KIP template: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=50859233#KafkaImprovementProposals-WhatshouldbeincludedinaKIP ? Perhaps there should be a bullet point added there and/or the link to the template? BR, Anton Den lör 2 nov. 2024 03:11Matthias J. Sax <mj...@apache.org> skrev: > The KIP already has a section "Documentation Plan" > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=50859709#KIPTemplate-DocumentationPlan > > If it's not used properly, committers reviewing (and voting) KIPs must > be reminded to pay more attention and emphasis on good documentation, > and also make sure the PRs contains corresponding doc updates... > > Overall, I agree and support to pay attention to good docs. I just don't > see that we need to change anything in the "official process" -- we just > need to change our behavior and stick to the exiting process? > > > -Matthias > > On 11/1/24 2:50 PM, Colin McCabe wrote: > > On Fri, Nov 1, 2024, at 07:08, Claude Warren, Jr wrote: > >> I like this idea. I'm not sure what the section should be called but It > >> should spell out what changes from a customer (I don't like the term > user, > >> drug dealers have users -- we should have customers) point of view and > from > >> a developer point of view. > > > > Hi Claude, > > > > Sorry to nitpick, but I think "user" really is the correct term. Apache > Kafka is an open source project. We do not have "customers" because we're > not a business. Indeed, users should feel free to contribute to Kafka and > not view it as a vendor / customer relationship. > > > > Of course, if you want a vendor / customer relationship, there are lots > of companies in the ecosystem that can provide that. But the open source > project should be a collaboration. I think most of us wear both "user" and > "developer" hats, and that's a good thing. > > > >> > >> I can see cases where the change is not visible to the customer but > impacts > >> how developers interact with the system. I can also see cases where the > >> change is almost 100% customer focused with little change to the > developers > >> perception. > >> > >> Whatever changes are noted in the section should be accounted for in > the PR > >> before it is accepted. Keeping in mind that as the change evolves > through > >> testing to documentation requirements may change too. > >> > >> I think we need more focus on documenting how to use and configure > kafka in > >> various environments, but I also perceive that we do not have the > people to > >> do that, so let's at least collect the information in some reasonable > form. > >> > > > > Many changes involve multiple PRs. I think documentation is no different > than any other aspect of a feature or change. It can be done incrementally > if necessary. Hopefully having the section in the KIP would help remind us > to do it, though! And motivate discussion about what kind of documentation > would be best. > > > > best, > > Colin > > > >> Claude > >> > >> On Thu, Oct 31, 2024 at 12:21 PM Anton Agestam > >> <anton.ages...@aiven.io.invalid> wrote: > >> > >>> Thanks for your response here Colin, > >>> > >>> > Perhaps there should be a "documentation" section in the KIP > template? > >>> > >>> I think that would do the trick. The nice idea behind formulating the > >>> section as "How to teach this?", is that it leaves it to the KIP > author how > >>> to answer it. In most cases I would expect the section to be filled in > like > >>> "We will update documentation section X and Y", but there might be > cases > >>> where the answer is different (. I'm not going to die on this hill, and > >>> would be very happy with an added "Documentation" section if that > option > >>> has more traction 👍 > >>> > >>>> the KIPs themselves are part of the documentation > >>> > >>> I understand this is how things currently work for many parts of Kafka > >>> documentation, but it's an idea I want to question and I am proposing > to > >>> work to phase this out over time. KIPs are by definition documenting a > >>> proposed change. It is necessary to make assumptions about the current > >>> state of the Kafka code base at the time of writing, and those > assumptions > >>> do not necessarily hold true at any arbitrary time later, when the KIP > is > >>> being read. And I think this is what you are saying too, that for > instance > >>> an implemented KIP that touches the protocol should also result in > changes > >>> to the documentation. > >>> > >>> tl;dr; It's useful to also be able to read KIPs in hind-sight, but it > >>> shouldn't be required to do in-brain materialization of a series of > KIPs to > >>> understand what the expected state of some feature currently is. > >>> > >>> What I am hoping with the proposed change to the KIP template is that > there > >>> will be less chance for documentation to be an after-thought for new > >>> changes, and also for documentation changes to be scrutinized and > reviewed > >>> during the KIP process, and that this will produce higher quality > >>> documentation over time. > >>> > >>> I'm curious if there are more opinions about this issue. > >>> > >>> BR, > >>> Anton > >>> > >>> Den tis 29 okt. 2024 kl 20:50 skrev Colin McCabe <cmcc...@apache.org>: > >>> > >>>> Hi Anton, > >>>> > >>>> Perhaps there should be a "documentation" section in the KIP template? > >>>> That might help raise awareness of the need to document these changes. > >>>> > >>>> I want to add, in the case of the wire protocol, the KIPs themselves > are > >>>> part of the documentation. But they shouldn't be all of the > >>> documentation. > >>>> We should update protocol.html and possibly other docs in cases where > >>> we're > >>>> changing the protocol. I don't think it necessary needs to be done > before > >>>> the change itself, but it should be done so that the release that > >>> includes > >>>> the protocol changes also includes their docs... > >>>> > >>>> best, > >>>> Colin > >>>> > >>>> > >>>> On Sat, Oct 26, 2024, at 10:53, Anton Agestam wrote: > >>>>> Hello Kafka devs 👋 > >>>>> > >>>>> Colin encouraged me in the 3.9.0 RC2 thread to contribute ideas > around > >>>> how > >>>>> the protocol documentation can be improved. While I have concrete > ideas > >>>> on > >>>>> this, the current biggest issue as I see it is that new changes are > not > >>>>> making it into documentation, and there seems to be a bit of a > general > >>>> lack > >>>>> of process with regards to this issue. > >>>>> > >>>>> KIP-893 was a very poignant example of this. It introduces a new > >>> concept > >>>> in > >>>>> the protocol's byte serialization format, none of which made it into > >>>>> documentation. This was extremely subtle and very time consuming to > >>> debug > >>>>> for me as an author of a third-party protocol implementation in > Python > >>>>> that must remain compatible with Apache Kafka. Based on the view of > the > >>>>> existing documentation, this flat out looked like a bug. > >>>>> > >>>>> The Python ecosystem solves this issue by requiring PEPs to have a > "How > >>>> to > >>>>> teach this section", forcing documentation to not be an > afterthought. I > >>>> am > >>>>> proposing to introduce the exact same concept for KIPs. I believe > this > >>>> will > >>>>> be useful for all KIPs, not just those of the sort mentioned above. > >>>>> > >>>>> For changes to the protocol, I will also suggest that it should be > >>>> required > >>>>> for specification to be updated _before_ implementation changes are > >>>> merged, > >>>>> but this should perhaps be a separate discussion. > >>>>> > >>>>> Forcing us to include a plan for documentation in all future KIPs is > >>>> also a > >>>>> solid strategy to incrementally improve documentation over time. > >>> Kafka's > >>>>> docs are lacking also in other regards not related to the protocol. > The > >>>>> proposed change will make sure we have a net positive development > over > >>>>> time, towards a greater state of Kafka docs. > >>>>> > >>>>> Without first ensuring that there is a mechanism like this that makes > >>>> sure > >>>>> documentation does not rot over time, it doesn't seem like the best > >>>>> investment of time to improve documentation of the current state. For > >>> the > >>>>> continued success of Kafka this is highly important because it is > what > >>>>> enables a thriving ecosystem. > >>>>> > >>>>> - Have there been something similar discussed previously? > >>>>> - What do you all think of this proposal? > >>>>> > >>>>> BR, > >>>>> Anton > >>>>> > >>>>> -- > >>>>> [image: Aiven] <https://www.aiven.io/> > >>>>> *Anton Agestam* (he/him or they/them) > >>>>> Software Engineer, *Aiven* > >>>>> anton.ages...@aiven.io | +46 704 486 289 > >>>>> aiven.io <https://www.aiven.io/> | > >>>>> <https://www.facebook.com/aivencloud> > >>>>> <https://www.linkedin.com/company/aiven/> < > >>>> https://twitter.com/aiven_io> > >>>> > >>> >
