It'a not a huge difference. Maybe just try it our yourself and click the
link "Create KIP" in step (1) here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=50859233#KafkaImprovementProposals-Process
It just uses the actual template to create a new wiki page, and opens it
in edit mode. It's a little bit more convenient than doing a c&p of the
"KIP Template" wiki page, but the difference is not big.
-Matthias
On 11/7/24 3:22 PM, Colin McCabe wrote:
Yeah, I agree, we should unify these!
I'm not sure which of the paths you described are better. Is there any advantage to the
"actual template" instead of the wiki page named template? The latter seems
easier for me to understand, but maybe others feel differently?
best,
Colin
On Thu, Nov 7, 2024, at 13:02, Matthias J. Sax wrote:
Oh I see...
The link actually goes to a "template" (this is wiki feature), while
there is also a "KIP Template" wiki page
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-Template) -- this
is not a "template" but just a regular wiki page...
Sounds like a split brain problem... We have two source of truth. I was
not aware of the actual "template", but I just copy the "KIP template"
wiki page when starting a KIP.
Seems we should unify both... Maybe simplest to get rid of the actual
"template" and update the link to point to the "KIP Template" wiki page
which is more up-to-date? Of course, using the "KIP Template" wiki page,
users need to copy the wiki page what is a little bit less convenient
than using an actual "template".
Thoughts? I guess I am fine either ways, but we should not have both.
-Matthias
On 11/6/24 8:50 PM, Sophie Blee-Goldman wrote:
It seems like the "Create KIP" button on the main KIP page actually does
not point to this template? I was actually surprised to hear that we had a
"Documentation Plan" section already because I have literally never seen
it.
I took a quick look at this button but as far as I can tell it's pointing
to the correct template. As far as I can tell there's only one
"KIP-template" page so presumably it's linked up to the correct one. I'm a
bit stumped -- anyone know what might be going on here?
(And can someone sanity check that this is happening for everyone and not
some weird local issue that only affects me?)
On Mon, Nov 4, 2024 at 2:40 PM Greg Harris <greg.har...@aiven.io.invalid>
wrote:
Hi all,
FYI the Documentation Plan was added to the template in September 2023
around the time ~979 was opened. I was not able to find any contemporary
discussion thread, does anyone know of where/when we discussed this change?
Not to say that I think the change to the template should be undone, as I
think it is very important that we write good documentation for our
features, and encourage this by changing our processes.
I think "how to teach the KIP's concepts" is more appealing to me than the
template's current wording of "List affected areas/files".
KIPs could also contain the first iteration of the user-facing
documentation, either an outline of topics that need to be documented, or
text that can later be directly promoted to a documentation page for
sufficiently small changes.
I also support moving away from our current practice of pointing users to
KIPs for first- or second-level documentation, because they are written for
contributors & committers, not end users.
As part of enforcing this practice, when reviewing KIP PRs, we could look
for places where the KIP is linked/mentioned and determine if a
purpose-written piece of documentation would be more appropriate.
Thanks,
Greg
On Mon, Nov 4, 2024 at 11:10 AM Matthias J. Sax <mj...@apache.org> wrote:
Guess it would be better to use the template when starting a new KIP --
that's why we have it :)
On 11/4/24 10:23 AM, Colin McCabe wrote:
Hi Matthias,
Thanks for pointing this out! Maybe what we need is to start enforcing
that this section appears (even if it just says "no change"). I think we
have people copying previous KIPs (including me...) and in that case just
we won't have the section, unless someone complains.
best,
Colin
On Fri, Nov 1, 2024, at 19:10, Matthias J. Sax wrote:
The KIP already has a section "Documentation Plan"
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=50859709#KIPTemplate-DocumentationPlan
If it's not used properly, committers reviewing (and voting) KIPs must
be reminded to pay more attention and emphasis on good documentation,
and also make sure the PRs contains corresponding doc updates...
Overall, I agree and support to pay attention to good docs. I just
don't
see that we need to change anything in the "official process" -- we
just
need to change our behavior and stick to the exiting process?
-Matthias
On 11/1/24 2:50 PM, Colin McCabe wrote:
On Fri, Nov 1, 2024, at 07:08, Claude Warren, Jr wrote:
I like this idea. I'm not sure what the section should be called
but
It
should spell out what changes from a customer (I don't like the term
user,
drug dealers have users -- we should have customers) point of view
and from
a developer point of view.
Hi Claude,
Sorry to nitpick, but I think "user" really is the correct term.
Apache Kafka is an open source project. We do not have "customers"
because
we're not a business. Indeed, users should feel free to contribute to
Kafka
and not view it as a vendor / customer relationship.
Of course, if you want a vendor / customer relationship, there are
lots of companies in the ecosystem that can provide that. But the open
source project should be a collaboration. I think most of us wear both
"user" and "developer" hats, and that's a good thing.
I can see cases where the change is not visible to the customer but
impacts
how developers interact with the system. I can also see cases where
the
change is almost 100% customer focused with little change to the
developers
perception.
Whatever changes are noted in the section should be accounted for in
the PR
before it is accepted. Keeping in mind that as the change evolves
through
testing to documentation requirements may change too.
I think we need more focus on documenting how to use and configure
kafka in
various environments, but I also perceive that we do not have the
people to
do that, so let's at least collect the information in some
reasonable
form.
Many changes involve multiple PRs. I think documentation is no
different than any other aspect of a feature or change. It can be done
incrementally if necessary. Hopefully having the section in the KIP would
help remind us to do it, though! And motivate discussion about what kind
of
documentation would be best.
best,
Colin
Claude
On Thu, Oct 31, 2024 at 12:21 PM Anton Agestam
<anton.ages...@aiven.io.invalid> wrote:
Thanks for your response here Colin,
> Perhaps there should be a "documentation" section in the KIP
template?
I think that would do the trick. The nice idea behind formulating
the
section as "How to teach this?", is that it leaves it to the KIP
author how
to answer it. In most cases I would expect the section to be filled
in like
"We will update documentation section X and Y", but there might be
cases
where the answer is different (. I'm not going to die on this hill,
and
would be very happy with an added "Documentation" section if that
option
has more traction 👍
the KIPs themselves are part of the documentation
I understand this is how things currently work for many parts of
Kafka
documentation, but it's an idea I want to question and I am
proposing to
work to phase this out over time. KIPs are by definition
documenting
a
proposed change. It is necessary to make assumptions about the
current
state of the Kafka code base at the time of writing, and those
assumptions
do not necessarily hold true at any arbitrary time later, when the
KIP is
being read. And I think this is what you are saying too, that for
instance
an implemented KIP that touches the protocol should also result in
changes
to the documentation.
tl;dr; It's useful to also be able to read KIPs in hind-sight, but
it
shouldn't be required to do in-brain materialization of a series of
KIPs to
understand what the expected state of some feature currently is.
What I am hoping with the proposed change to the KIP template is
that there
will be less chance for documentation to be an after-thought for
new
changes, and also for documentation changes to be scrutinized and
reviewed
during the KIP process, and that this will produce higher quality
documentation over time.
I'm curious if there are more opinions about this issue.
BR,
Anton
Den tis 29 okt. 2024 kl 20:50 skrev Colin McCabe <
cmcc...@apache.org
:
Hi Anton,
Perhaps there should be a "documentation" section in the KIP
template?
That might help raise awareness of the need to document these
changes.
I want to add, in the case of the wire protocol, the KIPs
themselves are
part of the documentation. But they shouldn't be all of the
documentation.
We should update protocol.html and possibly other docs in cases
where
we're
changing the protocol. I don't think it necessary needs to be done
before
the change itself, but it should be done so that the release that
includes
the protocol changes also includes their docs...
best,
Colin
On Sat, Oct 26, 2024, at 10:53, Anton Agestam wrote:
Hello Kafka devs 👋
Colin encouraged me in the 3.9.0 RC2 thread to contribute ideas
around
how
the protocol documentation can be improved. While I have concrete
ideas
on
this, the current biggest issue as I see it is that new changes
are not
making it into documentation, and there seems to be a bit of a
general
lack
of process with regards to this issue.
KIP-893 was a very poignant example of this. It introduces a new
concept
in
the protocol's byte serialization format, none of which made it
into
documentation. This was extremely subtle and very time consuming
to
debug
for me as an author of a third-party protocol implementation in
Python
that must remain compatible with Apache Kafka. Based on the view
of the
existing documentation, this flat out looked like a bug.
The Python ecosystem solves this issue by requiring PEPs to have
a
"How
to
teach this section", forcing documentation to not be an
afterthought. I
am
proposing to introduce the exact same concept for KIPs. I believe
this
will
be useful for all KIPs, not just those of the sort mentioned
above.
For changes to the protocol, I will also suggest that it should
be
required
for specification to be updated _before_ implementation changes
are
merged,
but this should perhaps be a separate discussion.
Forcing us to include a plan for documentation in all future KIPs
is
also a
solid strategy to incrementally improve documentation over time.
Kafka's
docs are lacking also in other regards not related to the
protocol. The
proposed change will make sure we have a net positive development
over
time, towards a greater state of Kafka docs.
Without first ensuring that there is a mechanism like this that
makes
sure
documentation does not rot over time, it doesn't seem like the
best
investment of time to improve documentation of the current state.
For
the
continued success of Kafka this is highly important because it is
what
enables a thriving ecosystem.
- Have there been something similar discussed previously?
- What do you all think of this proposal?
BR,
Anton
--
[image: Aiven] <https://www.aiven.io/>
*Anton Agestam* (he/him or they/them)
Software Engineer, *Aiven*
anton.ages...@aiven.io | +46 704 486 289
aiven.io <https://www.aiven.io/> |
<https://www.facebook.com/aivencloud>
<https://www.linkedin.com/company/aiven/> <
https://twitter.com/aiven_io>
