As someone navigating the AIP process for the first time, I’d also +1 this. Throughout the process, scope has changed, new features have been released, and additional (supplemental in my case) AIP’s have been created. There has been a good bit of feedback, thoughts, and changes, but not a great place for these to live. Definitely think that a framework like this would be quite helpful!
Jake On Apr 15, 2026 at 11:02:05 AM, Jarek Potiuk <[email protected]> wrote: > +1.With some (I think) pragmatic twist proposal. > > I would even propose a little more organized step between the AIP and > implementation starts (especially with regards to the "first" in the title) > that when we decide to implement something, the first step is to create an > ADR (Architecture Decision Record) [1] capturing the most important > decisions. Capture it semi-automatically from the discussions, AIP doc, > comments etc. and store it in the repository before we create user > documentation. It should be our "Anchor" of the implementation work. > > In short it should explain, why we are doing it, what options we considered > (and discarded, including the reasons why), what the consequences are > (i.e., the ripple-effect of the change and which areas are impacted), and > of course link to the AIP for details. Also, include information about what > User Documentation should be created as part of the implementation. I think > creating user documentation "beforehand" the code and feature it implements > is extremely difficult; it's much easier to decide before what kind of > documentation needs updating and then keep updating the docs while > developing the features - especially when using Agentic assistance. > > With the agentic framework, those kinds of thoughts and discussions should > be captured as close to the code as possible. This ensures that agents > implementing features find them right where they are needed and that these > discussions serve as authoritative sources for the "scope" and "feature" to > implement. And if we add to our AGENTS.MD: "Always update the documentation > when you implement or change Airflow features." The agents will generally > just **do it** by looking at the code being submitted and reading what it > does. I've been testing it recently on pr-auto-triage and it works really > well as a workflow. > > We are increasingly describing what we want and why in plain English—not > how we do it. Keeping records of these decisions seems really important > because the code we produce becomes more of a by-product. If we keep such > records we can generally regenerate or optimize parts of the code once we > know what we are doing and why, and have that information recorded. > > I also have old ADRs for Breeze [2] which I created when we rewrote Breeze > in Python. These ADRs show what it can be, but they can now be much better. > We just point agent at the AIP - ask to summarize, review and agree on it > (which is an important step to void all kinds of ambiguties and size of > AIPs). ADRs are simply well structured markdown files, we could even add > (actuially even generate from AIPs) some high-level mermaid markdown charts > in them and Agents are capable of handling them really well. > > [1] https://en.wikipedia.org/wiki/Architectural_decision > [2] https://github.com/apache/airflow/tree/main/dev/breeze/doc/adr > > > > On Wed, Apr 15, 2026 at 11:40 AM Elad Kalif <[email protected]> wrote: > > Hello everyone, > > > Following the last dev call, I'd like to open a discussion around > > documentation first approach. > > > My key observations around Airflow 3.2 development cycle is that in order > > to do affective testing I need to understand how a feature designed to > > work. While it is listed and explained in AIP page, dev discussions and > > possibly in PR description it is not reasonable to expect community users > > to go through all of these data points. We have multiple features that are > > being developed in parallel so it's huge effort for community members. By > > having documentation ready early we help community members to be able to > > efficiently test the feature in a real production-like scenario. It will > > also help us to gain valuable and early information about if users > > understand the feature as we intended. This avoid cases where documentation > > lands around/after beta release is cut leaving little time to testing. > > > Given the AI era, it might even be easier to create and update > > documentation as development of the feature progress - it can even make PR > > review easier. > > > The Proposal: > > > 1. The default AIP draft will include a specific entry if the author thinks > > it should be under the Documentation first policy. I think it's better to > > let authors choose while the community review the choice. It will also > > mention by what part of the development documentation should be ready > > (before development. by date, by specific milestone, etc...) > > > 2. Community will review it as part of the AIP review. > > > 3. Request to include a feature / AIP under the policy may also come after > > AIP was approved if circumstances justify it. This means that any community > > member can raise the request. If the AIP author agrees then the AIP page > > will be updated accordingly. In the cases where AIP author disagree, the > > community member can raise mailing list discussion. > > > Thanks, > > Elad > > >
