Thanks for the replies! I will move forward with drafting a NiFi Improvements Proposal page to keep the discussion going.
Regards, David Handermann On Tue, Aug 20, 2024 at 3:22 PM Mark Payne <marka...@hotmail.com> wrote: > > David, > > +1 to all of this. Especially for guaranteeing compatibility and > maintainability, I think instrumenting a more > formal approach for updating the API is a step in the right direction. The > huge amount of purging, cleanup, > and refactoring that has gone into 2.0 helps to highlight the importance of > ensuring maintainability. > > Thanks > -Mark > > > > On Aug 20, 2024, at 2:00 AM, Joe Witt <joe.w...@gmail.com> wrote: > > > > David > > > > Yeah well written and good points. > > > > I dont think this says we would relax our existing commitment we have shown > > for things such as the http based api but rather is calling out specific > > things which we will make changes to even more formalized. > > > > To that end +1 on the concept of proposals and +1 to breaking out the > > nifi-api (fundamental designed extension points for flow development) in > > particular. > > > > Thanks > > Joe > > > > On Fri, Aug 16, 2024 at 8:20 PM Adam Taft <a...@adamtaft.com> wrote: > > > >> These are all really appreciated concepts, David. Thank you for putting the > >> thoughts and time in on this! > >> > >> Regarding this: > >>> The NiFi REST API does not have quite the same level of concern, but may > >> warrant inclusion. > >> > >> I hear what you're saying. However, the REST API (from my > >> observation/experience) has gathered quite a number of useful tools and > >> "hacks" for NiFi. Quite often, many different monitoring and alerting tools > >> have been developed against the REST API by third parties and/or > >> integrators of NiFi against their internal workflows. Having stable API > >> versioning in the REST API possibly makes just as much sense as having the > >> same for the nifi-api itself. This is a prime entry point for extensions > >> and other features developed alongside NiFi, maybe even the weird stuff > >> that you can't do with the nifi-api directly. > >> > >> Food for thought of course, but I would hope that we can treat the REST API > >> as a proper first-class citizen in terms of documented versioning. It turns > >> out it's quite a useful means for interacting with a running NiFi instance. > >> > >> /Adam > >> > >> On Fri, Aug 16, 2024 at 9:59 AM David Handermann < > >> exceptionfact...@apache.org> wrote: > >> > >>> Team, > >>> > >>> As we wrap up remaining items for NiFi 2, we should consider how to > >>> continue improving the quality and maintainability of the NiFi > >>> ecosystem. > >>> > >>> One primary focus of NiFi 2 has been the reduction of technical debt, > >>> which involved the removal of numerous modules and thousands of lines > >>> of code. In that process, it is worth highlighting that the core NiFi > >>> API, and the NiFi Framework API, and the NiFi REST API have had > >>> comparatively few breaking changes. This a testament to the quality of > >>> the API design itself. The NiFi Version Schema and API Compatibility > >>> [1] has provided a strong direction thus far. > >>> > >>> With that background, we should consider adopting a more formal > >>> process around changes that impact the fundamental API contracts that > >>> NiFi provides. NiFi Feature Proposals [2] have provided elements of > >>> this in the past, but did not include approval requirements. Kafka [3] > >>> and Airflow [4] have more structured improvement proposal processes, > >>> and that is what we should adopt going forward. > >>> > >>> Part of moving in this direction requires identifying the areas that > >>> would require going through the Improvement Proposal process itself. > >>> At minimum, this should include the nifi-api [5] module. The > >>> nifi-framework-api [6] is also worth consideration for inclusion in > >>> this category. The NiFi REST API does not have quite the same level of > >>> concern, but may warrant inclusion. > >>> > >>> As part of this discussion, we should consider separating the nifi-api > >>> module into its own repository, with its own versioning scheme. This > >>> will provide a helpful distinction in terms of the scope of changes, > >>> and allow the API to be released independently of the application, > >>> providing strong version compatibility guarantees. > >>> > >>> Based on feedback for the general idea, I would be glad to draft a > >>> NiFi Improvement Proposal page, outlining the recommended steps in > >>> more detail so we can come to consensus on how this should work. > >>> > >>> Regards, > >>> David Handermann > >>> > >>> [1] > >>> > >> https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme+and+API+Compatibility > >>> [2] > >>> https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > >>> [3] > >>> > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > >>> [4] > >>> > >> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals > >>> [5] https://github.com/apache/nifi/tree/main/nifi-api > >>> [6] https://github.com/apache/nifi/tree/main/nifi-framework-api > >>> > >> >
