Team, As we wrap up remaining items for NiFi 2, we should consider how to continue improving the quality and maintainability of the NiFi ecosystem.
One primary focus of NiFi 2 has been the reduction of technical debt, which involved the removal of numerous modules and thousands of lines of code. In that process, it is worth highlighting that the core NiFi API, and the NiFi Framework API, and the NiFi REST API have had comparatively few breaking changes. This a testament to the quality of the API design itself. The NiFi Version Schema and API Compatibility [1] has provided a strong direction thus far. With that background, we should consider adopting a more formal process around changes that impact the fundamental API contracts that NiFi provides. NiFi Feature Proposals [2] have provided elements of this in the past, but did not include approval requirements. Kafka [3] and Airflow [4] have more structured improvement proposal processes, and that is what we should adopt going forward. Part of moving in this direction requires identifying the areas that would require going through the Improvement Proposal process itself. At minimum, this should include the nifi-api [5] module. The nifi-framework-api [6] is also worth consideration for inclusion in this category. The NiFi REST API does not have quite the same level of concern, but may warrant inclusion. As part of this discussion, we should consider separating the nifi-api module into its own repository, with its own versioning scheme. This will provide a helpful distinction in terms of the scope of changes, and allow the API to be released independently of the application, providing strong version compatibility guarantees. Based on feedback for the general idea, I would be glad to draft a NiFi Improvement Proposal page, outlining the recommended steps in more detail so we can come to consensus on how this should work. Regards, David Handermann [1] https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme+and+API+Compatibility [2] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals [3] https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals [4] https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals [5] https://github.com/apache/nifi/tree/main/nifi-api [6] https://github.com/apache/nifi/tree/main/nifi-framework-api
