I think this is a good proposal. I would also like to share my experience as an integration consultant implementing various integration systems for large enterprises.
In general, these organizations are very conservative. The milestone releases (like those for 2.0) are completely ignored. It doesn't matter if they are already running in production in other organizations. The first GA release of a new major release isn't trusted either. Only after a couple of minor releases and when there is a new long-term support release, they start to think. Upgrading often disrupts day-to-day operations, business requirements, and consumes resources. Only when an LTS arrives, the pressure from the security/dev team is high enough, the project budget is available, it fits into the company roadmap, and so on... does a migration/upgrade begin. With thousands of flows and daily operations, all the migration and retesting can take a long time. I think most organizations expect that version 1.x will be supported for many years, next to the new major that has just arrived. I'm not saying it's good, or possible, or safe, etc., but those are just the timelines and expectations in large organizations. Companies like Microsoft and Oracle understand this very well. They often offer (and charge for) very long timelines for their operating systems and software. An open source project relies mostly on other open source projects. These may or may not have an EOL. The problem on top of this with integration software like NiFi is that, by its nature, it is always connected to external systems, and mostly exposed to the Internet by connecting to many systems. Thus make it very clear what the risks are, but still offer bug fixes for some time, is probably the best way forward. Raymond On Tue, Nov 5, 2024 at 8:31 PM Joe Witt <joe.w...@gmail.com> wrote: > To be actionable and concrete here is a proposal: > > 1. Declare the NiFi 1.28.x line as the 'End of Life/End of Support' line. > This means we may still do periodic bug fixes or when possible/reasonable > bump vulnerable libs. But we will not be doing further analysis/triage of > security reports nor adding features. > > 2. Add a DISCLAIMER to the source and key binaries of the 1.x/1.28 line > [1]. > > 3. Update the downloads page making the links for nifi source and the main > nifi assembly of whatever is the latest NiFi 1.28.x release there but > clearly articulated as the end of support line for which bug fixes and some > narrow dependency updates may occur. Advise users of the 1.x line of the > importance of planning to migrate to the 2.x line. > > 4. Conduct a VOTE to codify this. > > 5. Conduct an Apache NiFi 1.28.1 release to pickle up (2) and the bug fixes > already available. > > 6. As we gather more user input on things which are helpful to them we > factor these into migration guidance/tooling as appropriate on the 2.x > line. > > Thanks > > [1] https://github.com/apache/nifi/pull/9491 > > On Tue, Nov 5, 2024 at 9:04 AM Joe Witt <joe.w...@gmail.com> wrote: > > > Team, > > > > I will start a discussion thread on the users list so we can hear more > > inputs from them and from that perspective. > > > > This thread needs to focus on what the contributors/committers/PMC in the > > community can/will/should do and the PMC in particular as we're obligated > > to ensure we're putting out software for which we can stand behind its > > security posture. > > > > We do not need to get worried about customers. They have vendors that > > support them. What we need to worry about and continue to do an > excellent > > job caring for is the apache nifi user base and we need to ensure they > > don't have the belief that the NiFi 1.x line will be fixed in the > presence > > of vulnerability reports. I'll ask on the users list how folks would > like > > us to communicate about the state of things. > > > > What I think we need to ask here is more in the spirit of what this > thread > > was started about. When do we as a contributor/committer/PMC base want > to > > make it official in our own sense that we will not be producing releases > > for the 1.x line? How we best communicate/help the user base then > follows > > from that. Stated another way those who feel they will be in a good > > position to do security reviews, vulnerability scans and remediation, > > conduct releases for some period of time please share what you think > you'll > > be able to do and roughly for how long. > > > > Thanks > > > > On Tue, Nov 5, 2024 at 3:36 AM Pierre Villard < > pierre.villard...@gmail.com> > > wrote: > > > >> Hi all, > >> > >> While I think we could set an EOL date a bit further in the future, it > is > >> important to keep in mind what EOL means. It only means we won't be > >> providing security fixes / bug fixes through new releases. It does not > >> mean that NiFi 1.x is gone. If that is a big concern for some users when > >> running EOL software then we should remind those users that they've been > >> doing it for 2+ years already when using NiFi 1.x (taking Jetty as an > >> example here). And Joe is definitely right when saying that we have a > >> smaller and smaller group of people willing to spend an extensive amount > >> of > >> time taking care of PR/reviews, of release candidates, > testing/validating > >> RCs, etc, for the 1.x line. > >> > >> I also agree that, even if many users are already using NiFi 2 in > >> production, many places have strict policies to not adopt a new major > >> release. I don't want to start a debate whether this is making sense or > >> not > >> but we know those rules exist in many places :) And the fact that we had > >> milestone releases for one year is not going to be enough of an > argument. > >> > >> Given what we've seen in the past, we usually make a new release every 3 > >> months or so. It's probably fair to assume a 2.1.0 release will happen > >> early next year. With that in mind, I tend to agree with Michael > >> suggesting > >> an EOL date at the end of January (3 months from now). We could also say > >> that 1.28.1 will happen at this time and will be the last one in the > >> community. > >> > >> Vendors have already announced support for NiFi 1.x for multiple > >> additional > >> years so this approach follows what we see in other projects where > >> extended > >> support is only provided through paid options with specific companies. > >> > >> It is awesome to finally see 2.0 out and this decision will help drive > >> users to that new release, which is much better in so many ways... > >> > >> Thanks, > >> Pierre > >> > >> > >> Le mar. 5 nov. 2024 à 10:36, Isha Lamboo < > isha.lam...@virtualsciences.nl> > >> a > >> écrit : > >> > >> > Hi all, > >> > > >> > I understand the reasons to declare an EOL quickly, given the external > >> > dependencies, but like Russell said before the short notice is going > to > >> > cause trouble with our bigger corporate customers. It would have been > >> nice > >> > to have the EOL date announced about a year ago, even if it had been a > >> > provisional one. The more you can delay it now, the less credibility I > >> (and > >> > NiFi itself) lose :-\ > >> > > >> > I've been pushing since the first announcement of NiFi 2.0 for our > >> > customers to prepare. The smaller NiFi instances are all prepared. But > >> > there are also big customers with hundreds of flows that depend on > >> > variables and XML templates, and as you can imagine this was never a > >> > priority for them without either a NiFi 2.0 GA to move to or an actual > >> EOL > >> > date to get security officers upping the priority. > >> > > >> > Now we have a GA release finally, but corporate Q4 plans are set in > >> stone > >> > and Q1 2025 plans are already filling up. Telling the customers' > >> > development teams to upend their plans and tell their business > >> customers to > >> > forget deliveries because NiFi needs to be fixed ASAP is probably not > >> going > >> > to fly and instead going to seriously dent NiFi's reputation and > >> position. > >> > Unless we can automate the flow migration process it's going to be a > >> > year-long migration at least. > >> > > >> > That said, are there any tools or scripts to make the migration > >> smoother? > >> > Configuring multiple levels of parameter contexts with inheritance is > a > >> > labor-intensive process if we are to mirror the current setup with > >> > variables being inherited from main canvas, team PG, subject PG and > flow > >> > PG, etc. Anything that could go through the process groups and > configure > >> > this automatically would be greatly appreciated. I will look into that > >> > myself too, but anything helps really. > >> > > >> > Regards, > >> > > >> > Isha > >> > > >> > -----Oorspronkelijk bericht----- > >> > Van: Joe Witt <joe.w...@gmail.com> > >> > Verzonden: maandag 4 november 2024 23:44 > >> > Aan: dev@nifi.apache.org > >> > Onderwerp: Re: [DISCUSS] End-of-life timing for NiFi 1 > >> > > >> > The EOL discussion is not here because we have a new problem. It is > >> here > >> > because we finally have an answer. > >> > > >> > The inability to address reported vulnerabilities or fundamental end > of > >> > life status for key underlying components in the 1.x line is a problem > >> that > >> > was fully recognized three years ago. > >> > > >> > In that time we created a plan for what NiFi 2.0 would be and how we'd > >> > manage both maintaining the 1.x line while building to the 2.x GA. In > >> the > >> > past year we've conducted four milestone releases of NiFi 2.x and > we've > >> > continued putting out feature, bug fix, and security improvement > >> releases > >> > of 1.x. > >> > > >> > Feature bearing releases of 1.x are no longer appropriate as 2.x is > here > >> > and GA and that was the plan all along. > >> > > >> > Bug fixes are still reasonable in spirit but you need people to submit > >> the > >> > JIRAs, fix the JIRAs, peer review the changes, and to conduct releases > >> and > >> > make votes. That is in increasingly short supply as it has been quite > >> the > >> > task splitting attention across two major lines and naturally > developers > >> > and users will gravitate toward the go forward path. > >> > > >> > Vulnerability/Security related considerations are where things are > >> > fundamentally problematic. We had a security report today about the > >> super > >> > old/outdated front-end libraries we use in 1.x. That won't change. > We > >> had > >> > a report last week about Spring libraries needing updated except you > >> can't > >> > unless you have Pivotal support so not an option. Those won't change. > >> We > >> > have had questions around Jetty changes but that is tied to Java 8. > >> We've > >> > had questions about Java 8 being end of life and even Java 11 and even > >> now > >> > Java 17 in terms of its codebase permissive licensing changing. The > >> things > >> > we can reasonably address in the 1.x line are getting smaller and > >> smaller > >> > and the time required to address any new thing is higher and higher. > >> > > >> > We as a community, regardless of good intentions, cannot fix the > >> illities > >> > of the 1.x line and thus the 2.x line is here. The 1.x line will > >> > absolutely continue to atrophy and it will accelerate. If we do not > >> signal > >> > EOL on 1.x that means we're saying we can keep fixing problems. While > >> that > >> > is true for bugs, that is not true for vulnerabilities broadly and for > >> our > >> > most critical components. > >> > > >> > If you still fix bugs people assume this means you still reasonably > fix > >> > vulnerabilities/etc.. And unless we declare EOL on the 1.x line we > will > >> > continue to get non-serviceable reports and mislead the user base. > >> > > >> > The answer is to clearly signal that users should transition to the > 2.x > >> > line and focus our help on answering questions people might have on > how > >> to > >> > do that. > >> > > >> > I am supportive of EOL for the 1.x line. I also like the poetic > nature > >> of > >> > the decade timing. > >> > > >> > On Mon, Nov 4, 2024 at 2:47 PM David Handermann < > >> > exceptionfact...@apache.org> > >> > wrote: > >> > > >> > > >
