Hi Pierre, Thanks for sharing all these details and thanks to all who worked on it. This will be a great addition to the NiFi framework.
I just did a quick scroll through and I saw you mentioned that the API is experimental for now and can break going forward until it stabilizes. I saw (and anticipate) Manifest and FlowDefinition changes, are you expecting these to break as well in a non backward compatible manner? I didn't do a proper review yet. I just saw that the connector related constructs were introduced. As this is a core nifi change would this mean a NiFi 3.0 major version bump when it is complete or it is still too early for that discussion. Thanks, Csaba On Thu, Feb 19, 2026 at 2:22 PM Pierre Villard <[email protected]> wrote: > Hi everyone, > > I wanted to bring up a discussion around releasing the initial > implementation of the Connectors feature that was introduced in NIP-11 > [1]. The initial discussion happened on the mailing list back in > August 2025 [2], followed by a vote in September 2025 to start the > implementation work [3]. Since then, significant progress has been > made over the past few months and we believe the feature is ready for > an initial release. > > As a reminder, the Connector concept introduces a new extension point > in Apache NiFi that encapsulates a complete NiFi dataflow behind a > higher-level, guided configuration experience. Rather than exposing > the full complexity of a flow to the user, a Connector presents a > structured set of Configuration Steps and Property Groups (for > example: "Source Configuration", "Destination Configuration", > "Transformations") that allow a user to configure an end-to-end data > pipeline without needing to understand or manipulate the underlying > flow directly. Under the hood, the Connector is responsible for > dynamically building and reconfiguring its encapsulated flow based on > the user's configuration choices. This includes managing the lifecycle > (starting/stopping) of all components within the flow. The goal is to > significantly simplify the user experience for common data movement > use cases, making NiFi more accessible to a broader audience while > also streamlining operations for experienced practitioners. > > Given the scope of the feature, the API is marked as experimental and > may change in a non-backward compatible manner between minor versions > until the API has been more fully validated through developing > concrete Connector implementations.All of the development work > happened in a feature branch named NIFI-15258 in both the nifi-api [4] > and nifi [5] repositories. Here is a high-level summary of what was > implemented: > > 1. NiFi API - The Connector API has been introduced under > org.apache.nifi.components.connector. This includes the core Connector > interface and an AbstractConnector base class, the ConfigurationStep > and ConnectorPropertyGroup abstractions for structuring configuration, > ConnectorPropertyDescriptor for defining properties with dependencies > and allowable values, a FlowContext abstraction that provides both an > active flow and a working flow for configuration verification, > component facades (ProcessorFacade, ControllerServiceFacade, > ConnectionFacade, ProcessGroupFacade, ParameterContextFacade) for > Connectors to interact with their encapsulated flow, Secret references > for handling sensitive values, Asset support, ConnectorMethod for > exposing custom web endpoints, documentation writers for > auto-generating Connector documentation, and updates to the Versioned > flow model to accommodate Connectors. Comprehensive unit tests have > been added for the API. > > 2. NiFi Maven NAR Plugin - Support has been added for creating > Connector extension manifests in NAR files, so that the NiFi runtime > can discover and load Connector extensions. This change is already on > the main branch. > > 3. NiFi (Framework, REST API, UI, Tests) - The bulk of the > implementation resides here: > - Framework core: ConnectorNode and lifecycle management, connector > facades for standalone process groups, a connector configuration > repository, flow synchronization support for connectors, parameter > context and secrets management integration, and cluster replication > support. > - REST API: Full set of endpoints for connector CRUD operations, > lifecycle management (start/stop), configuration step retrieval and > updates, property group management, status reporting, data drainage, > connector actions, secrets, state management, verification, and > documentation. > - Frontend UI: Connectors are integrated into the flow designer, the > documentation browser, and access policies. > - Mock framework: A connector mock server and test runner to support > unit and integration testing of Connector implementations. > - Example connector: A Kafka-to-S3 connector implementation that > serves as a reference for building Connectors. > - CLI Toolkit: A ConnectorClient has been added for command-line > interaction with Connectors. > - System tests: 11 system test classes covering standalone and > clustered scenarios including CRUD, lifecycle, assets, data drainage, > parameter contexts, version resolution, and method marshalling. > > Release plan: > > We would like to proceed iteratively with three releases: > - NiFi Maven NAR Plugin - This has no dependency on the other > projects. The Connector manifest support is already on main so we can > get started with a release candidate right away. I volunteer to take > care of the release duties. > - NiFi API - A pull request [6] has been filed to merge the NIFI-15258 > feature branch into main. Once merged, we will proceed with a release > candidate. > - NiFi - Once the NiFi API release is published, we will proceed with > the proper PRs (bumping the API version dependency and merging the > NIFI-15258 feature branch), and we will look at releasing a new > version of NiFi. > > Please have a look at the feature branches and share your feedback. > Any comments, questions, or concerns are welcome. > > Thanks, > Pierre > > [1] https://issues.apache.org/jira/browse/NIP-11 > [2] https://lists.apache.org/thread/77c7o4yf8lt9owpjsjb8gl2slk3f7mcf > [3] https://lists.apache.org/thread/skhn5kjkq6kfmrbqsshxs5348d968jvb > [4] https://github.com/apache/nifi-api/tree/NIFI-15258 > [5] https://github.com/apache/nifi/tree/NIFI-15258 > [6] https://github.com/apache/nifi-api/pull/70 >
