markap14 commented on code in PR #11872:
URL: https://github.com/apache/nifi/pull/11872#discussion_r4643863508
##########
nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/replication/ClusteredConnectorRequestReplicator.java:
##########
@@ -41,30 +49,69 @@ public class ClusteredConnectorRequestReplicator implements
ConnectorRequestRepl
private static final String GET = "GET";
private final Supplier<RequestReplicator> requestReplicatorSupplier;
+ private final Supplier<ClusterCoordinator> clusterCoordinatorSupplier;
private final String replicationScheme;
- public ClusteredConnectorRequestReplicator(final
Supplier<RequestReplicator> requestReplicatorSupplier, final boolean
httpsEnabled) {
+ public ClusteredConnectorRequestReplicator(final
Supplier<RequestReplicator> requestReplicatorSupplier, final
Supplier<ClusterCoordinator> clusterCoordinatorSupplier,
+ final boolean httpsEnabled) {
this.requestReplicatorSupplier =
Objects.requireNonNull(requestReplicatorSupplier, "Request Replicator Supplier
required");
+ this.clusterCoordinatorSupplier =
Objects.requireNonNull(clusterCoordinatorSupplier, "Cluster Coordinator
Supplier required");
this.replicationScheme = httpsEnabled ? "https" : "http";
}
@Override
public ConnectorState getState(final String connectorId) throws
IOException {
+ final ConnectorEntity connectorEntity =
getConnectorEntity(connectorId, false);
Review Comment:
[Claude Opus 5.5] Running `verifyClusterMembership` here changes the
behavior of every Connector update, not just evolutions. `getState` is what
`waitForState` uses for ordinary `applyUpdate` calls, and `waitForState` now
retries `ConnectorClusterUnavailableException` every second with no limit
([line
902](https://github.com/apache/nifi/blob/4901e516579b426c6aa8767f8675c3ae17a1285d/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorRepository.java#L902)).
After a full cluster restart, the coordinator restores the nodes it knew
about and marks each one DISCONNECTED with `NOT_YET_CONNECTED` until it comes
back (`NodeClusterCoordinator.recoverState`). If one node never returns and
nobody removes it from the cluster, then every Connector update:
1. is still accepted once the user acknowledges the disconnected node,
2. stops the Connector's flow in `prepareForUpdate`,
3. then waits in PREPARING_FOR_UPDATE forever, because this check keeps
throwing.
Before this change, `getState` simply replicated to the connected nodes. If
the waiting is meant for an evolution that resumes during startup, could
ordinary updates keep the old behavior (or stop waiting after some time), with
the strict check used only on the evolution path?
##########
nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorRepository.java:
##########
@@ -586,10 +616,152 @@ private void syncFromProviderForRead(final ConnectorNode
connector) {
}
@Override
- public Future<Void> startConnector(final ConnectorNode connector) {
+ public Future<Void> startConnector(final ConnectorNode connector, final
ConnectorUpdateContext persistenceContext) {
+ Objects.requireNonNull(persistenceContext, "Flow persistence context
is required");
+
+ final String connectorId = connector.getIdentifier();
+ final CompletableFuture<Void> runningEvolution =
evolveThenStartFuturesByConnectorId.get(connectorId);
+ if (runningEvolution != null) {
+ return runningEvolution;
+ }
+ if (!connector.isPendingFlowEvolution()) {
+ return connector.start(lifecycleExecutor);
+ }
+
+ final CompletableFuture<Void> evolutionFuture = new
CompletableFuture<>();
+ final CompletableFuture<Void> existingEvolution =
evolveThenStartFuturesByConnectorId.putIfAbsent(connectorId, evolutionFuture);
+ if (existingEvolution != null) {
+ return existingEvolution;
+ }
+
+ try {
+ connector.prepareFlowEvolution(lifecycleExecutor);
+ } catch (final Throwable failure) {
+ logger.error("Failed to prepare flow evolution for {}", connector,
failure);
+ evolutionFuture.completeExceptionally(failure);
+ evolveThenStartFuturesByConnectorId.remove(connectorId,
evolutionFuture);
+ if (failure instanceof final Error error) {
+ throw error;
+ }
+ return evolutionFuture;
+ }
+
+ try {
+ flowEvolutionCoordinatorExecutor.submit(() ->
evolveThenStart(connector, evolutionFuture, persistenceContext));
+ } catch (final Throwable failure) {
+ try {
+ connector.abortUpdate(failure);
+ } catch (final Throwable cleanupFailure) {
+ failure.addSuppressed(cleanupFailure);
+ }
+ evolutionFuture.completeExceptionally(failure);
+ evolveThenStartFuturesByConnectorId.remove(connectorId,
evolutionFuture);
+ if (failure instanceof final Error error) {
+ throw error;
+ }
+ }
+
+ return evolutionFuture;
+ }
+
+ private void evolveThenStart(final ConnectorNode connector, final
CompletableFuture<Void> evolutionFuture, final ConnectorUpdateContext
persistenceContext) {
+ boolean startAttempted = false;
+ try {
+ final Future<Void> startFuture = runFlowEvolution(connector,
persistenceContext);
+ startAttempted = true;
+ // The evolution is complete once its start is issued. A later
START or STOP then acts on the connector directly.
+
evolveThenStartFuturesByConnectorId.remove(connector.getIdentifier(),
evolutionFuture);
+ startFuture.get();
+ evolutionFuture.complete(null);
+ } catch (final Throwable failure) {
+ if (failure instanceof InterruptedException) {
+ Thread.currentThread().interrupt();
+ }
+
+ if (startAttempted) {
+ logger.error("{} was evolved but did not start", connector,
failure);
+ } else {
+ logger.error("Flow evolution failed for {}", connector,
failure);
+ }
+
+ handleFlowEvolutionFailure(connector, evolutionFuture, failure,
startAttempted);
+
+ if (failure instanceof final Error error) {
+ throw error;
+ }
+ } finally {
+
evolveThenStartFuturesByConnectorId.remove(connector.getIdentifier(),
evolutionFuture);
+ }
+ }
+
+ /**
+ * Runs the evolve-drain phase of the evolution, then applies the upgraded
flow as a configuration update.
+ *
+ * @return the future for starting the evolved connector
+ */
+ private Future<Void> runFlowEvolution(final ConnectorNode connector, final
ConnectorUpdateContext persistenceContext) throws Throwable {
+ awaitFlowEvolutionDrain(connector,
connector.evolveFlow(flowEvolutionCoordinatorExecutor));
+
+ // No node applies the upgraded flow until every node that still has
the outgoing flow has drained it.
+ waitForFlowEvolutionStatus(connector);
+
+ // Every node has drained, so as at startup the preparation is local
and no cluster-wide wait for UPDATING is needed.
+ // A peer that has not yet observed the drain barrier may still be
DRAINING or preparing.
+ connector.beginEvolvedFlowUpdate();
+ connector.prepareForUpdate();
+ completeUpdate(connector, persistenceContext,
Set.of(ConnectorState.PREPARING_FOR_UPDATE, ConnectorState.UPDATING,
ConnectorState.DRAINING));
Review Comment:
[Claude Opus 5.5] There's a window here where the cluster can end up split
between the old and new flows:
1. Nodes A and B both finish draining. A's next poll of the drain barrier
sees `EVOLVE_DRAIN_COMPLETE`, so A applies the upgraded flow, reaches UPDATED,
and waits here for the whole cluster to reach UPDATED.
2. B is still sleeping between its own barrier polls, which back off to 30
seconds.
3. B's next poll throws an `IOException` rather than the retried
`ConnectorClusterUnavailableException`, for example because another node
disconnected or one node's GET failed. B's coordinator calls `abortUpdate`, and
since B is still DRAINING, it goes back to STOPPED on the old flow.
4. A's `waitForState` reads the merged cluster state. In
`ConnectorEntityMerger`, UPDATED has priority 3 and STOPPED has priority 1
([lines
264–266](https://github.com/apache/nifi/blob/4901e516579b426c6aa8767f8675c3ae17a1285d/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/manager/ConnectorEntityMerger.java#L264-L266)),
so the merged state is still UPDATED. A treats that as success and starts the
upgraded flow.
A now runs the upgraded flow while B sits stopped on the old one, and the
START reports success. No FlowFiles are lost and another START fixes it, but
nothing tells the user. A failure on B after it begins applying is caught,
because UPDATE_FAILED wins the merge. Only the fall back to STOPPED is
invisible.
One option: while waiting for UPDATED during an evolution, also check the
merged flow evolution status and fail if it is `FAILED`, which B reports once
its attempt fails.
##########
nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/registry/flow/mapping/VersionedComponentFlowMapper.java:
##########
@@ -1102,33 +1108,48 @@ private org.apache.nifi.flow.ScheduledState
mapScheduledState(final ScheduledSta
}
/**
- * Map the given ConnectorNode to a VersionedConnector. When the Connector
is in Troubleshooting state, the Connector's
- * Managed Process Group is also serialized into the VersionedConnector so
that any user modifications made while in
- * Troubleshooting mode survive a restart. The provided
ControllerServiceProvider is required for mapping the Managed
- * Process Group; if {@code null}, the Managed Process Group is not
serialized even if the Connector is in Troubleshooting.
+ * Map the given ConnectorNode to a VersionedConnector. The parameter
context, managed-flow bundle, and active
+ * configuration are written from the node's atomic snapshot. While the
connector is in Troubleshooting, the live
+ * managed group is written so the user's edits survive a restart;
otherwise the managed group comes from that
+ * snapshot. The provided ControllerServiceProvider is required for the
live mapping; if {@code null} while
+ * Troubleshooting, the managed group is omitted.
*
* @param connectorNode the connector node to map
- * @param controllerServiceProvider the controller service provider used
when mapping the Managed Process Group; may be null
+ * @param controllerServiceProvider the controller service provider used
when mapping the live Managed Process Group; may be null
* @return the mapped VersionedConnector
*/
public VersionedConnector mapConnector(final ConnectorNode connectorNode,
final ControllerServiceProvider controllerServiceProvider) {
final VersionedConnector versionedConnector = new VersionedConnector();
versionedConnector.setInstanceIdentifier(connectorNode.getIdentifier());
versionedConnector.setName(connectorNode.getName());
-
versionedConnector.setScheduledState(mapConnectorState(connectorNode.getDesiredState()));
+ versionedConnector.setScheduledState(mapConnectorState(connectorNode));
versionedConnector.setType(connectorNode.getCanonicalClassName());
+
+ final ConnectorFlowSnapshot authoritativeSnapshot =
connectorNode.getAuthoritativeFlowSnapshot();
+ final FrameworkFlowContext activeFlowContext =
connectorNode.getActiveFlowContext();
versionedConnector.setBundle(mapBundle(connectorNode.getBundleCoordinate()));
- final List<VersionedConfigurationStep> activeFlowConfiguration =
createVersionedConfigurationSteps(connectorNode.getActiveFlowContext());
+ final ConnectorConfiguration activeConfiguration =
authoritativeSnapshot == null
+ ?
activeFlowContext.getConfigurationContext().toConnectorConfiguration()
+ : authoritativeSnapshot.activeConfiguration();
Review Comment:
[Claude Opus 5.5] **Blocker.** I think this can silently undo a Connector
migration. Once a Connector holds a snapshot, every flow save writes the active
configuration and managed flow from the snapshot instead of from the live
Connector. Flow sync gives every Connector a snapshot on startup and on cluster
join ([`StandardConnectorRepository` line
321](https://github.com/apache/nifi/blob/4901e516579b426c6aa8767f8675c3ae17a1285d/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorRepository.java#L321)).
But migration never refreshes it:
[`applyMigratedConfiguration`](https://github.com/apache/nifi/blob/4901e516579b426c6aa8767f8675c3ae17a1285d/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorNode.java#L345)
rebuilds the managed flow by calling the Connector's `applyUpdate` directly,
and [`commitMigratedConfiguration`](https://github.com
/apache/nifi/blob/4901e516579b426c6aa8767f8675c3ae17a1285d/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorNode.java#L384-L403)
writes the merged configuration onto the live active configuration. Neither
touches the snapshot.
So:
1. Create a Connector.
2. Restart NiFi. In a cluster, it's enough for any node to join after the
Connector was created.
3. Migrate a Process Group into the Connector. The save that follows writes
the pre-migration configuration and the initial flow.
4. On the next restart, the Connector is rebuilt from that old
configuration. The migration is gone, MIGRATE is offered again, and the source
Process Group has already been disabled and renamed. FlowFiles that were queued
in the migrated flow may also be dropped, because their connections no longer
exist.
`ConnectorVersionedFlowMigrationRestartIT` doesn't catch this because it
migrates before any restart, when no snapshot exists yet. Could
`commitMigratedConfiguration` capture a fresh snapshot once the merged
configuration is committed? And could that test restart once before migrating?
##########
nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorNode.java:
##########
@@ -1619,6 +1773,240 @@ public boolean isExtensionMissing() {
return extensionMissing;
}
+ @Override
+ public boolean isPendingFlowEvolution() {
+ return flowEvolution.isPending();
+ }
+
+ @Override
+ public FlowEvolutionStatus getReportedFlowEvolutionStatus() {
+ return flowEvolution.getReportedStatus();
+ }
+
+ boolean hasFlowEvolutionFailed() {
+ return flowEvolution.hasFailed();
+ }
+
+ @Override
+ public ConnectorFlowSnapshot getAuthoritativeFlowSnapshot() {
+ return flowEvolution.getSnapshot();
+ }
+
+ @Override
+ public boolean restorePendingFlowEvolution(final VersionedConnector
persisted, final List<VersionedConfigurationStep> activeConfiguration,
+ final List<VersionedConfigurationStep> workingConfiguration)
throws FlowUpdateException {
+ final Bundle managedFlowBundle = resolveManagedFlowBundle(persisted);
+ final VersionedProcessGroup persistedManagedGroup =
persisted.getManagedProcessGroup();
+ final Map<String, SecretReference> parameterSecretReferences =
ParameterSecretReferences.convertManagedParameterSecretReferences(persisted);
+ // Components in the persisted group may name NAR versions that are no
longer installed. A missing coordinate
+ // is rewritten only when exactly one compatible bundle is installed.
A ghost has no initialization context,
+ // and its coordinates have to stay as persisted.
+ if (!extensionMissing) {
+ resolvePersistedBundles(persistedManagedGroup);
+ }
+
+ if (extensionMissing && persistedManagedGroup != null) {
+ // Adopt before inheritConfiguration. A flow save with no snapshot
omits the managed group, and inherit
+ // rebuilds the live group before the persisted one is installed
below.
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ return false;
+ }
+
+ if (!extensionMissing && !sameManagedBundle(managedFlowBundle) &&
persistedManagedGroup != null) {
+ // Skip inheritConfiguration. Its applyUpdate would replace the
pre-upgrade group with the new bundle's
+ // flow. The restored group keeps the previous bundle, so the next
start evolves it. A failed restore still loads
+ // the configuration and adopts the persisted snapshot, so the
next flow save keeps the pre-upgrade flow.
+ try {
+ restorePersistedFlow(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
ResumeComponentState.RESUME_AS_PERSISTED);
+ } catch (final Exception e) {
+ logger.error("{} failed to restore the persisted pre-upgrade
managed flow; the Connector cannot start until NiFi is restarted and the flow
is restored", this, e);
+ markManagedFlowRestoreFailed();
+ }
+
+ loadConfigurationsWithoutRebuildingFlow(activeConfiguration,
workingConfiguration);
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ return true;
+ }
+
+ return false;
+ }
+
+ @Override
+ public void restorePersistedManagedFlow(final VersionedConnector
persisted) {
+ if (!extensionMissing) {
+ throw new IllegalStateException("Cannot restore the persisted
managed flow for " + this + " because its extension is not missing");
+ }
+
+ final VersionedProcessGroup persistedManagedGroup =
persisted.getManagedProcessGroup();
+ if (persistedManagedGroup == null) {
+ return;
+ }
+
+ final Bundle managedFlowBundle = resolveManagedFlowBundle(persisted);
+ final Map<String, SecretReference> parameterSecretReferences =
ParameterSecretReferences.convertManagedParameterSecretReferences(persisted);
+
+ // Install the persisted flow while ghosted so queues exist before the
FlowFile repository recovers.
+ // Components stay stopped: a ghost cannot start, and resuming would
run them under a missing extension.
+ try {
+ restorePersistedFlow(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
ResumeComponentState.LEAVE_STOPPED);
+ } catch (final Exception e) {
+ logger.error("{} failed to restore the persisted managed flow
while ghosted; "
+ + "queued FlowFiles may not rejoin until NiFi restarts
after the Connector becomes loadable again", this, e);
+ markManagedFlowRestoreFailed();
+ }
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ }
+
+ @Override
+ public void restoreTroubleshootingManagedFlow(final VersionedConnector
persisted) throws FlowUpdateException {
+ final Bundle managedFlowBundle = resolveManagedFlowBundle(persisted);
+ final VersionedProcessGroup persistedManagedGroup =
persisted.getManagedProcessGroup();
+ final Map<String, SecretReference> parameterSecretReferences =
ParameterSecretReferences.convertManagedParameterSecretReferences(persisted);
+
+ if (extensionMissing && persistedManagedGroup != null) {
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ restoreTroubleshootingFlow(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences);
+ return;
+ }
+
+ if (sameManagedBundle(managedFlowBundle) && persistedManagedGroup !=
null) {
+ try {
+ restorePersistedFlow(persistedManagedGroup, null, Map.of(),
ResumeComponentState.RESUME_AS_PERSISTED);
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ } catch (final Exception e) {
+ logger.error("{} failed to restore the managed flow while
restoring Troubleshooting", this, e);
+ throw new FlowUpdateException("Failed to restore Managed
Process Group from persisted flow: " + e.getMessage(), e);
+ }
+ }
+ }
+
+ @Override
+ public void ensureAuthoritativeFlowSnapshot() {
+ flowEvolution.ensureSnapshot();
+ }
+
+ private void restoreTroubleshootingFlow(final VersionedProcessGroup
persistedManagedGroup, final VersionedParameterContext parameterContext,
+ final Map<String, SecretReference> parameterSecretReferences)
throws FlowUpdateException {
+ logger.info("{} was persisted in Troubleshooting mode; restoring
Managed Process Group from persisted snapshot", this);
+ try {
+ restorePersistedFlow(persistedManagedGroup, parameterContext,
parameterSecretReferences, ResumeComponentState.LEAVE_STOPPED);
+ } catch (final Exception e) {
+ logger.error("{} failed to restore the managed flow while
restoring Troubleshooting", this, e);
+ throw new FlowUpdateException("Failed to restore Managed Process
Group from persisted flow: " + e.getMessage(), e);
+ }
+ }
+
+ /**
+ * A group saved before managedFlowBundle existed is attributed to the
connector bundle stored in the flow definition.
+ */
+ private static Bundle resolveManagedFlowBundle(final VersionedConnector
versionedConnector) {
+ if (versionedConnector.getManagedFlowBundle() != null) {
+ return versionedConnector.getManagedFlowBundle();
+ }
+ if (versionedConnector.getManagedProcessGroup() != null) {
+ return versionedConnector.getBundle();
Review Comment:
[Claude Opus 5.5] This fallback has a side effect for Connectors that were
in Troubleshooting under an older NiFi. Before this PR, `flow.json` stored only
`managedProcessGroup` for a Connector in Troubleshooting, with no
`managedFlowBundle` and no `managedParameterContext`. If NiFi is upgraded and
the Connector's NAR version changes too, which is the usual case for Connectors
that ship with NiFi:
1. This method returns the old Connector bundle, so
`restorePendingFlowEvolution` treats the upgrade as pending.
2. `restorePersistedFlow` runs with a null parameter context, so
`StandardFlowContext.finishRestorePersistedFlow` sets no parameter values.
3. `inheritConfiguration`, which used to fill the managed Parameter Context
before the Troubleshooting group was restored on top, is skipped.
The Connector comes back with an empty (or initial-flow) managed Parameter
Context, so components that reference parameters are invalid, and processors
saved as RUNNING can't start. Leaving Troubleshooting recovers, but that's a
confusing first experience after an upgrade. Could this case keep the old order
(inherit the configuration, then restore the Troubleshooting group on top), or
otherwise fill the parameters when the persisted parameter context is missing?
##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ConnectorResource.java:
##########
@@ -725,6 +728,19 @@ public Response updateRunStatus(
);
}
+ /**
+ * Fills the coordinator's loaded bundle onto the run-status request so
each node can reject a START in the
+ * validation phase when its own loaded bundle differs. Avoids an extra
replicated GET.
+ */
+ private void populateExpectedBundle(final String connectorId, final
ConnectorRunStatusEntity requestRunStatus) {
+ final ConnectorEntity localConnector =
serviceFacade.getConnector(connectorId);
Review Comment:
[Claude Opus 5.5] Reading the bundle through `serviceFacade.getConnector`
causes two problems:
1. The entity is built with the requesting user's permissions, and
`EntityFactory.createConnectorEntity` only sets the component when the user can
read the Connector. A user with only the operate permission passes the
run-status authorization but gets a null component here, so `expectedBundle` is
never set and no node checks for a mismatched bundle.
2. `getConnector(id)` uses `SYNC_WITH_PROVIDER`, so on the coordinator only,
this START pulls any external edit from the configuration provider into the
working configuration. Every other node verifies and starts with `LOCAL_ONLY`.
The upgraded flow is built from the working configuration, so the coordinator
can apply a different flow than its peers. That's the situation the comment in
`StandardNiFiServiceFacade.verifyUpdateConnector` says start must avoid.
Reading the bundle straight from the local `ConnectorNode` with
`ConnectorSyncMode.LOCAL_ONLY`, without going through the permission-filtered
entity, would avoid both.
##########
nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/connector/StandardConnectorNode.java:
##########
@@ -1619,6 +1773,240 @@ public boolean isExtensionMissing() {
return extensionMissing;
}
+ @Override
+ public boolean isPendingFlowEvolution() {
+ return flowEvolution.isPending();
+ }
+
+ @Override
+ public FlowEvolutionStatus getReportedFlowEvolutionStatus() {
+ return flowEvolution.getReportedStatus();
+ }
+
+ boolean hasFlowEvolutionFailed() {
+ return flowEvolution.hasFailed();
+ }
+
+ @Override
+ public ConnectorFlowSnapshot getAuthoritativeFlowSnapshot() {
+ return flowEvolution.getSnapshot();
+ }
+
+ @Override
+ public boolean restorePendingFlowEvolution(final VersionedConnector
persisted, final List<VersionedConfigurationStep> activeConfiguration,
+ final List<VersionedConfigurationStep> workingConfiguration)
throws FlowUpdateException {
+ final Bundle managedFlowBundle = resolveManagedFlowBundle(persisted);
+ final VersionedProcessGroup persistedManagedGroup =
persisted.getManagedProcessGroup();
+ final Map<String, SecretReference> parameterSecretReferences =
ParameterSecretReferences.convertManagedParameterSecretReferences(persisted);
+ // Components in the persisted group may name NAR versions that are no
longer installed. A missing coordinate
+ // is rewritten only when exactly one compatible bundle is installed.
A ghost has no initialization context,
+ // and its coordinates have to stay as persisted.
+ if (!extensionMissing) {
+ resolvePersistedBundles(persistedManagedGroup);
+ }
+
+ if (extensionMissing && persistedManagedGroup != null) {
+ // Adopt before inheritConfiguration. A flow save with no snapshot
omits the managed group, and inherit
+ // rebuilds the live group before the persisted one is installed
below.
+ adoptAuthoritativeSnapshot(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
managedFlowBundle);
+ return false;
+ }
+
+ if (!extensionMissing && !sameManagedBundle(managedFlowBundle) &&
persistedManagedGroup != null) {
+ // Skip inheritConfiguration. Its applyUpdate would replace the
pre-upgrade group with the new bundle's
+ // flow. The restored group keeps the previous bundle, so the next
start evolves it. A failed restore still loads
+ // the configuration and adopts the persisted snapshot, so the
next flow save keeps the pre-upgrade flow.
+ try {
+ restorePersistedFlow(persistedManagedGroup,
persisted.getManagedParameterContext(), parameterSecretReferences,
ResumeComponentState.RESUME_AS_PERSISTED);
+ } catch (final Exception e) {
+ logger.error("{} failed to restore the persisted pre-upgrade
managed flow; the Connector cannot start until NiFi is restarted and the flow
is restored", this, e);
+ markManagedFlowRestoreFailed();
+ }
+
+ loadConfigurationsWithoutRebuildingFlow(activeConfiguration,
workingConfiguration);
Review Comment:
[Claude Opus 5.5] If `loadConfigurationsWithoutRebuildingFlow` throws here,
`adoptAuthoritativeSnapshot` on the next line never runs. That can happen when
the new Connector version's `onConfigurationStepConfigured` rejects the
persisted working configuration, since `notifyStepConfigured` rethrows. At
startup the Connector has no snapshot yet, so it ends up with none at all.
`syncConnector` marks it invalid and moves on, and then:
1. The next flow save writes no `managedProcessGroup`, `managedFlowBundle`,
or `managedParameterContext`.
2. `isPendingFlowEvolution()` becomes false, so the guard in `start()` no
longer refuses a direct start of the new code against the pre-upgrade flow.
3. After another restart, `resolveManagedFlowBundle` returns null,
`inheritConfiguration` builds the upgraded flow with no drain, and FlowFiles in
connections that the upgraded flow doesn't have are dropped.
The comment above already says a failed restore should still adopt the
persisted snapshot. Only this failure skips it. Adopting the snapshot before
loading the configuration, or in a `finally`, would keep the pre-upgrade flow
saved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]