[
https://issues.apache.org/jira/browse/NIFI-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133680#comment-18133680
]
Alaksiej Ščarbaty commented on NIFI-16310:
------------------------------------------
_Mentioning the behavior change in Jira, as suggested in the PR._
*Behavior change:* while synchronizing a flow, a controller service that the
proposed flow doesn't contain is now removed only if no remaining component
references it, directly or through other removed services, once property
migration has finished. Previously it was always removed, even while a
component still referenced it.
The visible effect is with nested versioned child groups that aren't updated.
If a processor in this child group references a controller service in the
parent-group service that the new version drops, the service is now kept, and
an info message names the components that still reference it. Previously the
service was deleted and the processor was left invalid.
> A controller service created by migrateProperties is removed on versioned
> group upgrade
> ---------------------------------------------------------------------------------------
>
> Key: NIFI-16310
> URL: https://issues.apache.org/jira/browse/NIFI-16310
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
> Affects Versions: 2.11.0
> Reporter: Alaksiej Ščarbaty
> Assignee: Alaksiej Ščarbaty
> Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> h2. Problem
> A Controller Service created by a component's {{migrateProperties}} is
> removed and recreated whenever the version of a versioned Process Group
> changes. During that the service's state is lost.
> A stateless service is unaffected by that. Whereas for stateful service it
> causes a silent state loss: the replacement service is correctly configured
> and enabled, but it holds no state from the removed controller service.
> _Related jiras:_ NIFI-14995 and NIFI-16149 stopped these services being
> reported as local modifications when a flow is compared with its registered
> version. That handling does not extend to applying a version change.
> h2. Root cause
> {{StandardControllerServiceFactory.create}} does not assign a versioned
> component id to the service it creates.
> {{StandardVersionedComponentSynchronizer}} generates a new versioned id from
> the service's instance identifier for any controller services without a
> versioned id. Then it removes every local controller service whose version id
> is absent from the proposed snapshot. There is no exception for controller
> services created in {{{}migrateProperties{}}}.
> h2. Proposed change
> The case to support first: a Process Group is upgraded to a version that
> declares the Controller Service which was already created locally in
> {{{}migrateProperties{}}}. The service must survive with its current state,
> and the flow must end up with one controller service, not two.
> # Synchronization must not delete a Controller Service that was created in
> {{{}migrateProperties{}}}.
> # This local Controller Service must be set with a versioned component id
> assigned to that Controller Service in the versioned process group.
> The matching of Controller Services should go through the component that
> references them: if a component's Controller Service property points at a
> migration-created service now, and the same property in the new version
> points at a declared service of the same type in the same group, they must be
> treated as the same service.
> After matching the created Controller Service looks identical to its
> counterpart in the versioned process group, and it should carry no signs of
> being created in {{migrateProperties.}}
> h3. Other matching strategies
> Neither the identifier nor the versioned component id can be used to match
> these two services.
> Matching by class and property values work well only when controller service
> created by {{migrateProperties}} is identical to a service in the versioned
> group (see edge cases for properties drift). This matching strategy also
> won't work if there are 2 identical controller services in the versioned
> group.
> h2. Edge cases
> This solution creates a lot of edge cases, most of which are caused by
> Controller Service implementation/properties drift.
> Normally a versioned process group author must ensure there is no properties
> drift between a Controller Service created in {{migrateProperties}} and the
> Service from versioned process group.
> For each edge case I provide synchronization behavior.
> *New version does not declare the service.* Keep the local service,
> unversioned. Currently it is removed and then recreated empty by calling
> {{{}migrateProperties{}}}.
> *Declared service has different property values* {*}than the local
> service{*}{*}.{*} The local service becomes versioned, then new properties
> are applied to it.
> *A single declared service is present instead of several local services.* The
> first controller service becomes versioned, the rest are kept as local
> services.
> The following cases are related to Controller Services drift when versioned
> Controller Services don't match local Controller Services created in
> {{{}migrateProperties{}}}.
> If that happens, we create the declared services from the versioned group,
> which are used by the components. Local Controller Services are kept and not
> removed. The drift has to be resolved manually.
> # *New version declares a service of a different implementation type than
> the local service.*
> # *Declared service is in a different process group from the local service.*
> # *A single local service is referenced by several components.*
> # *Referencing component is absent from the new version, either deleted or
> itself a migration-created service with no versioned component id.*
> h2. Reproduction
> Reproduced against 2.12.0-SNAPSHOT on main.
> System tests:
> [https://github.com/awelless/nifi/tree/migration-created-cs-versioning-tests]
> {{MigrationCreatedControllerServiceVersioningIT}} uses a processor whose
> {{migrateProperties}} creates a stateful Controller SErvice.
> * {{testFlowUpgradePreservesMigrationCreatedControllerService}} - fails: the
> service is replaced by the one the new version declares;
> *
> {{testFlowUpgradeAddingUnrelatedProcessorPreservesMigrationCreatedControllerService}}
> - fails: the service is recreated under the same id with an empty store;
> * -{{testFlowDowngradePreservesMigrationCreatedControllerService}} - fails
> at its precondition, because the upgrade it performs first already removes
> the service, leaving the downgrade path unverified.- During downgrade we
> remove a Controller Service, as it's done for normal Controller Services.
> Two further tests pass, showing the NiFi upgrade and a plain NiFi restart are
> unaffected.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
