Xinyu Wang created NIFI-16444:
---------------------------------
Summary: Property migration creates a new Controller Service and
overwrites a parameter reference when the referenced parameter has no value
Key: NIFI-16444
URL: https://issues.apache.org/jira/browse/NIFI-16444
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 2.5.0
Reporter: Xinyu Wang
When a processor property that identifies a Controller Service is set to a
parameter reference (for example {{#\{aws-cs\}}}) and the parameter has no
value at the time the flow is loaded, {{migrateProperties}} treats the property
as unset. Processors whose migration creates a default Controller Service for
an unset property then create one and overwrite the parameter reference with
the new service ID.
The overwrite is written back to the flow, so the parameter reference is lost
permanently. Setting the parameter value afterwards and restarting does not
restore it.
h3. Cause
{{StandardPropertyConfiguration.isPropertySet()}} evaluates the effective value:
{code:java}
public boolean isPropertySet(final String propertyName) {
// Use Effective Properties here because the value may be set to
#{MY_PARAM} but if parameter MY_PARAM is not set, the property should be
considered unset.
return effectiveProperties.get(propertyName) != null;
}
{code}
{{AbstractAWSCredentialsProviderProcessor.migrateAuthenticationProperties()}}
(and the equivalent code in {{AbstractAwsProcessor}}) then does:
{code:java}
if (!config.isPropertySet(AWS_CREDENTIALS_PROVIDER_SERVICE)) {
...
} else {
final String serviceId =
config.createControllerService(CREDENTIALS_SERVICE_CLASSNAME, Map.of(
AUTH_SERVICE_ANONYMOUS_CREDENTIALS, "true"));
config.setProperty(AWS_CREDENTIALS_PROVIDER_SERVICE, serviceId);
}
}
{code}
A property set to {{#\{param\}}} is configured by the user, so it should not be
treated as unset. The newly created service has no credentials, and replaces
the intended service.
The same {{isPropertySet}} pattern guards {{createControllerService}} in
{{ProxyServiceMigration}}, {{SchemaRegistryService}},
{{SchemaRegistryRecordSetWriter}} and {{PutElasticsearchRecord}}.
h3. Steps to reproduce (NiFi 2.5.0)
# Create a Parameter Context with parameter {{aws-cs}} whose value is the ID of
an existing AWSCredentialsProviderControllerService, and assign it to the root
group.
# Add a FetchS3Object processor and set {{AWS Credentials Provider service}} to
{{#\{aws-cs\}}}.
# Stop NiFi and remove the {{value}} field of parameter {{aws-cs}} from
{{flow.json.gz}} (this simulates the parameter having no value at startup, for
example a Parameter Provider that has not fetched yet).
# Start NiFi.
Expected: the property remains {{#\{aws-cs\}}}.
Actual: a new AWSCredentialsProviderControllerService with
{{anonymous-credentials=true}} and no keys is created and the property is
replaced by its ID. The log contains:
{noformat}
o.a.n.m.StandardControllerServiceFactory Created
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=...],
name=AWSCredentialsProviderControllerService, ...] as a step in the migration
of FetchS3Object[id=...]
{noformat}
With the parameter value present, restarting creates no new service. After the
failure, setting the parameter value and restarting again leaves the processor
pointing at the new service.
h3. Proposed fix
Do not change {{isPropertySet()}}, since other migrations rely on a parameter
without a value being treated as unset for plain properties. Instead, the
migration code should check whether a raw value exists for properties that
identify a Controller Service, for example:
{code:java}
if (config.getRawPropertyValue(AWS_CREDENTIALS_PROVIDER_SERVICE).isEmpty()) {
{code}
A one-line change of this kind in {{AbstractAWSCredentialsProviderProcessor}}
was verified on 2.5.0: with the parameter value removed, no service is created
and the property keeps {{#\{aws-cs\}}}; restoring the value and restarting also
keeps it. The other migration call sites listed above need the same review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
