exceptionfactory commented on a change in pull request #5098:
URL: https://github.com/apache/nifi/pull/5098#discussion_r674831997
##########
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java
##########
@@ -226,9 +239,12 @@ protected String getPath(ProcessContext context) {
options.setRecursive(recurseSubdirectories);
Pattern baseDirectoryPattern = Pattern.compile("^" + baseDirectory
+ "/?");
+ final String tempFilePrefix =
defaultIfBlank(System.getProperty("tempFilePrefix"), TEMP_FILE_PREFIX);
Review comment:
Thanks for the clarification @timeabarna.
It seems like there are multiple competing use cases here, so thank you for
working through the feedback.
On the one hand, the purpose of the Temporary File Prefix seems to be
something internal to NiFi so that the PutAzureDataLakeStorage can write files
with a temporary name that ListAzureDataLakeStorage will not pick up. For that
use case, having an internal shared value, that cannot be changed, seems like
the most logical approach.
One the other hand, if other components or services outside of NiFi may
write files that could be considered temporary, then perhaps more configuration
is necessary using a standard processor property descriptor. However, since
ListAzureDataLakeStorage already has a `File Filter` property, it seems like
that could be used to exclude temporary files from other sources.
If the goal is to support maximum configurability in
`PutAzureDataLakeStorage` when writing temporary files, then it should be a
processor property. However, if the temporary file prefix is specific enough to
NiFi, using something like `AzureDataLakeStorage.` or perhaps prefixed with a
period as `.AzureDataLakeStorage.`, then that seems like it should cover the
intended use cases without needing to be configured.
Perhaps @jfrazee has some additional thoughts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
