turcsanyip commented on code in PR #9753:
URL: https://github.com/apache/nifi/pull/9753#discussion_r1977650094
##########
nifi-extension-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/drive/ListGoogleDrive.java:
##########
@@ -150,6 +177,29 @@ public class ListGoogleDrive extends
AbstractListProcessor<GoogleDriveFileInfo>
.dependsOn(LISTING_STRATEGY, BY_ENTITIES)
.build();
+ public static final PropertyDescriptor ENRICH_FILE_METADATA = new
PropertyDescriptor.Builder()
Review Comment:
@exceptionfactory Thanks for sharing your suggestions!
The original idea was to avoid impacting the current behavior and
performance.
The extra metadata needs to be retrieved from the Google Drive service (same
query but additional fields) which may affect response time. Then the metadata
fields are added in the processors output as FlowFile attributes or record
fields in content which increases its size.
I did some measurements and it seems the performance loss is not
significant: 7-8% overhead in case of 1000 files in a folder, basically the
same performance (or unmeasurable difference) in case of 10 or 100 files.
Regarding the FF size: it is roughly doubled with the new fields but the
size is still small (261 vs 561 KB for 1000 files in record mode).
All in all, I'm inclined to remove the configurable property and add the new
metadata fields in a static way. That makes the code much simpler too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
