This is an automated email from the ASF dual-hosted git repository. acosentino pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/camel.git
commit 813e65578de0f92ee45f31ee08ba92b47e06ee12 Author: Raffaele Marcello <[email protected]> AuthorDate: Sat Feb 13 19:21:20 2021 +0100 CAMEL-15964 create camel-google-storage component --- .../catalog/docs/google-storage-component.adoc | 41 +++--- .../GoogleCloudStorageComponentConfigurer.java | 6 + .../GoogleCloudStorageEndpointConfigurer.java | 6 + .../GoogleCloudStorageEndpointUriFactory.java | 3 +- .../component/google/storage/google-storage.json | 20 +-- .../src/main/docs/google-storage-component.adoc | 37 +++--- .../GoogleCloudStorageComponentConfiguration.java | 40 ++++-- .../google/storage/GoogleCloudStorageEndpoint.java | 9 +- .../dsl/GoogleStorageComponentBuilderFactory.java | 29 ++++- .../builder/endpoint/StaticEndpointBuilders.java | 4 +- .../GoogleCloudStorageEndpointBuilderFactory.java | 137 ++++++++++++++++++--- .../ROOT/pages/google-storage-component.adoc | 41 +++--- 12 files changed, 259 insertions(+), 114 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc index 7e9d166..8e5250c 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/google-storage-component.adoc @@ -47,16 +47,11 @@ String endpoint = "google-storage://myCamelBucket?serviceAccountKey=/home/user/D Or by providing the path to the GCP credentials file location: -[source,text] --------------------------------------------------------- -export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" --------------------------------------------------------- - -or for windows: +Provide authentication credentials to your application code by setting the environment variable `GOOGLE_APPLICATION_CREDENTIALS` : [source,text] -------------------------------------------------------- -$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" +export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" -------------------------------------------------------- @@ -67,15 +62,15 @@ $Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" google-storage://bucketNameOrArn?[options] -------------------------------------------------------- -The bucket will be created if it don't already exists. + - You can append query options to the URI in the following format, +By default the bucket will be created if it don't already exists. + +You can append query options to the URI in the following format, ?options=value&option2=value&... -For example in order to read file `hello.txt` from bucket `helloBucket`, use the following snippet: +For example in order to read file `hello.txt` from bucket `myCamelBucket`, use the following snippet: [source,java] -------------------------------------------------------------------------------- -from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&prefix=hello.txt") +from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&objectName=hello.txt") .to("file:/var/downloaded"); -------------------------------------------------------------------------------- @@ -83,7 +78,7 @@ from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-k == URI Options // component options: START -The Google Storage component supports 15 options, which are listed below. +The Google Storage component supports 16 options, which are listed below. @@ -92,17 +87,18 @@ The Google Storage component supports 15 options, which are listed below. | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean | *configuration* (common) | The component configuration | | GoogleCloudStorageComponentConfiguration -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | boolean |=== @@ -123,24 +119,25 @@ with the following path and query parameters: [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type -| *bucketName* | *Required* Bucket name | | String +| *bucketName* | *Required* Bucket name or ARN | | String |=== -=== Query Parameters (31 parameters): +=== Query Parameters (32 parameters): [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean @@ -148,7 +145,7 @@ with the following path and query parameters: | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern | *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *backoffErrorThreshold* (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. | | int | *backoffIdleThreshold* (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. | | int diff --git a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java index 649f85b..951b4cf 100644 --- a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java +++ b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfigurer.java @@ -52,6 +52,8 @@ public class GoogleCloudStorageComponentConfigurer extends PropertyConfigurerSup case "operation": getOrCreateConfiguration(target).setOperation(property(camelContext, org.apache.camel.component.google.storage.GoogleCloudStorageComponentOperations.class, value)); return true; case "serviceaccountkey": case "serviceAccountKey": getOrCreateConfiguration(target).setServiceAccountKey(property(camelContext, java.lang.String.class, value)); return true; + case "storageclass": + case "storageClass": getOrCreateConfiguration(target).setStorageClass(property(camelContext, com.google.cloud.storage.StorageClass.class, value)); return true; case "storageclient": case "storageClient": getOrCreateConfiguration(target).setStorageClient(property(camelContext, com.google.cloud.storage.Storage.class, value)); return true; case "storagelocation": @@ -87,6 +89,8 @@ public class GoogleCloudStorageComponentConfigurer extends PropertyConfigurerSup case "operation": return org.apache.camel.component.google.storage.GoogleCloudStorageComponentOperations.class; case "serviceaccountkey": case "serviceAccountKey": return java.lang.String.class; + case "storageclass": + case "storageClass": return com.google.cloud.storage.StorageClass.class; case "storageclient": case "storageClient": return com.google.cloud.storage.Storage.class; case "storagelocation": @@ -123,6 +127,8 @@ public class GoogleCloudStorageComponentConfigurer extends PropertyConfigurerSup case "operation": return getOrCreateConfiguration(target).getOperation(); case "serviceaccountkey": case "serviceAccountKey": return getOrCreateConfiguration(target).getServiceAccountKey(); + case "storageclass": + case "storageClass": return getOrCreateConfiguration(target).getStorageClass(); case "storageclient": case "storageClient": return getOrCreateConfiguration(target).getStorageClient(); case "storagelocation": diff --git a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java index edb8a63..105ba9a 100644 --- a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java +++ b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointConfigurer.java @@ -71,6 +71,8 @@ public class GoogleCloudStorageEndpointConfigurer extends PropertyConfigurerSupp case "serviceAccountKey": target.getConfiguration().setServiceAccountKey(property(camelContext, java.lang.String.class, value)); return true; case "startscheduler": case "startScheduler": target.setStartScheduler(property(camelContext, boolean.class, value)); return true; + case "storageclass": + case "storageClass": target.getConfiguration().setStorageClass(property(camelContext, com.google.cloud.storage.StorageClass.class, value)); return true; case "storageclient": case "storageClient": target.getConfiguration().setStorageClient(property(camelContext, com.google.cloud.storage.Storage.class, value)); return true; case "storagelocation": @@ -136,6 +138,8 @@ public class GoogleCloudStorageEndpointConfigurer extends PropertyConfigurerSupp case "serviceAccountKey": return java.lang.String.class; case "startscheduler": case "startScheduler": return boolean.class; + case "storageclass": + case "storageClass": return com.google.cloud.storage.StorageClass.class; case "storageclient": case "storageClient": return com.google.cloud.storage.Storage.class; case "storagelocation": @@ -202,6 +206,8 @@ public class GoogleCloudStorageEndpointConfigurer extends PropertyConfigurerSupp case "serviceAccountKey": return target.getConfiguration().getServiceAccountKey(); case "startscheduler": case "startScheduler": return target.isStartScheduler(); + case "storageclass": + case "storageClass": return target.getConfiguration().getStorageClass(); case "storageclient": case "storageClient": return target.getConfiguration().getStorageClient(); case "storagelocation": diff --git a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java index a243bfb..31332e9 100644 --- a/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java +++ b/components/camel-google-storage/src/generated/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpointUriFactory.java @@ -20,13 +20,14 @@ public class GoogleCloudStorageEndpointUriFactory extends org.apache.camel.suppo private static final Set<String> PROPERTY_NAMES; private static final Set<String> SECRET_PROPERTY_NAMES; static { - Set<String> props = new HashSet<>(32); + Set<String> props = new HashSet<>(33); props.add("backoffMultiplier"); props.add("bucketName"); props.add("destinationBucket"); props.add("storageLocation"); props.add("initialDelay"); props.add("scheduler"); + props.add("storageClass"); props.add("bridgeErrorHandler"); props.add("useFixedDelay"); props.add("runLoggingLevel"); diff --git a/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json b/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json index f8578d8..9f6761f 100644 --- a/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json +++ b/components/camel-google-storage/src/generated/resources/org/apache/camel/component/google/storage/google-storage.json @@ -24,30 +24,32 @@ "componentProperties": { "autoCreateBucket": { "kind": "property", "displayName": "Auto Create Bucket", "group": "common", "label": "common", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Setting the autocreation of the bucket bucketName." }, "configuration": { "kind": "property", "displayName": "Configuration", "group": "common", "label": "", "required": false, "type": "object", "javaType": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "deprecated": false, "autowired": false, "secret": false, "description": "The component configuration" }, - "serviceAccountKey": { "kind": "property", "displayName": "Service Account Key", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Service account key" }, - "storageClient": { "kind": "property", "displayName": "Storage Client", "group": "common", "label": "", "required": false, "type": "object", "javaType": "com.google.cloud.storage.Storage", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Set strage client" }, + "serviceAccountKey": { "kind": "property", "displayName": "Service Account Key", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Service account key to authenticate an application as a service account" }, + "storageClass": { "kind": "property", "displayName": "Storage Class", "group": "common", "label": "common", "required": false, "type": "object", "javaType": "com.google.cloud.storage.StorageClass", "deprecated": false, "autowired": false, "secret": false, "defaultValue": "STANDARD", "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Cloud Storage class to use when creat [...] + "storageClient": { "kind": "property", "displayName": "Storage Client", "group": "common", "label": "", "required": false, "type": "object", "javaType": "com.google.cloud.storage.Storage", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The storage client" }, "storageLocation": { "kind": "property", "displayName": "Storage Location", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "defaultValue": "US-EAST1", "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Cloud Storage location to use when creating the new [...] "bridgeErrorHandler": { "kind": "property", "displayName": "Bridge Error Handler", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a me [...] "deleteAfterRead": { "kind": "property", "displayName": "Delete After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Delete objects from the bucket after they have been retrieved. Th [...] "destinationBucket": { "kind": "property", "displayName": "Destination Bucket", "group": "consumer", "label": "consumer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Define the destination bucket where an object must be moved when moveAfterRe [...] - "includeBody": { "kind": "property", "displayName": "Include Body", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the Object exchange will be consumed and put into the body [...] + "includeBody": { "kind": "property", "displayName": "Include Body", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the Object exchange will be consumed and put into the body. [...] "includeFolders": { "kind": "property", "displayName": "Include Folders", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the folders\/directories will be consumed. If it is f [...] "moveAfterRead": { "kind": "property", "displayName": "Move After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Move objects from the origin bucket to a different bucket after they [...] "lazyStartProducer": { "kind": "property", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during star [...] - "objectName": { "kind": "property", "displayName": "Object Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Object name" }, + "objectName": { "kind": "property", "displayName": "Object Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Object name inside the bucket" }, "operation": { "kind": "property", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentOperations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "createDownloadLink" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.Googl [...] "autowiredEnabled": { "kind": "property", "displayName": "Autowired Enabled", "group": "advanced", "label": "advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "description": "Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which t [...] }, "properties": { - "bucketName": { "kind": "path", "displayName": "Bucket Name", "group": "common", "label": "common", "required": true, "type": "string", "javaType": "java.lang.String", "deprecated": false, "deprecationNote": "", "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Bucket name" }, + "bucketName": { "kind": "path", "displayName": "Bucket Name", "group": "common", "label": "common", "required": true, "type": "string", "javaType": "java.lang.String", "deprecated": false, "deprecationNote": "", "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Bucket name or ARN" }, "autoCreateBucket": { "kind": "parameter", "displayName": "Auto Create Bucket", "group": "common", "label": "common", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Setting the autocreation of the bucket bucketName." }, - "serviceAccountKey": { "kind": "parameter", "displayName": "Service Account Key", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Service account key" }, - "storageClient": { "kind": "parameter", "displayName": "Storage Client", "group": "common", "label": "", "required": false, "type": "object", "javaType": "com.google.cloud.storage.Storage", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Set strage client" }, + "serviceAccountKey": { "kind": "parameter", "displayName": "Service Account Key", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Service account key to authenticate an application as a service account" }, + "storageClass": { "kind": "parameter", "displayName": "Storage Class", "group": "common", "label": "common", "required": false, "type": "object", "javaType": "com.google.cloud.storage.StorageClass", "deprecated": false, "autowired": false, "secret": false, "defaultValue": "STANDARD", "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Cloud Storage class to use when crea [...] + "storageClient": { "kind": "parameter", "displayName": "Storage Client", "group": "common", "label": "", "required": false, "type": "object", "javaType": "com.google.cloud.storage.Storage", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The storage client" }, "storageLocation": { "kind": "parameter", "displayName": "Storage Location", "group": "common", "label": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "defaultValue": "US-EAST1", "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Cloud Storage location to use when creating the new [...] "bridgeErrorHandler": { "kind": "parameter", "displayName": "Bridge Error Handler", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a m [...] "deleteAfterRead": { "kind": "parameter", "displayName": "Delete After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Delete objects from the bucket after they have been retrieved. T [...] "destinationBucket": { "kind": "parameter", "displayName": "Destination Bucket", "group": "consumer", "label": "consumer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Define the destination bucket where an object must be moved when moveAfterR [...] - "includeBody": { "kind": "parameter", "displayName": "Include Body", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the Object exchange will be consumed and put into the body [...] + "includeBody": { "kind": "parameter", "displayName": "Include Body", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the Object exchange will be consumed and put into the body [...] "includeFolders": { "kind": "parameter", "displayName": "Include Folders", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "If it is true, the folders\/directories will be consumed. If it is [...] "moveAfterRead": { "kind": "parameter", "displayName": "Move After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Move objects from the origin bucket to a different bucket after the [...] "sendEmptyMessageWhenIdle": { "kind": "parameter", "displayName": "Send Empty Message When Idle", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead." }, @@ -55,7 +57,7 @@ "exchangePattern": { "kind": "parameter", "displayName": "Exchange Pattern", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.ExchangePattern", "enum": [ "InOnly", "InOut", "InOptionalOut" ], "deprecated": false, "autowired": false, "secret": false, "description": "Sets the exchange pattern when the consumer creates an exchange." }, "pollStrategy": { "kind": "parameter", "displayName": "Poll Strategy", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.PollingConsumerPollStrategy", "deprecated": false, "autowired": false, "secret": false, "description": "A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation [...] "lazyStartProducer": { "kind": "parameter", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during sta [...] - "objectName": { "kind": "parameter", "displayName": "Object Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "Object name" }, + "objectName": { "kind": "parameter", "displayName": "Object Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration", "configurationField": "configuration", "description": "The Object name inside the bucket" }, "operation": { "kind": "parameter", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.google.storage.GoogleCloudStorageComponentOperations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "createDownloadLink" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.google.storage.Goog [...] "backoffErrorThreshold": { "kind": "parameter", "displayName": "Backoff Error Threshold", "group": "scheduler", "label": "consumer,scheduler", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "description": "The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in." }, "backoffIdleThreshold": { "kind": "parameter", "displayName": "Backoff Idle Threshold", "group": "scheduler", "label": "consumer,scheduler", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "description": "The number of subsequent idle polls that should happen before the backoffMultipler should kick-in." }, diff --git a/components/camel-google-storage/src/main/docs/google-storage-component.adoc b/components/camel-google-storage/src/main/docs/google-storage-component.adoc index b40cd70..8e5250c 100644 --- a/components/camel-google-storage/src/main/docs/google-storage-component.adoc +++ b/components/camel-google-storage/src/main/docs/google-storage-component.adoc @@ -47,16 +47,11 @@ String endpoint = "google-storage://myCamelBucket?serviceAccountKey=/home/user/D Or by providing the path to the GCP credentials file location: -[source,text] --------------------------------------------------------- -export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" --------------------------------------------------------- - -or for windows: +Provide authentication credentials to your application code by setting the environment variable `GOOGLE_APPLICATION_CREDENTIALS` : [source,text] -------------------------------------------------------- -$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" +export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" -------------------------------------------------------- @@ -67,8 +62,8 @@ $Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" google-storage://bucketNameOrArn?[options] -------------------------------------------------------- -The bucket will be created if it don't already exists. + - You can append query options to the URI in the following format, +By default the bucket will be created if it don't already exists. + +You can append query options to the URI in the following format, ?options=value&option2=value&... For example in order to read file `hello.txt` from bucket `myCamelBucket`, use the following snippet: @@ -83,7 +78,7 @@ from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-k == URI Options // component options: START -The Google Storage component supports 15 options, which are listed below. +The Google Storage component supports 16 options, which are listed below. @@ -92,17 +87,18 @@ The Google Storage component supports 15 options, which are listed below. | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean | *configuration* (common) | The component configuration | | GoogleCloudStorageComponentConfiguration -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | boolean |=== @@ -123,24 +119,25 @@ with the following path and query parameters: [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type -| *bucketName* | *Required* Bucket name | | String +| *bucketName* | *Required* Bucket name or ARN | | String |=== -=== Query Parameters (31 parameters): +=== Query Parameters (32 parameters): [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean @@ -148,7 +145,7 @@ with the following path and query parameters: | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern | *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *backoffErrorThreshold* (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. | | int | *backoffIdleThreshold* (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. | | int diff --git a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java index e058a5a..737aeab 100644 --- a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java +++ b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageComponentConfiguration.java @@ -17,6 +17,7 @@ package org.apache.camel.component.google.storage; import com.google.cloud.storage.Storage; +import com.google.cloud.storage.StorageClass; import org.apache.camel.RuntimeCamelException; import org.apache.camel.spi.Metadata; import org.apache.camel.spi.UriParam; @@ -26,24 +27,28 @@ import org.apache.camel.spi.UriPath; @UriParams public class GoogleCloudStorageComponentConfiguration implements Cloneable { - @UriPath(label = "common", description = "Bucket name") + @UriPath(label = "common", description = "Bucket name or ARN") @Metadata(required = true) private String bucketName; - @UriParam(label = "common", description = "Service account key") + @UriParam(label = "common", description = "Service account key to authenticate an application as a service account") private String serviceAccountKey; @UriParam(label = "producer", enums = "copyObject,listObjects,deleteObject,deleteBucket,listBuckets,getObject,createDownloadLink") private GoogleCloudStorageComponentOperations operation; - @UriParam(label = "producer", description = "Object name") + @UriParam(label = "producer", description = "The Object name inside the bucket") private String objectName; @UriParam(label = "common", defaultValue = "US-EAST1", description = "The Cloud Storage location to use when creating the new buckets") private String storageLocation; + @UriParam(label = "common", defaultValue = "STANDARD", + description = "The Cloud Storage class to use when creating the new buckets") + private StorageClass storageClass = StorageClass.STANDARD; + @UriParam(label = "common", defaultValue = "true") private boolean autoCreateBucket = true; @@ -94,16 +99,12 @@ public class GoogleCloudStorageComponentConfiguration implements Cloneable { } /** - * The bjectName (the file insisde the bucket) + * The ObjectName (the file insisde the bucket) */ public void setObjectName(String objectName) { this.objectName = objectName; } - public Storage getStorageClient() { - return storageClient; - } - public String getStorageLocation() { return storageLocation; } @@ -116,8 +117,25 @@ public class GoogleCloudStorageComponentConfiguration implements Cloneable { this.storageLocation = storageLocation; } + public StorageClass getStorageClass() { + return storageClass; + } + + /** + * The Cloud Storage class to use when creating the new buckets + * + * @param storageClass + */ + public void setStorageClass(StorageClass storageClass) { + this.storageClass = storageClass; + } + + public Storage getStorageClient() { + return storageClient; + } + /** - * Set strage client + * The storage client * * @param storageClient */ @@ -200,8 +218,8 @@ public class GoogleCloudStorageComponentConfiguration implements Cloneable { } /** - * If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream - * will be put raw into the body and the headers will be set with the object metadata. + * If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put + * raw into the body and the headers will be set with the object metadata. */ public void setIncludeBody(boolean includeBody) { this.includeBody = includeBody; diff --git a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java index 011430d..677cd75 100644 --- a/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java +++ b/components/camel-google-storage/src/main/java/org/apache/camel/component/google/storage/GoogleCloudStorageEndpoint.java @@ -53,9 +53,6 @@ public class GoogleCloudStorageEndpoint extends ScheduledPollEndpoint { private static final Logger LOG = LoggerFactory.getLogger(GoogleCloudStorageEndpoint.class); - private static final String DEFAULT_LOCATION = "US-EAST1"; - private static final StorageClass DEFAULT_STORAGE_CLASS = StorageClass.STANDARD; - @UriParam private GoogleCloudStorageComponentConfiguration configuration; @@ -98,11 +95,11 @@ public class GoogleCloudStorageEndpoint extends ScheduledPollEndpoint { return; } else { // creates the new bucket because it doesn't exist yet - final String location = configuration.getStorageLocation() != null - ? configuration.getStorageLocation() : DEFAULT_LOCATION; + final String location = configuration.getStorageLocation(); + final StorageClass storageClass = configuration.getStorageClass(); Builder bucketBuilder = BucketInfo.newBuilder(configuration.getBucketName()) - .setStorageClass(DEFAULT_STORAGE_CLASS) + .setStorageClass(storageClass) .setLocation(location); BucketInfo bucketInfo = bucketBuilder.build(); bucket = storageClient.create(bucketInfo); diff --git a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java index 51b4ad9..afcec5a 100644 --- a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java +++ b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/GoogleStorageComponentBuilderFactory.java @@ -85,7 +85,8 @@ public interface GoogleStorageComponentBuilderFactory { return this; } /** - * Service account key. + * Service account key to authenticate an application as a service + * account. * * The option is a: <code>java.lang.String</code> type. * @@ -100,7 +101,24 @@ public interface GoogleStorageComponentBuilderFactory { return this; } /** - * Set strage client. + * The Cloud Storage class to use when creating the new buckets. + * + * The option is a: + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleStorageComponentBuilder storageClass( + com.google.cloud.storage.StorageClass storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The storage client. * * The option is a: * <code>com.google.cloud.storage.Storage</code> type. @@ -190,8 +208,8 @@ public interface GoogleStorageComponentBuilderFactory { } /** * If it is true, the Object exchange will be consumed and put into the - * body and closed. If false the Object stream will be put raw into the - * body and the headers will be set with the object metadata. + * body. If false the Object stream will be put raw into the body and + * the headers will be set with the object metadata. * * The option is a: <code>boolean</code> type. * @@ -268,7 +286,7 @@ public interface GoogleStorageComponentBuilderFactory { return this; } /** - * Object name. + * The Object name inside the bucket. * * The option is a: <code>java.lang.String</code> type. * @@ -346,6 +364,7 @@ public interface GoogleStorageComponentBuilderFactory { case "autoCreateBucket": getOrCreateConfiguration((GoogleCloudStorageComponent) component).setAutoCreateBucket((boolean) value); return true; case "configuration": ((GoogleCloudStorageComponent) component).setConfiguration((org.apache.camel.component.google.storage.GoogleCloudStorageComponentConfiguration) value); return true; case "serviceAccountKey": getOrCreateConfiguration((GoogleCloudStorageComponent) component).setServiceAccountKey((java.lang.String) value); return true; + case "storageClass": getOrCreateConfiguration((GoogleCloudStorageComponent) component).setStorageClass((com.google.cloud.storage.StorageClass) value); return true; case "storageClient": getOrCreateConfiguration((GoogleCloudStorageComponent) component).setStorageClient((com.google.cloud.storage.Storage) value); return true; case "storageLocation": getOrCreateConfiguration((GoogleCloudStorageComponent) component).setStorageLocation((java.lang.String) value); return true; case "bridgeErrorHandler": ((GoogleCloudStorageComponent) component).setBridgeErrorHandler((boolean) value); return true; diff --git a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java index 24f0a94..72fbb64 100644 --- a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java +++ b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java @@ -6707,7 +6707,7 @@ public class StaticEndpointBuilders { * Syntax: <code>google-storage:bucketName</code> * * Path parameter: bucketName (required) - * Bucket name + * Bucket name or ARN * * @param path bucketName * @return the dsl builder @@ -6728,7 +6728,7 @@ public class StaticEndpointBuilders { * Syntax: <code>google-storage:bucketName</code> * * Path parameter: bucketName (required) - * Bucket name + * Bucket name or ARN * * @param componentName to use a custom component name for the endpoint * instead of the default name diff --git a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java index 104f697..f1eac32 100644 --- a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java +++ b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/GoogleCloudStorageEndpointBuilderFactory.java @@ -81,7 +81,8 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Service account key. + * Service account key to authenticate an application as a service + * account. * * The option is a: <code>java.lang.String</code> type. * @@ -96,7 +97,41 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The Cloud Storage class to use when creating the new buckets. + * + * The option is a: + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointConsumerBuilder storageClass( + Object storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The Cloud Storage class to use when creating the new buckets. + * + * The option will be converted to a + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointConsumerBuilder storageClass( + String storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The storage client. * * The option is a: * <code>com.google.cloud.storage.Storage</code> type. @@ -112,7 +147,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The storage client. * * The option will be converted to a * <code>com.google.cloud.storage.Storage</code> type. @@ -245,8 +280,8 @@ public interface GoogleCloudStorageEndpointBuilderFactory { } /** * If it is true, the Object exchange will be consumed and put into the - * body and closed. If false the Object stream will be put raw into the - * body and the headers will be set with the object metadata. + * body. If false the Object stream will be put raw into the body and + * the headers will be set with the object metadata. * * The option is a: <code>boolean</code> type. * @@ -263,8 +298,8 @@ public interface GoogleCloudStorageEndpointBuilderFactory { } /** * If it is true, the Object exchange will be consumed and put into the - * body and closed. If false the Object stream will be put raw into the - * body and the headers will be set with the object metadata. + * body. If false the Object stream will be put raw into the body and + * the headers will be set with the object metadata. * * The option will be converted to a <code>boolean</code> * type. @@ -1043,7 +1078,8 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Service account key. + * Service account key to authenticate an application as a service + * account. * * The option is a: <code>java.lang.String</code> type. * @@ -1058,7 +1094,41 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The Cloud Storage class to use when creating the new buckets. + * + * The option is a: + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointProducerBuilder storageClass( + Object storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The Cloud Storage class to use when creating the new buckets. + * + * The option will be converted to a + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointProducerBuilder storageClass( + String storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The storage client. * * The option is a: * <code>com.google.cloud.storage.Storage</code> type. @@ -1074,7 +1144,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The storage client. * * The option will be converted to a * <code>com.google.cloud.storage.Storage</code> type. @@ -1155,7 +1225,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Object name. + * The Object name inside the bucket. * * The option is a: <code>java.lang.String</code> type. * @@ -1258,7 +1328,8 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Service account key. + * Service account key to authenticate an application as a service + * account. * * The option is a: <code>java.lang.String</code> type. * @@ -1273,7 +1344,41 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The Cloud Storage class to use when creating the new buckets. + * + * The option is a: + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointBuilder storageClass( + Object storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The Cloud Storage class to use when creating the new buckets. + * + * The option will be converted to a + * <code>com.google.cloud.storage.StorageClass</code> type. + * + * Default: STANDARD + * Group: common + * + * @param storageClass the value to set + * @return the dsl builder + */ + default GoogleCloudStorageEndpointBuilder storageClass( + String storageClass) { + doSetProperty("storageClass", storageClass); + return this; + } + /** + * The storage client. * * The option is a: * <code>com.google.cloud.storage.Storage</code> type. @@ -1289,7 +1394,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { return this; } /** - * Set strage client. + * The storage client. * * The option will be converted to a * <code>com.google.cloud.storage.Storage</code> type. @@ -1361,7 +1466,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { * Syntax: <code>google-storage:bucketName</code> * * Path parameter: bucketName (required) - * Bucket name + * Bucket name or ARN * * @param path bucketName * @return the dsl builder @@ -1381,7 +1486,7 @@ public interface GoogleCloudStorageEndpointBuilderFactory { * Syntax: <code>google-storage:bucketName</code> * * Path parameter: bucketName (required) - * Bucket name + * Bucket name or ARN * * @param componentName to use a custom component name for the endpoint * instead of the default name diff --git a/docs/components/modules/ROOT/pages/google-storage-component.adoc b/docs/components/modules/ROOT/pages/google-storage-component.adoc index bcc8d78..9b83c19 100644 --- a/docs/components/modules/ROOT/pages/google-storage-component.adoc +++ b/docs/components/modules/ROOT/pages/google-storage-component.adoc @@ -49,16 +49,11 @@ String endpoint = "google-storage://myCamelBucket?serviceAccountKey=/home/user/D Or by providing the path to the GCP credentials file location: -[source,text] --------------------------------------------------------- -export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" --------------------------------------------------------- - -or for windows: +Provide authentication credentials to your application code by setting the environment variable `GOOGLE_APPLICATION_CREDENTIALS` : [source,text] -------------------------------------------------------- -$Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" +export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json" -------------------------------------------------------- @@ -69,15 +64,15 @@ $Env:GOOGLE_APPLICATION_CREDENTIALS = "/home/user/Downloads/my-key.json" google-storage://bucketNameOrArn?[options] -------------------------------------------------------- -The bucket will be created if it don't already exists. + - You can append query options to the URI in the following format, +By default the bucket will be created if it don't already exists. + +You can append query options to the URI in the following format, ?options=value&option2=value&... -For example in order to read file `hello.txt` from bucket `helloBucket`, use the following snippet: +For example in order to read file `hello.txt` from bucket `myCamelBucket`, use the following snippet: [source,java] -------------------------------------------------------------------------------- -from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&prefix=hello.txt") +from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-key.json&objectName=hello.txt") .to("file:/var/downloaded"); -------------------------------------------------------------------------------- @@ -85,7 +80,7 @@ from("google-storage://myCamelBucket?serviceAccountKey=/home/user/Downloads/my-k == URI Options // component options: START -The Google Storage component supports 15 options, which are listed below. +The Google Storage component supports 16 options, which are listed below. @@ -94,17 +89,18 @@ The Google Storage component supports 15 options, which are listed below. | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean | *configuration* (common) | The component configuration | | GoogleCloudStorageComponentConfiguration -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | boolean |=== @@ -125,24 +121,25 @@ with the following path and query parameters: [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type -| *bucketName* | *Required* Bucket name | | String +| *bucketName* | *Required* Bucket name or ARN | | String |=== -=== Query Parameters (31 parameters): +=== Query Parameters (32 parameters): [width="100%",cols="2,5,^1,2",options="header"] |=== | Name | Description | Default | Type | *autoCreateBucket* (common) | Setting the autocreation of the bucket bucketName. | true | boolean -| *serviceAccountKey* (common) | Service account key | | String -| *storageClient* (common) | Set strage client | | Storage +| *serviceAccountKey* (common) | Service account key to authenticate an application as a service account | | String +| *storageClass* (common) | The Cloud Storage class to use when creating the new buckets | STANDARD | StorageClass +| *storageClient* (common) | The storage client | | Storage | *storageLocation* (common) | The Cloud Storage location to use when creating the new buckets | US-EAST1 | String | *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean | *deleteAfterRead* (consumer) | Delete objects from the bucket after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. | true | boolean | *destinationBucket* (consumer) | Define the destination bucket where an object must be moved when moveAfterRead is set to true. | | String -| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body and closed. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean +| *includeBody* (consumer) | If it is true, the Object exchange will be consumed and put into the body. If false the Object stream will be put raw into the body and the headers will be set with the object metadata. | true | boolean | *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean | *moveAfterRead* (consumer) | Move objects from the origin bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean | *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean @@ -150,7 +147,7 @@ with the following path and query parameters: | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern | *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy | *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...] -| *objectName* (producer) | Object name | | String +| *objectName* (producer) | The Object name inside the bucket | | String | *operation* (producer) | Set the operation for the producer. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, createDownloadLink | | GoogleCloudStorageComponentOperations | *backoffErrorThreshold* (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. | | int | *backoffIdleThreshold* (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. | | int
