jack created AIRFLOW-3538:
-----------------------------
Summary: BigQueryToCloudStorageOperator to support bucket
Key: AIRFLOW-3538
URL: https://issues.apache.org/jira/browse/AIRFLOW-3538
Project: Apache Airflow
Issue Type: Improvement
Components: gcp, operators
Affects Versions: 2.0.0
Reporter: jack
Currently
{code:java}
BigQueryToCloudStorageOperator {code}
does not support bucket property.
This means that
{code:java}
destination_cloud_storage_uris {code}
needs the full path e.g
{code:java}
gs://bucket....{code}
This behavior isn't consistent with other operators like:
GoogleCloudStorageDownloadOperator
MySqlToGoogleCloudStorageOperator
GoogleCloudStorageToBigQueryOperator
and many more... all support bucket and don't require the user to define the
full path but only relative path after the bucket name.
Consistency is important because when using many different operators if they
have similar templates it reduce the need to define specific variables.
For example in my case I must have two variables
a - gs://bucket_name/folder
b - folder (bucket_name already defined in the bucket property).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
