raphaelauv opened a new issue #18789:
URL: https://github.com/apache/airflow/issues/18789
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
5.1.0
### Apache Airflow version
2.1.2
### Operating System
GCP Container-Optimized OS
### Deployment
Composer
### Deployment details
_No response_
### What happened
When a variable have a non allow character like a `.` the secret manager is
going to fail even if the variable is inside the airflow database itelf (
registred in the the airflow variable )
### What you expected to happen
The secret manager should catch the error and say that it can't find it
```
[2021-10-07 06:22:09,330] {secret_manager_client.py:93} ERROR - Google Cloud
API Call Error (PermissionDenied): No access for Secret ID
example-variables-prefix-toto.tata.
```
and then look for the airflow variable
### How to reproduce
[secrets]
backend =
airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend
backend_kwargs = {"connections_prefix": "example-connections-prefix",
"variables_prefix": "example-variables-prefix"}
"{{ var.value.get('toto.tata') }}"
throw this error :
```
Traceback (most recent call last):
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1158, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1296, in _prepare_and_execute_task_with_callbacks
self.render_templates(context=context)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1796, in render_templates
self.task.render_template_fields(context)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py",
line 992, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context,
jinja_env, set())
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py",
line 1005, in _do_render_template_fields
rendered_content = self.render_template(content, context, jinja_env,
seen_oids)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py",
line 1042, in render_template
return jinja_env.from_string(content).render(**context)
File "/opt/python3.8/lib/python3.8/site-packages/jinja2/environment.py",
line 1090, in render
self.environment.handle_exception()
File "/opt/python3.8/lib/python3.8/site-packages/jinja2/environment.py",
line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/opt/python3.8/lib/python3.8/site-packages/jinja2/_compat.py", line
28, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
File "/opt/python3.8/lib/python3.8/site-packages/jinja2/sandbox.py", line
462, in call
return __context.call(__obj, *args, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1682, in get
return Variable.get(item, default_var=default_var)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/variable.py", line
135, in get
var_val = Variable.get_variable_from_secrets(key=key)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/variable.py", line
204, in get_variable_from_secrets
var_val = secrets_backend.get_variable(key=key)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/secrets/secret_manager.py",
line 154, in get_variable
return self._get_secret(self.variables_prefix, key)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/secrets/secret_manager.py",
line 178, in _get_secret
return self.client.get_secret(secret_id=secret_id,
project_id=self.project_id)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/_internal_client/secret_manager_client.py",
line 86, in get_secret
response = self.client.access_secret_version(name)
File
"/opt/python3.8/lib/python3.8/site-packages/google/cloud/secretmanager_v1/gapic/secret_manager_service_client.py",
line 967, in access_secret_version
return self._inner_api_calls["access_secret_version"](
File
"/opt/python3.8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py",
line 145, in __call__
return wrapped_func(*args, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/google/api_core/retry.py", line
286, in retry_wrapped_func
return retry_target(
File
"/opt/python3.8/lib/python3.8/site-packages/google/api_core/retry.py", line
189, in retry_target
return target()
File
"/opt/python3.8/lib/python3.8/site-packages/google/api_core/timeout.py", line
214, in func_with_timeout
return func(*args, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py",
line 69, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.InvalidArgument: 400 The provided Secret Version
ID
[projects/XXXXXXXXXXX/secrets/example-variables-prefix-toto.tata/versions/latest]
does not match the expected format [projects/*/secrets/*/versions/*]
```
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
