rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793518186
Oh, I didn't notice that the main CI isn't working either. The condition looks fine. Maybe the problem is that we are not setting the environment variables here: https://github.com/apache/flink/blob/master/tools/azure-pipelines/build-apache-repo.yml#L51 The build-apache-repo.yml definition is used for triggering the PR builds. Maybe azure is putting the variable name if the variable is not defined (hence the test is executed in the PR Ci build because the variable is replaced by the variable name, and the test is executed on my personal ci, because there the correct variable is set). > Why we need to pass the value of $IT_CASE_GLUE_SCHEMA_ACCESS_KEY to $SECRET_GLUE_SCHEMA_ACCESS_KEY? These are the secret variables I have defined in my personal CI:  The definitions in `azure-pipelines.yml` get the defined variables (for example `variables.IT_CASE_GLUE_SCHEMA_ACCESS_KEY` and store them in an internal, temporary variable (for example `SECRET_GLUE_SCHEMA_ACCESS_KEY`), then in `jobs-template.yml` we make the environment variable available again, under the original name: `IT_CASE_GLUE_SCHEMA_ACCESS_KEY: $(SECRET_GLUE_SCHEMA_ACCESS_KEY)`. Why this weird transformation from "defined variable" to "internal variable" back to "environment variable"? I'm not 100% sure anymore, but I remember that I've struggled a lot setting up the secrets. What I know is that you need to manually declare the secret variables on the respective Azure tasks like this: ```yaml env: IT_CASE_S3_BUCKET: $(SECRET_S3_BUCKET) IT_CASE_S3_ACCESS_KEY: $(SECRET_S3_ACCESS_KEY) IT_CASE_S3_SECRET_KEY: $(SECRET_S3_SECRET_KEY) IT_CASE_GLUE_SCHEMA_ACCESS_KEY: $(SECRET_GLUE_SCHEMA_ACCESS_KEY) IT_CASE_GLUE_SCHEMA_SECRET_KEY: $(SECRET_GLUE_SCHEMA_SECRET_KEY) ``` ... otherwise they won't work. Afaik using the same name for the variable also didn't seem to have worked. What we know is that this approach works well for the S3 tests. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
