Re: Flink compatibility with MINIO S3 Operator/Tenant

2021-05-17 Thread Robert Cullen
Youngwoo, I was trying to configure my flink/MINIO connector based on "s3_setup_with_provider" in the end-to-end test setup script; this requires that AWS_ACCES and AWS _SECRET keys be set as ENV variables but I think that needs to be set during the installation of MINIO, not flink. I'm just not

Re: Flink compatibility with MINIO S3 Operator/Tenant

2021-05-17 Thread 김영우
Hey Robert, I’m not sure why you need to set env var. but That’s a flink configuration, See https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/filesystems/s3/#configure-path-style-access Thanks, Youngwoo 2021년 5월 17일 (월) 오후 10:47, Robert Cullen 님이 작성: > Arvid, Is there

Re: Flink compatibility with MINIO S3 Operator/Tenant

2021-05-17 Thread Robert Cullen
Arvid, Is there a way to set environment variables in the flink-conf.yaml? Setting them on the CLI isn't working. On Sat, May 15, 2021 at 3:56 PM Arvid Heise wrote: > Hi Robert, > > we have an end-to-end-test with minio. You have to use > s3.path.style.access; I think the actual key depends on t

Re: Flink compatibility with MINIO S3 Operator/Tenant

2021-05-15 Thread Arvid Heise
Hi Robert, we have an end-to-end-test with minio. You have to use s3.path.style.access; I think the actual key depends on the used s3 connector, so just try out both. For reference, the actual test can be found here [2]. [1] https://github.com/apache/flink/blob/master/flink-end-to-end-tests/test-

Flink compatibility with MINIO S3 Operator/Tenant

2021-05-14 Thread Robert Cullen
I’m testing out my flink 1.13.0 cluster with the latest Minio RELEASE.2021-04-06T23-11-00Z but flink fails to connect to the bucket. (s3://flink/checkpoints) The 400 Bad Request error does not give any context for the precise problem. I'm wondering if this is a compatibility issue with AWS Signat