Re: Iceberg 0.12.0 Release Plan

Hi Dan, By the time I saw your email I had already gotten approval from Jack and committed the patch. I created a new PR to track any additional changes we want to make before announcing the release: https://github.com/apache/iceberg/pull/2986 Thanks. - Carl On Mon, Aug 16, 2021 at 8:56 PM Dan

Re: Iceberg 0.12.0 Release Plan

I think there might be a couple more bug fixes that we want to call out. I'm still reviewing, but will try to add any additions tomorrow. Thanks, Dan On Mon, Aug 16, 2021, 8:23 PM Carl Steinbach wrote: > Update: still waiting for a +1 on the release notes > https://github.com/apache/iceberg/p

Re: Iceberg 0.12.0 Release Plan

Update: still waiting for a +1 on the release notes https://github.com/apache/iceberg/pull/2973 - Carl On Sat, Aug 14, 2021 at 6:25 PM Carl Steinbach wrote: > The 0.12.0 release has been approved, but before we can announce it we > need to add the release notes. > > Please help us get over this

Re: create iceberg on minio s3 got "The AWS Access Key Id you provided does not exist in our records."

Talked with Lian on Slack, the user is using a hadoop 3.2.1 + hive (postgres) + spark + minio docker installation. There might be some S3A related dependencies missing on the Hive server side based on the stack trace. Let's see if that fixes the issue. -Jack On Mon, Aug 16, 2021 at 7:32 PM Lian Ji

Re: create iceberg on minio s3 got "The AWS Access Key Id you provided does not exist in our records."

This is my full script launching spark-shell: # add Iceberg dependency export AWS_REGION=us-east-1 export AWS_ACCESS_KEY_ID=minio export AWS_SECRET_ACCESS_KEY=minio123 ICEBERG_VERSION=0.11.1 DEPENDENCIES="org.apache.iceberg:iceberg-spark3-runtime:$ICEBERG_VERSION,org.apache.iceberg:iceberg-hive-r

Re: create iceberg on minio s3 got "The AWS Access Key Id you provided does not exist in our records."

Have you included the hadoop-aws jar? https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws -Jack On Mon, Aug 16, 2021 at 7:09 PM Lian Jiang wrote: > Jack, > > You are right. S3FileIO will not work on minio since minio does not > support ACL: https://docs.min.io/docs/minio-server-limit

Re: create iceberg on minio s3 got "The AWS Access Key Id you provided does not exist in our records."

Jack, You are right. S3FileIO will not work on minio since minio does not support ACL: https://docs.min.io/docs/minio-server-limits-per-tenant.html To use iceberg, minio + s3a, I used below script to launch spark-shell: /spark/bin/spark-shell --packages $DEPENDENCIES \ --conf spark.sql.catal

Re: Identify watermark in the iceberg table properties

+ Sundaram, as he may have some input. regards, Anjali. On Fri, Aug 13, 2021 at 2:41 AM 1 wrote: > Hi,all: > > I need to embed the iceberg table, which is regarded as real-time > table, into our workflow. That is to say, Flink writes data into Iceberg > table in real-time, I need something to

Re: [CWS] Re: Subject: [VOTE] Release Apache Iceberg 0.12.0 RC3

Ok, after the vote, but I did: * verified tag is same as the tar ball * verified checksums and signatures * built and ran the tests My one complaint is that I get test failures that look like they are timezone related. ORC and Parquet tests failing with timestamps 7 or 8 hours off. .. Owen On Su

Re: create iceberg on minio s3 got "The AWS Access Key Id you provided does not exist in our records."

Sorry for the late reply, I thought I replied on Friday but the email did not send successfully. As Daniel said, you don't need to setup S3A if you are using S3FileIO. Th S3FileIO by default reads the default credentials chain to check credential setups one by one: https://docs.aws.amazon.com/sdk

Re: Iceberg python library sync

Hi everyone, I have sent the meeting invite using the replied emails in the threads. Here is the meeting agenda: https://docs.google.com/document/d/1oMKodaZJrOJjPfc8PDVAoTdl02eGQKHlhwuggiw7s9U/edit?usp=sharing . Similar to iceberg community sync, please join the iceberg-python-sync