Re: [DISCUSS] Storing Table Metadata in the Metastore

Hi Thanks Eric. We quickly discussed together about metadata json in London, so it’s aligned. If the caching makes sense, I wonder for the persistence layer. Maybe we should clearly state who is responsible of what. Overall it looks a great idea. Regards JB Le ven. 23 mai 2025 à 23:52, Yufei G

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

Hi Dmitri Can you please verify again and maybe change your vote if it's good ? Thanks ! Regards JB On Sat, May 24, 2025 at 1:31 AM Dmitri Bourlatchkov wrote: > > -1 (binding) > > Indeed, many (all?) .asc files appear to be full signed archives (large > files), not detached signatures. > > For

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

It's now fixed on dist.apache.org (the signatures are good on the Staging Maven repo). Can you please verify? Thanks, Regards JB On Sat, May 24, 2025 at 7:37 AM Jean-Baptiste Onofré wrote: > > It seems the upload failed. Let me check. > > Le sam. 24 mai 2025 à 00:26, Russell Spitzer a > écrit

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

It seems the upload failed. Let me check. Le sam. 24 mai 2025 à 00:26, Russell Spitzer a écrit : > +1 (Binding) > > Checked all the normal things > 1. Build / Test > 2. Checksums > 3. Smoke tested Server and Admin jars > 4. GPG Signatures (Issues below) > > Only did a quick pass on Helm Licenses

Re: [DISCUSS] Storing Table Metadata in the Metastore

An older version of the implementation took this approach, but some months ago Dennis left comments suggesting that we use an internal property and the PR was updated. Replies to your specific concerns inline: > Regarding the specific proposal of using a internal property of entities for holding t

Re: Proposal for Implementation of the Proposed Iceberg REST Spec - Events API

> filtering all the reads to find the valuable changes is a > headache I would guess most service owners would rather avoid. The Iceberg Events API would let users specify filters, which I think would help filter out read events quite easily (with a single filter predicate). But once we start pu

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

Yeah, the file size isn't correct except the source code tarball -rw-r--r--@ 1 ygu staff 179B May 23 05:57 polaris-quarkus-server-0.10.0-beta-incubating.zip.sha512 -rw-r--r--@ 1 ygu staff 178M May 23 05:57 polaris-quarkus-server-0.10.0-beta-incubating.zip.asc -rw-r--r--@ 1 ygu staff 178M

Re: [DISCUSS] Storing Table Metadata in the Metastore

Thanks for starting this discussion, Eric! Overall, I think the idea of storing (some) table metadata in the Polaris database is very relevant and a sound approach to improving query performance on the engine side. Regarding the specific proposal of using a internal property of entities for holdi

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

-1 (binding) Indeed, many (all?) .asc files appear to be full signed archives (large files), not detached signatures. For example: $ file polaris-quarkus-server-0.10.0-beta-incubating.tgz.asc polaris-quarkus-server-0.10.0-beta-incubating.tgz.asc: PGP signed message $ du -sm polaris-quarku

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

+1 (Binding) Checked all the normal things 1. Build / Test 2. Checksums 3. Smoke tested Server and Admin jars 4. GPG Signatures (Issues below) Only did a quick pass on Helm Licenses/Notice and Friends but all look good to me now. I do have one question because I seem to be having issues (if this

Re: [DISCUSS] Remove realm_id metric tag

Hey folks, I’m not sold on dropping realm_id. Why it matters 1. It's pretty useful for first-line triage in multi-tenant clusters. When a latency spike or error burst hits the dashboards, the first SRE question is “Which realm(s)?” Having the tag lets us slice the time-series instant

Re: [DISCUSS] Storing Table Metadata in the Metastore

Thanks for doing this, Eric! It will boost performance a lot for tables with reasonable size metadata.json files. We also automatically get an in-memory cache since the Polaris entity is cached by default. Agreed to defer any separated caching mechanism so that we don't have to care about consisten

[VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc4)

Hi everyone I propose that we release the following RC (RC4) as the official Apache Polaris 0.10.0-beta-incubating release. Compared to RC3, RC4 fixes the LICENSE/NOTICE in the Heml chart. * This corresponds to the tag: apache-polaris-0.10.0-beta-incubating-rc4 * https://github.com/apache/polaris

Re: GitHub pull-requests

After seeing a few Conventional Commits messages, I personally do not find them very useful. Granted, the intention of clearly describing the change is good. However, the "feat(abc)" prefixes do not add much value IMHO, but require extra cognitive effort to parse. I still have to read the whole me

AWS S3 KMS support in Polaris

Hi folks, This is regarding the addition of KMS support for AWS S3 in Polaris. Several approaches were proposed in the PR , and based on our last community sync, we aligned on the following: - Using an IAM policy with *Resource: '*'* at the catal

[DISCUSS] Storing Table Metadata in the Metastore

Hi all, Some time ago I opened this PR which proposes to store/cache TableMetadata in the Polaris metastore, avoiding a trip to object storage in many cases. Based on this recent comment

Re: [DISCUSS] Remove realm_id metric tag

Hi, For reference the PR is here: https://github.com/apache/polaris/pull/1662 Alex On Fri, May 23, 2025 at 2:42 PM Robert Stupp wrote: > > +1 on adding a config and default to false - and later entirely remove > the "issue". > > Sounds like good plan to me! > > On 23.05.25 13:52, Alex Dutra wro

Re: [DISCUSS] Remove realm_id metric tag

+1 on adding a config and default to false - and later entirely remove the "issue". Sounds like good plan to me! On 23.05.25 13:52, Alex Dutra wrote: Hi Mike, That sounds like a plan! And yes, I agree with the slow approach. I will open a PR shortly with the new enablement flag. Alex On Fri

Re: [DISCUSS] Remove realm_id metric tag

Hi Mike, That sounds like a plan! And yes, I agree with the slow approach. I will open a PR shortly with the new enablement flag. Alex On Fri, May 23, 2025 at 2:42 AM Michael Collado wrote: > > My immediate reaction is that a configuration flag would be good enough for > us. Our setup prevents