Adding that there's been consensus in the meeting to start with a pure Java interface and go from there.

I'm not sure that the statement "Ordering guarantees are **only** possible ... event creation time" (emphasis mine) is correct.

During the meeting I mentioned that I strongly believe that it is not a good idea to let users (as Apache Polaris we do NOT have customers) figure out constraints and limitations and issues on their own.

Neither the Iceberg proposal nor PR 1844 are suitable for the auditing use case, because auditing is a _very_ different beast and implies way more requirements than "just store some data w/o any consistency guarantees".

There are still concerns mentioned in the Iceberg events proposal, with a huge impact to this effort. So I strongly believe that both efforts are very tightly coupled and not orthogonal efforts that could be handled independently. For example, I do not see how the "metadata payload size concern" is mitigated in PR 1844. We had the same discussion around "caching Iceberg metadata in Polaris".

During the meeting some people raised concerns about the "buffering" that is strictly speaking not a necessity, but also introduced in 1844. That introduces additional consistency issues, additional risk of losing events and additional ordering issues. That is a very different problem than "just storing some data".


On 17.06.25 21:16, Adnan Hemani wrote:
Hi everyone,

In lieu of a recording of today’s Community Sync on Events, I am posting some 
notes regarding what was discussed:
What is the relationship between Iceberg Events API and Polaris Events, which 
are proposed in https://github.com/apache/polaris/pull/1844?
Persisting Polaris events are a pre-requisite of the Iceberg Events API - but 
are not strictly tied to this. Users could find value in being able to persist 
the Polaris Events without using the Iceberg Events API.
What Query Patterns are we expecting?
Going based on the assumption that the Iceberg Events API will be a primary 
consumer of the Polaris Events and that it is almost finalized. The proposed 
data schema for events is designed to work efficiently with the current state 
of the Iceberg Events API.
What’s the Intended Use-Case?
This will go out in a different email later today under the original proposal 
thread to ensure all context is in the same email thread.
If auditing is a potential use-case, then what guarantees are we able to 
provide?
Ordering guarantees are only possible in that the event creation time is listed 
with the Polaris Event. When querying Polaris Events from the database, we can 
always sort events based on this timestamp.
Durability guarantees can be found in some implementations - but this is up to 
the customer to choose which implementation they choose and how they’d like to 
configure that implementation. All of these configurations are present in the 
PR as it stands today.
A potential Kafka implementation may help with these concerns - but lacks an 
end-to-end customer experience within Polaris and may be pushing the concerns 
forward to Kafka rather than solving them. Unsure how this may work with 
Iceberg Events API in the future.
Can the PR be broken up further?
Yes, it is possible - but unclear what parts are not necessary at this time. 
Community to review and make suggestions on the PR.

Next Steps/Action Items:
Community: to review PR as it stands and provide high-level 
recommendations/suggestions
Adnan Hemani: Send email regarding intended use cases.
Adnan Hemani: To respond to all reviews on PRs.

Please do respond to this email with anything I may have missed out on! Thanks 
to everyone who was able to make it to this morning’s sync and for everyone’s 
contributions :)

Best,
Adnan Hemani


On Jun 13, 2025, at 4:43 PM, Adnan Hemani <adnan.hem...@snowflake.com> wrote:

Hi all,

As we were not able to discuss at the previous community sync, I’m setting a 
quick sync early next week to discuss Events in Persistence (PR: 
https://github.com/apache/polaris/pull/1844). Everyone is welcome to join and 
discuss on next steps here. Thanks!

Best,
ADNAN HEMANI

Polaris Community Sync on Events
Tuesday, June 17 · 9:00 – 9:30am
Time zone: America/Los_Angeles
Google Meet joining info
Video call link: https://meet.google.com/ear-kiij-sur
Or dial: ‪(US) +1 402-410-2280‬ PIN: ‪350 919 847‬#
More phone numbers: https://tel.meet/ear-kiij-sur?pin=5036846369686

Reply via email to