> Events are not meant for that. Events were meant for exactly that — if you review the original proposal you see discussion of exactly this kind of thing and relatively little discussion of auditing, which we now seem to regard as the main purpose for events. Much larger projects than Polaris — like React or Java itself — use the term “event listener” for exactly this purpose and I think the original design explicitly references Spark’s similar framework.
Rather than piecemeal re-purposing the framework across a dozen email threads think it might be better to explicitly submit a revised design and take an explicit stance as a project on what exactly events are meant for. On Tue, Nov 11, 2025 at 10:57 AM Alexandre Dutra <[email protected]> wrote: > Hi all, > > > Are you envisioning anything for being able to make dependencies between > event listeners? > > No, they would be completely independent. > > > In some listeners we have the ability to make events emission > synchronous. > > Indeed, this is the case today when using the AWS CloudWatch sink in > the so-called "synchronous" mode where it waits (blocks) for the > server acknowledgement. > > By doing so, this listener blocks the event loop thread serving the > REST request. **This is bad practice**. Such behavior could seriously > hurt the server responsiveness, not to mention that Quarkus may > interrupt the event loop thread that is being blocked by a slow > listener. > > My event bus proposal does not allow for this use case. All events > would be processed asynchronously, in fire-and-forget mode. They can, > however, be processed by a listener performing blocking operations > (I/O etc.) – in which case they will be executed in a dedicated thread > pool. > > > If you wish, for example, to prevent the creation of more than 1k tables > with some given prefix, you can do so using a listener. > > I don't think that's a good usage of an event listener. If we want to > do this, I would rather see a proper validation framework. Events are > not meant for that. > > In my opinion, events should never influence the course of the REST > request in any way. They are meant to notify interested parties, not > for interested parties to alter Polaris' behavior. > > Thanks, > Alex > > On Mon, Nov 10, 2025 at 10:55 PM Eric Maynard <[email protected]> > wrote: > > > > In fact, shouldn’t it be exclusively a listener’s decision on whether an > > event is handled in a blocking way or not? As was noted in a past thread > on > > events, much of the utility of the event framework comes from the ability > > to introduce custom logic and hooks into the normal operation of Polaris. > > > > If you wish, for example, to prevent the creation of more than 1k tables > > with some given prefix, you can do so using a listener. If the event > which > > might trigger that logic becomes non-blocking, you would no longer be > able > > to block/fail the create table request. > > > > I think maybe it’s the name “event”, but we seem to keep conflating these > > hooks with the iceberg events or auditing events when they are not > exactly > > the same thing. > > > > —EM > > > > On Mon, Nov 10, 2025 at 8:47 PM Adnan Hemani > > <[email protected]> wrote: > > > > > Hi Alex, > > > > > > Thanks for writing down the proposal for this! As I had previously > > > suggested this when implementing the Persistence of Polaris Events > > > <https://github.com/apache/polaris/pull/1844>, I am obviously very > much in > > > favor of doing this :) > > > > > > A few questions I have regarding your vision of how we should implement > > > this: > > > * Are you envisioning anything for being able to make dependencies > between > > > event listeners? Or are we taking a set direction that Event Listeners > > > should be independent of each other? > > > * In some listeners we have the ability to make events emission > synchronous > > > [example > > > < > > > > https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/events/jsonEventListener/aws/cloudwatch/AwsCloudWatchEventListener.java#L186 > > > >]. > > > How do we plan to support/advise (or not...) that with the introduction > > > of @Blocking annotations. > > > > > > Best, > > > Adnan Hemani > > > > > > On Mon, Nov 10, 2025 at 11:29 AM Yufei Gu <[email protected]> > wrote: > > > > > > > Thanks for the reply. It's overall a good idea to have async event > > > > listeners so that they are not blocking each other. > > > > > > > > One downside of the async ones is that event order isn't > deterministic. > > > > For example, event listeners of Spark need the order to understand > the > > > > execution semantics. I think Polaris is fine with that, given the ts > of > > > > each event is generated by Polaris. The downstream can still figure > out > > > the > > > > order. > > > > > > > > Thanks Pierre for sharing, I think any I/O-bound or potentially slow > > > > listener should be annotated with @Blocking. That ensures we keep the > > > event > > > > loop responsive and avoid impacting REST latency. > > > > > > > > Yufei > > > > > > > > > > > > On Mon, Nov 10, 2025 at 9:43 AM Alexandre Dutra <[email protected]> > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > Answering the questions above: > > > > > > > > > > > However, we can easily make sure that we use Quarkus's SmallRye > Fault > > > > > Tolerance > > > > > > > > > > Yes, that was my idea. It's not so much the bus itself that needs > to > > > > > be fault tolerant, but the receiving end, that is, the listeners. A > > > > > listener can fail for a variety of reasons (e.g. remote broker > > > > > unavailable), it would be nice to be able to backoff and retry > > > > > automatically. > > > > > > > > > > > Since the Vert.x event bus runs on event-loop threads [...] could > > > > > blocking or slow event listeners potentially stall REST requests > and > > > > impact > > > > > latency? > > > > > > > > > > What Pierre said: this could indeed happen, but it's possible to > > > > > annotate the receiving end with @Blocking, in which case, the > listener > > > > > will be invoked in a separate pool. > > > > > > > > > > > With asynchronous event listeners, is there a guarantee of > delivery > > > to > > > > > all listeners for a given event? > > > > > > > > > > If I understand the question correctly: with asynchronous > delivery, a > > > > > slow or failing listener wouldn't impact the delivery of the same > > > > > event to other listeners. > > > > > > > > > > Thanks, > > > > > Alex > > > > > > > > > > On Mon, Nov 10, 2025 at 10:12 AM Pierre Laporte < > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > > Thanks for the proposal, Alex. This sounds like a great > improvement. > > > > > > > > > > > > @Yufei As per Quarkus documentation, slow event listeners should > be > > > > > marked > > > > > > with @Blocking so that they are not run on the event loop > threads. > > > > > > -- > > > > > > > > > > > > Pierre > > > > > > > > > > > > > > > > > > On Sat, Nov 8, 2025 at 2:14 AM Michael Collado < > > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > > > With asynchronous event listeners, is there a guarantee of > delivery > > > > to > > > > > all > > > > > > > listeners for a given event? The downside of synchronous > listeners > > > is > > > > > that > > > > > > > everything is serial, but also if something fails, processing > > > stops. > > > > > This > > > > > > > feels important for auditing purposes, though less important > for > > > > other > > > > > > > cases. > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > On Fri, Nov 7, 2025 at 2:28 PM Yufei Gu <[email protected]> > > > > wrote: > > > > > > > > > > > > > > > Thanks, Alex and Adam. One concern I have is about the shared > > > > runtime > > > > > > > > thread pool. > > > > > > > > Since the Vert.x event bus runs on event-loop threads that > are > > > also > > > > > used > > > > > > > by > > > > > > > > Quarkus’ reactive REST endpoints, could blocking or slow > event > > > > > listeners > > > > > > > > potentially stall REST requests and impact latency? > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Nov 7, 2025 at 11:25 AM Adam Christian < > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > I think that this would be a great enhancement. Thanks for > > > > > proposing > > > > > > > it! > > > > > > > > > > > > > > > > > > The only concern I would have is around fault-tolerance. > From > > > > what > > > > > I > > > > > > > can > > > > > > > > > tell, from the Quarkus documentation, the Quarkus event bus > > > uses > > > > > Vert.x > > > > > > > > > EventBus which does not guarantee message delivery if > failure > > > of > > > > > part > > > > > > > of > > > > > > > > > the event bus occurs [1]. However, we can easily make sure > that > > > > we > > > > > use > > > > > > > > > Quarkus's SmallRye Fault Tolerance [2]. Is my rough > > > understanding > > > > > > > inline > > > > > > > > > with your proposal? > > > > > > > > > > > > > > > > > > Go community, > > > > > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > https://vertx.io/docs/apidocs/io/vertx/core/eventbus/EventBus.html > > > > > > > > > [2]: https://quarkus.io/guides/smallrye-fault-tolerance > > > > > > > > > > > > > > > > > > On Fri, Nov 7, 2025 at 11:49 AM Alexandre Dutra < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > I'd like to propose an enhancement to our existing events > > > > > feature: > > > > > > > the > > > > > > > > > > ability to support multiple listeners. > > > > > > > > > > > > > > > > > > > > Currently, only a single listener can be active at a > time, > > > > which > > > > > is > > > > > > > > > > quite limiting. For example, we might need to persist > events > > > > for > > > > > > > audit > > > > > > > > > > purposes and simultaneously send them to a message queue > for > > > > > > > > > > optimization. With the current setup, this isn't easily > > > > > achievable. > > > > > > > > > > > > > > > > > > > > While a composite listener could be created, it feels > like a > > > > less > > > > > > > > > > elegant solution, and the delivery would be strictly > serial, > > > > > > > > > > processing one listener after another. > > > > > > > > > > > > > > > > > > > > My suggestion is to leverage Quarkus internal event bus > [1]: > > > > > > > > > > > > > > > > > > > > 1) There will be one central event emitter responsible > for > > > > > publishing > > > > > > > > > > events to the bus. > > > > > > > > > > > > > > > > > > > > 2) We will have zero to N listeners, each independently > > > > watching > > > > > the > > > > > > > > > > event bus for relevant events. They will be discovered by > > > CDI. > > > > > > > > > > > > > > > > > > > > 3) We could apply filters to each listener, e.g. > listener A > > > > > listens > > > > > > > > > > for event types X and Y, listener B only listens to event > > > type > > > > Y. > > > > > > > > > > > > > > > > > > > > 4) This approach would ensure fully asynchronous > delivery of > > > > > events > > > > > > > to > > > > > > > > > > all interested listeners. > > > > > > > > > > > > > > > > > > > > 5) Fault-tolerance could also be easily implemented > (event > > > > > delivery > > > > > > > > > > retries, timeouts, etc.). > > > > > > > > > > > > > > > > > > > > What do you all think? > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > [1]: https://quarkus.io/guides/reactive-event-bus > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
