Re: Polaris persistence refactor POC

> > DAO is rather specific to JPA - so it is in practice specific to > relational databases. That's not true. DAO could be a pure Java interface without JPA as I mentioned above. Also my POC and design intentionally avoid any JPA related APIs. In fact, both implementers in the POC are non-JPA ones

Re: Polaris persistence refactor POC

Reminder: not all databases, especially NoSQL databases, are usable with JPA. DAO is rather specific to JPA - so it is in practice specific to relational databases. Further, DAOs, if implemented _like_ JPA, require a ton of logic to be implemented. I strongly prefer simple and database agnosti

Re: Polaris persistence refactor POC

Yes I know (remember I proposed to use DAO so I know what it is :)). If it’s clear for everyone, ok for DAO (just wanted to be sure that it’s clear for the implementers). So ok to have CatalogDao for operations and CatalogRecord for carriage data (for instance). Thanks Regards JB Le mar. 18 févr

Re: Polaris persistence refactor POC

Hi JB, DAO is often defined as an interface, used exactly for operations. Why do we need another name for that? See the examples here , it can be with JPA or non-JPA. In our case, it has to be compatible with non-JPA implementation. inter

Re: Polaris persistence refactor POC

Hi Yufei I agree with you. My only concern with CatalogDAO and NamespaceDAO in the PoC is that it could not be obvious for implementer. Being very abstract force us to clearly decouple service and persistence. One interface or multiple interfaces is totally fine for me (for the storage operations)

Re: Polaris persistence refactor POC

Hi JB, The Java objects you mentioned should either belong to the service/business layer or be unnecessary. Apart from the business entities in Polaris, each backend will maintain its own set of persistence-layer entities. The backend implementations (whether Postgres, FDB, or MongoDB) will be res

Re: Polaris persistence refactor POC

On 17.02.25 18:36, Yufei Gu wrote: Hi JB, The Java objects you mentioned should either belong to the service/business layer or be unnecessary. Apart from the business entities in Polaris, each backend will maintain its own set of persistence-layer entities. The backend implementations (whether

Re: Polaris persistence refactor POC

The purpose of code generation from the openapi spec is to ensure that api changes are made spec-first. There is no opportunity to forget to change the spec and there’s no worry about the code and the spec deviating (at least, the endpoints don’t differ). The generated code, if “not great” is extre

Re: Polaris persistence refactor POC

Everything you have to code manually requires explicit tests. Immutables is a very mature library that has exceptionally good test coverage that prevents a ton of accidental coding mistakes. We've been using it for many many years in multiple projects and literally never produced wrong code.

Re: Polaris persistence refactor POC

Generating code based on the spec is completely different than generating code based on the annotations introduced by Immutables. More importantly, we shouldn’t allow a discussion about pushing the project forward to devolve into a debate around some third-party library that’s not necessary to the

Re: Polaris persistence refactor POC

I prefer native Java constructs over third party libraries and compiler plugins, whenever possible. I’m a fan of Java records. Mike On Mon, Feb 17, 2025 at 6:55 AM Jean-Baptiste Onofré wrote: > Hi Robert, > > Yes, my intention about Java records (e.g. > https://openjdk.org/jeps/395) is to lever

Re: Polaris persistence refactor POC

Hi Robert, Yes, my intention about Java records (e.g. https://openjdk.org/jeps/395) is to leverage: - Devise an object-oriented construct that expresses a simple aggregation of values. - Focus on modeling immutable data rather than extensible behavior. - Automatically implement data-driven methods

Re: Polaris persistence refactor POC

Agree with JB, except I think "immutables" serves the need much better. Java records are okay, but do not ensure that all fields are immutable - especially collections. The other pro of immutables is that there are proper, descriptive builders - hence no need to constructors with a gazillion pa

Re: Polaris persistence refactor POC

Hi Yufei I left comments in the PR. Thanks for that. That's an interesting approach but slightly different to what I had in mind. My proposal was: 1. The DAOs are Java records clearly descriptive, without "storage operations". For instance, we can imagine: public record CatalogDao(String id, S

Polaris persistence refactor POC

Hi Folks: Here is a POC for persistence layer refactor. Please check it out and let me know what you think. Please note this is a POC, we still need a lot of effort to complete the refactor. PR: https://github.com/apache/polaris/pull/1011. Design doc: https://docs.google.com/document/d/1Vuhw5b9-6