Re: EntityCache race conditions and JCStress in Polaris

[Robert Stupp]

Yes, jcstress is GPL2+CE. However, jcstress would not be distributed in 
any way and definitely not be included in "production code". It's a 
test-only dependency, which should be fine.


On 26.03.25 14:08, Pierre Laporte wrote:

Hello all

In https://github.com/apache/polaris/issues/761, there is a discussion on
whether the EntityCache is thread-safe, prone to race conditions,
inconsistencies, etc...

To determine whether these issues are real, I worked on JCStress [1]
tests.  JCStress is a testing framework developed by Oracle that helps
verify correct behavior of concurrent Java code.  I would like to discuss
the possibility to add such tests to the Polaris project.

Here is an example of tests:
https://github.com/pingtimeout/polaris/blob/jcstress/jcstress-tests/src/jcstress/java/org/apache/polaris/core/persistence/cache/EntityCacheCoherenceTest.java#L34-L99.
And here is an example of the insights we can get out of them:
https://docs.google.com/document/d/1ZF5WGiSOoqg9JpSvFfhuPOK0QrqeY431/edit?tab=t.0#bookmark=id.op4496ui0vzi
.

These tests are similar to unit tests, in that there should be no
dependency on external components, they should be fast so that millions of
invocations can be performed in a reasonable timeframe, and they should be
fine-grained.  To that extent, it would make sense to colocate them with
the codebase, in the main repository.

But JCStress is released under the GPL-v2, which seems incompatible with
the ASLv2 [2].  Does anybody have any experience with such integrations?
We would make use of the JCStress framework, as opposed to extending it.
But couldn't this be considered derivative work?  In which case, should the
concurrency tests be developed as a separate project (and/or in a separate
repository) so that they themselves are released under the GPL-v2 ?

Thoughts?

[1] https://github.com/openjdk/jcstress
[2] https://www.apache.org/licenses/GPL-compatibility.html

--

Pierre


--
Robert Stupp
@snazy

EntityCache race conditions and JCStress in Polaris

[Pierre Laporte]

Hello all

In https://github.com/apache/polaris/issues/761, there is a discussion on
whether the EntityCache is thread-safe, prone to race conditions,
inconsistencies, etc...

To determine whether these issues are real, I worked on JCStress [1]
tests.  JCStress is a testing framework developed by Oracle that helps
verify correct behavior of concurrent Java code.  I would like to discuss
the possibility to add such tests to the Polaris project.

Here is an example of tests:
https://github.com/pingtimeout/polaris/blob/jcstress/jcstress-tests/src/jcstress/java/org/apache/polaris/core/persistence/cache/EntityCacheCoherenceTest.java#L34-L99.
And here is an example of the insights we can get out of them:
https://docs.google.com/document/d/1ZF5WGiSOoqg9JpSvFfhuPOK0QrqeY431/edit?tab=t.0#bookmark=id.op4496ui0vzi
.

These tests are similar to unit tests, in that there should be no
dependency on external components, they should be fast so that millions of
invocations can be performed in a reasonable timeframe, and they should be
fine-grained.  To that extent, it would make sense to colocate them with
the codebase, in the main repository.

But JCStress is released under the GPL-v2, which seems incompatible with
the ASLv2 [2].  Does anybody have any experience with such integrations?
We would make use of the JCStress framework, as opposed to extending it.
But couldn't this be considered derivative work?  In which case, should the
concurrency tests be developed as a separate project (and/or in a separate
repository) so that they themselves are released under the GPL-v2 ?

Thoughts?

[1] https://github.com/openjdk/jcstress
[2] https://www.apache.org/licenses/GPL-compatibility.html

--

Pierre

Re: Clarification on Unique Entity Identifier in Polaris

[Eric Maynard]

I believe that entity ID by itself is meant to be a unique identifier.

On Wed, Mar 26, 2025 at 12:27 PM Honah J.  wrote:

> Hi folks,
>
> I have a question about what constitutes a unique identifier for an entity
> in Polaris in the future.
>
> Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> following a recent refactoring (in a PR
> ). On the other hand,
> `lookupEntities` currently takes a list of PolarisEntityId, which only
> includes catalogId and entityId, and is assumed to represent unique
> entities.
>
> I understand the refactoring is still in progress, but I’d like to clarify
> the intended direction:
> Will the unique identifier eventually include typeCode as well? Or do we
> still consider catalogId + entityId sufficient for uniqueness, with
> typeCode used solely for lookup optimization?
>
> Best regards,
> Honah (Jonas)
>

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Michael Collado]

Wow! Awesome news. Congrats folks!

Mike

On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer 
wrote:

> Hi y'all!
>
> I'm excited to let the Polaris Community know that the PPMC has added three
> new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all now
> members of the Polaris PPMC.
>
> Please join me in welcoming them,
> Russ
>

Re: Clarification on Unique Entity Identifier in Polaris

[Dennis Huo]

Correct, the entityId must be unique across all types.

On Wed, Mar 26, 2025 at 12:30 PM Eric Maynard 
wrote:

> I believe that entity ID by itself is meant to be a unique identifier.
>
> On Wed, Mar 26, 2025 at 12:27 PM Honah J.  wrote:
>
> > Hi folks,
> >
> > I have a question about what constitutes a unique identifier for an
> entity
> > in Polaris in the future.
> >
> > Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> > following a recent refactoring (in a PR
> > ). On the other hand,
> > `lookupEntities` currently takes a list of PolarisEntityId, which only
> > includes catalogId and entityId, and is assumed to represent unique
> > entities.
> >
> > I understand the refactoring is still in progress, but I’d like to
> clarify
> > the intended direction:
> > Will the unique identifier eventually include typeCode as well? Or do we
> > still consider catalogId + entityId sufficient for uniqueness, with
> > typeCode used solely for lookup optimization?
> >
> > Best regards,
> > Honah (Jonas)
> >
>

Re: [VOTE] REST API changes for Catalog Federation - addition of ConnnectionConfigInto to ExternalCatalog

[Dennis Huo]

Thanks everyone for voting and providing input!

The vote has now been open for about two days, and since all stakeholders
who have been active on this topic have already cast votes and no new
comments have been raised in the associated discussion thread on the
mailing list or in the PR since last week, I'm going to go ahead and
proceed.

Results:

+1: 5 (binding), 4 (non-binding)
+0: 0 (binding), 0 (non-binding)
-1: 0 (binding), 0 (non-binding)

On Tue, Mar 25, 2025 at 9:30 AM yun zou  wrote:

> +1
>
> Best,
> Yun
>
> On Tue, Mar 25, 2025 at 9:22 AM Honah J.  wrote:
>
> > +1 (non-binding)
> >
> > Best,
> > Honah (Jonas)
> >
> > On Tue, Mar 25, 2025 at 8:22 AM Yufei Gu  wrote:
> >
> > > +1
> > >
> > > Yufei
> > >
> > >
> > > On Tue, Mar 25, 2025 at 8:03 AM Prashant Singh
> > >  wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > Best,
> > > > Prashant
> > > >
> > > > On Tue, Mar 25, 2025 at 7:01 AM Dmitri Bourlatchkov <
> di...@apache.org>
> > > > wrote:
> > > >
> > > > > +1 LGTM.
> > > > >
> > > > > Cheers,
> > > > > Dmitri.
> > > > >
> > > > > On Tue, Mar 25, 2025 at 12:30 AM Dennis Huo 
> > wrote:
> > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > We've had some productive discussion in various places on the
> > mailing
> > > > > list,
> > > > > > in the github PR, and in the live community sync now about the
> > > initial
> > > > > > minimal updates to the Polaris REST API (under the "management
> > API")
> > > > for
> > > > > > adding a ConnectionConfigInfo field to ExternalCatalogs to
> > represent
> > > > > > external federation:
> > > > > >
> > > > > > https://lists.apache.org/thread/szq8nzhblgwz0w71g5wh8kkcwbm7nhkn
> > > > > > https://github.com/apache/polaris/pull/1026
> > > > > >
> > > > > > To recap, the the ConnectionConfigInfo is designed to be
> extensible
> > > so
> > > > > that
> > > > > > we can eventually define a variety of federation sources, but
> > > currently
> > > > > > just defines a minimal Iceberg REST federation so that we have a
> > good
> > > > > > starting point to start implementing the internals of federation
> > > > without
> > > > > > biting off too much.
> > > > > >
> > > > > > We've also agreed the current proposed API is marked as
> > > "experimental"
> > > > > and
> > > > > > "subject to change", so this step also doesn't entrench the API
> > > > > definition
> > > > > > in stone yet.
> > > > > >
> > > > > > This thread commences our formal vote for addition of this
> initial
> > > API
> > > > > and
> > > > > > merging the PR listed above.
> > > > > >
> > > > > > [ ] +1 Merge  https://github.com/apache/polaris/pull/1026
> > > > > > [ ] +0
> > > > > > [ ] -1 Do not merge because...
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Ajantha Bhat]

Congratulations guys. Well deserved.

On Thu, Mar 27, 2025 at 6:47 AM Prashant Singh
 wrote:

> Congratulations folks, well deserved !
>
> Best,
> Prashant Singh
>
> On Wed, Mar 26, 2025 at 5:02 PM yun zou 
> wrote:
>
> > Congratulations!
> >
> > Best Regards,
> > Yun
> >
> > On Wed, Mar 26, 2025 at 4:28 PM Michael Collado 
> > wrote:
> >
> > > Wow! Awesome news. Congrats folks!
> > >
> > > Mike
> > >
> > > On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer <
> > russell.spit...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi y'all!
> > > >
> > > > I'm excited to let the Polaris Community know that the PPMC has added
> > > three
> > > > new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all
> now
> > > > members of the Polaris PPMC.
> > > >
> > > > Please join me in welcoming them,
> > > > Russ
> > > >
> > >
> >
>

Re: Polaris benchmarks proposal

[Russell Spitzer]

I think having a tool like this is a great idea. Would we be able to host
the results over time as well? Like an official build run that triggers on
a daily basis?

On Wed, Mar 19, 2025 at 10:07 AM Pierre Laporte 
wrote:

> Hi
>
> I have been working on a set of benchmarks for Polaris [1] and would like
> to contribute them to the project.  I have opened a PR with the code, in
> case anybody is interested.
>
> The benchmarks are written using Gatling.  The core design decision
> consists in building a procedural dataset, loading it to Polaris, and then
> reusing it for all subsequent benchmarks.  The procedural aspect makes it
> possible to deterministically regenerate the same dataset at runtime over
> and over, without having to store the actual data.
>
> With this, it is trivial to generate large number of Polaris entities.
> Typically, I used this to benchmark the NoSQL persistence implementation
> with 65k namespaces, 65k tables and 65k views.  Increasing that to millions
> would only require a one parameter change.  Additionally, the dataset
> currently includes property updates for namespaces, tables and views, which
> can quickly create hundreds of manifests.  This can be useful for table
> maintenance testing.
>
> Three benchmarks have been created so far:
>
>- A benchmark that populates an empty Polaris server with a dataset that
>have predefined attributes
>- A benchmark that issues only read queries over that dataset
>- A benchmark that issues read and write queries (entity updates) over
>that dataset, with a configurable read/write ratio
>
> The benchmarks/README.md contains instructions to build and run the
> benchmarks, as well as to describe the kind of dataset that should be
> generated.
>
> As with every Gatling benchmark, an HTML page is generated with interactive
> charts showing query performance over time, response time percentiles,
> etc...
>
> I would love to head your feedback on it.
>
> Pierre
>
> [1] https://github.com/apache/polaris/pull/1208
> --
>
> Pierre Laporte
> @pingtimeout 
> pie...@pingtimeout.fr
> http://www.pingtimeout.fr/
>

RE: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Kyle Bader]

congrats!

—
Kyle Bader
STSM, Principal Portfolio Architect
Ceph @ IBM

“two is one and one is none”
- a byzantine general

From: yun zou 
Sent: Wednesday, March 26, 2025 5:02:39 PM
To: dev@polaris.apache.org 
Subject: [EXTERNAL] Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and 
Yufei Gu to the Apache Polaris PPMC

Congratulations!

Best Regards,
Yun

On Wed, Mar 26, 2025 at 4:28 PM Michael Collado 
wrote:

> Wow! Awesome news. Congrats folks!
>
> Mike
>
> On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer  >
> wrote:
>
> > Hi y'all!
> >
> > I'm excited to let the Polaris Community know that the PPMC has added
> three
> > new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all now
> > members of the Polaris PPMC.
> >
> > Please join me in welcoming them,
> > Russ
> >
>

Clarification on Unique Entity Identifier in Polaris

[Honah J.]

Hi folks,

I have a question about what constitutes a unique identifier for an entity
in Polaris in the future.

Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
following a recent refactoring (in a PR
). On the other hand,
`lookupEntities` currently takes a list of PolarisEntityId, which only
includes catalogId and entityId, and is assumed to represent unique
entities.

I understand the refactoring is still in progress, but I’d like to clarify
the intended direction:
Will the unique identifier eventually include typeCode as well? Or do we
still consider catalogId + entityId sufficient for uniqueness, with
typeCode used solely for lookup optimization?

Best regards,
Honah (Jonas)

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Prashant Singh]

Congratulations folks, well deserved !

Best,
Prashant Singh

On Wed, Mar 26, 2025 at 5:02 PM yun zou  wrote:

> Congratulations!
>
> Best Regards,
> Yun
>
> On Wed, Mar 26, 2025 at 4:28 PM Michael Collado 
> wrote:
>
> > Wow! Awesome news. Congrats folks!
> >
> > Mike
> >
> > On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer <
> russell.spit...@gmail.com
> > >
> > wrote:
> >
> > > Hi y'all!
> > >
> > > I'm excited to let the Polaris Community know that the PPMC has added
> > three
> > > new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all now
> > > members of the Polaris PPMC.
> > >
> > > Please join me in welcoming them,
> > > Russ
> > >
> >
>

Re: Clarification on Unique Entity Identifier in Polaris

[Yufei Gu]

I'm OK with passing more through the interface. It's helpful in some use
cases. That said, we should clearly distinguish which parameters are
required and which are optional. Also, it's important to keep things
consistent across methods, such as between lookupEntity and lookupEntities.

>From what I understand, the entity ID is required, while the other
parameters can be optional. Can we enforce these constraints consistently
across both methods?

Yufei


On Wed, Mar 26, 2025 at 3:58 PM Eric Maynard 
wrote:

> This is more of an interface design question... I tend to think passing
> more information down into the persistence layer is a good thing. So if
> providing catalog ID, entity type, maybe even subtype, etc. can help the
> persistence layer, we shouldn't be afraid to pass it down.
>
> One other thing worth calling out -- there is an implied uniqueness
> constraint on (parent ID, entity type, entity name) as well as on entity
> ID.
>
> On Wed, Mar 26, 2025 at 2:07 PM Honah J.  wrote:
>
> > Thanks for the clarification! Just to confirm — even though the entityId
> is
> > globally unique, we’ll still require catalogId + entityId at the
> > persistence API level to perform an entity lookup now and in the future,
> > correct?
> >
> > On Wed, Mar 26, 2025 at 1:45 PM Dennis Huo  wrote:
> >
> > > Correct, the entityId must be unique across all types.
> > >
> > > On Wed, Mar 26, 2025 at 12:30 PM Eric Maynard <
> eric.w.mayn...@gmail.com>
> > > wrote:
> > >
> > > > I believe that entity ID by itself is meant to be a unique
> identifier.
> > > >
> > > > On Wed, Mar 26, 2025 at 12:27 PM Honah J.  wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > I have a question about what constitutes a unique identifier for an
> > > > entity
> > > > > in Polaris in the future.
> > > > >
> > > > > Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> > > > > following a recent refactoring (in a PR
> > > > > ). On the other hand,
> > > > > `lookupEntities` currently takes a list of PolarisEntityId, which
> > only
> > > > > includes catalogId and entityId, and is assumed to represent unique
> > > > > entities.
> > > > >
> > > > > I understand the refactoring is still in progress, but I’d like to
> > > > clarify
> > > > > the intended direction:
> > > > > Will the unique identifier eventually include typeCode as well? Or
> do
> > > we
> > > > > still consider catalogId + entityId sufficient for uniqueness, with
> > > > > typeCode used solely for lookup optimization?
> > > > >
> > > > > Best regards,
> > > > > Honah (Jonas)
> > > > >
> > > >
> > >
> >
>

[ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Russell Spitzer]

Hi y'all!

I'm excited to let the Polaris Community know that the PPMC has added three
new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all now
members of the Polaris PPMC.

Please join me in welcoming them,
Russ

Re: Clarification on Unique Entity Identifier in Polaris

[Eric Maynard]

This is more of an interface design question... I tend to think passing
more information down into the persistence layer is a good thing. So if
providing catalog ID, entity type, maybe even subtype, etc. can help the
persistence layer, we shouldn't be afraid to pass it down.

One other thing worth calling out -- there is an implied uniqueness
constraint on (parent ID, entity type, entity name) as well as on entity ID.

On Wed, Mar 26, 2025 at 2:07 PM Honah J.  wrote:

> Thanks for the clarification! Just to confirm — even though the entityId is
> globally unique, we’ll still require catalogId + entityId at the
> persistence API level to perform an entity lookup now and in the future,
> correct?
>
> On Wed, Mar 26, 2025 at 1:45 PM Dennis Huo  wrote:
>
> > Correct, the entityId must be unique across all types.
> >
> > On Wed, Mar 26, 2025 at 12:30 PM Eric Maynard 
> > wrote:
> >
> > > I believe that entity ID by itself is meant to be a unique identifier.
> > >
> > > On Wed, Mar 26, 2025 at 12:27 PM Honah J.  wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I have a question about what constitutes a unique identifier for an
> > > entity
> > > > in Polaris in the future.
> > > >
> > > > Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> > > > following a recent refactoring (in a PR
> > > > ). On the other hand,
> > > > `lookupEntities` currently takes a list of PolarisEntityId, which
> only
> > > > includes catalogId and entityId, and is assumed to represent unique
> > > > entities.
> > > >
> > > > I understand the refactoring is still in progress, but I’d like to
> > > clarify
> > > > the intended direction:
> > > > Will the unique identifier eventually include typeCode as well? Or do
> > we
> > > > still consider catalogId + entityId sufficient for uniqueness, with
> > > > typeCode used solely for lookup optimization?
> > > >
> > > > Best regards,
> > > > Honah (Jonas)
> > > >
> > >
> >
>

Re: Clarification on Unique Entity Identifier in Polaris

[Honah J.]

Thanks for the clarification! Just to confirm — even though the entityId is
globally unique, we’ll still require catalogId + entityId at the
persistence API level to perform an entity lookup now and in the future,
correct?

On Wed, Mar 26, 2025 at 1:45 PM Dennis Huo  wrote:

> Correct, the entityId must be unique across all types.
>
> On Wed, Mar 26, 2025 at 12:30 PM Eric Maynard 
> wrote:
>
> > I believe that entity ID by itself is meant to be a unique identifier.
> >
> > On Wed, Mar 26, 2025 at 12:27 PM Honah J.  wrote:
> >
> > > Hi folks,
> > >
> > > I have a question about what constitutes a unique identifier for an
> > entity
> > > in Polaris in the future.
> > >
> > > Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> > > following a recent refactoring (in a PR
> > > ). On the other hand,
> > > `lookupEntities` currently takes a list of PolarisEntityId, which only
> > > includes catalogId and entityId, and is assumed to represent unique
> > > entities.
> > >
> > > I understand the refactoring is still in progress, but I’d like to
> > clarify
> > > the intended direction:
> > > Will the unique identifier eventually include typeCode as well? Or do
> we
> > > still consider catalogId + entityId sufficient for uniqueness, with
> > > typeCode used solely for lookup optimization?
> > >
> > > Best regards,
> > > Honah (Jonas)
> > >
> >
>

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Keith Chapman]

Congratulations all, well deserved. It's nice to see the community
expanding.

Regards,
Keith.

http://keith-chapman.com


On Wed, Mar 26, 2025 at 7:40 PM Ajantha Bhat  wrote:

> Congratulations guys. Well deserved.
>
> On Thu, Mar 27, 2025 at 6:47 AM Prashant Singh
>  wrote:
>
> > Congratulations folks, well deserved !
> >
> > Best,
> > Prashant Singh
> >
> > On Wed, Mar 26, 2025 at 5:02 PM yun zou 
> > wrote:
> >
> > > Congratulations!
> > >
> > > Best Regards,
> > > Yun
> > >
> > > On Wed, Mar 26, 2025 at 4:28 PM Michael Collado <
> collado.m...@gmail.com>
> > > wrote:
> > >
> > > > Wow! Awesome news. Congrats folks!
> > > >
> > > > Mike
> > > >
> > > > On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer <
> > > russell.spit...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi y'all!
> > > > >
> > > > > I'm excited to let the Polaris Community know that the PPMC has
> added
> > > > three
> > > > > new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are all
> > now
> > > > > members of the Polaris PPMC.
> > > > >
> > > > > Please join me in welcoming them,
> > > > > Russ
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Initial location for Spark Client

[Jean-Baptiste Onofré]

Hi

I’m fine to start with the plugin as part of main repo.
We can always move later if needed.

I guess we also include the plugin in Polaris releases (same version). I
will check the dep and distributed artifacts to verify there is no “legal”
issue.

Regards
JB


Le mar. 25 mars 2025 à 21:20, yun zou  a écrit :

> Hi Team,
>
> Along with the support of Generic Table, we are also introducing a Polaris
> Spark client to interact with the new APIs.
>
> To start the work, I propose to put the client in the Polaris main repo for
> now. As we proceed, we can decide whether to move it to a different repo
> depending on the complexity the client introduces.
>
> Does anyone have a strong opposition on starting the development with the
> main repo?
>
> Best Regards,
> Yun
>

Re: Clarification on Unique Entity Identifier in Polaris

[Jean-Baptiste Onofré]

Hi

Afair entityId is unique.

Regards
JB

Le mer. 26 mars 2025 à 20:27, Honah J.  a écrit :

> Hi folks,
>
> I have a question about what constitutes a unique identifier for an entity
> in Polaris in the future.
>
> Right now, `lookupEntity` takes catalogId, entityId, and typeCode,
> following a recent refactoring (in a PR
> ). On the other hand,
> `lookupEntities` currently takes a list of PolarisEntityId, which only
> includes catalogId and entityId, and is assumed to represent unique
> entities.
>
> I understand the refactoring is still in progress, but I’d like to clarify
> the intended direction:
> Will the unique identifier eventually include typeCode as well? Or do we
> still consider catalogId + entityId sufficient for uniqueness, with
> typeCode used solely for lookup optimization?
>
> Best regards,
> Honah (Jonas)
>

Re: EntityCache race conditions and JCStress in Polaris

[Jean-Baptiste Onofré]

Hi

If the dependency is used for test (and not bundled in one of our
distributed artifacts) and we don’t copy code from this dependency, we can
use GPL/LGPL dependency. What matters is the code we copy from another
project and the dependencies we bundle in our artifacts.

Regards
JB

Le mer. 26 mars 2025 à 14:08, Pierre Laporte  a
écrit :

> Hello all
>
> In https://github.com/apache/polaris/issues/761, there is a discussion on
> whether the EntityCache is thread-safe, prone to race conditions,
> inconsistencies, etc...
>
> To determine whether these issues are real, I worked on JCStress [1]
> tests.  JCStress is a testing framework developed by Oracle that helps
> verify correct behavior of concurrent Java code.  I would like to discuss
> the possibility to add such tests to the Polaris project.
>
> Here is an example of tests:
>
> https://github.com/pingtimeout/polaris/blob/jcstress/jcstress-tests/src/jcstress/java/org/apache/polaris/core/persistence/cache/EntityCacheCoherenceTest.java#L34-L99
> .
> And here is an example of the insights we can get out of them:
>
> https://docs.google.com/document/d/1ZF5WGiSOoqg9JpSvFfhuPOK0QrqeY431/edit?tab=t.0#bookmark=id.op4496ui0vzi
> .
>
> These tests are similar to unit tests, in that there should be no
> dependency on external components, they should be fast so that millions of
> invocations can be performed in a reasonable timeframe, and they should be
> fine-grained.  To that extent, it would make sense to colocate them with
> the codebase, in the main repository.
>
> But JCStress is released under the GPL-v2, which seems incompatible with
> the ASLv2 [2].  Does anybody have any experience with such integrations?
> We would make use of the JCStress framework, as opposed to extending it.
> But couldn't this be considered derivative work?  In which case, should the
> concurrency tests be developed as a separate project (and/or in a separate
> repository) so that they themselves are released under the GPL-v2 ?
>
> Thoughts?
>
> [1] https://github.com/openjdk/jcstress
> [2] https://www.apache.org/licenses/GPL-compatibility.html
>
> --
>
> Pierre
>

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

[Eric Maynard]

Congrats all! The podling is in good hands.

On Wed, Mar 26, 2025 at 9:23 PM Keith Chapman 
wrote:

> Congratulations all, well deserved. It's nice to see the community
> expanding.
>
> Regards,
> Keith.
>
> http://keith-chapman.com
>
>
> On Wed, Mar 26, 2025 at 7:40 PM Ajantha Bhat 
> wrote:
>
> > Congratulations guys. Well deserved.
> >
> > On Thu, Mar 27, 2025 at 6:47 AM Prashant Singh
> >  wrote:
> >
> > > Congratulations folks, well deserved !
> > >
> > > Best,
> > > Prashant Singh
> > >
> > > On Wed, Mar 26, 2025 at 5:02 PM yun zou 
> > > wrote:
> > >
> > > > Congratulations!
> > > >
> > > > Best Regards,
> > > > Yun
> > > >
> > > > On Wed, Mar 26, 2025 at 4:28 PM Michael Collado <
> > collado.m...@gmail.com>
> > > > wrote:
> > > >
> > > > > Wow! Awesome news. Congrats folks!
> > > > >
> > > > > Mike
> > > > >
> > > > > On Wed, Mar 26, 2025 at 2:47 PM Russell Spitzer <
> > > > russell.spit...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi y'all!
> > > > > >
> > > > > > I'm excited to let the Polaris Community know that the PPMC has
> > added
> > > > > three
> > > > > > new members.  Dmitri Bourlatchkov, Dennis Huo and Yufei Gu are
> all
> > > now
> > > > > > members of the Polaris PPMC.
> > > > > >
> > > > > > Please join me in welcoming them,
> > > > > > Russ
> > > > > >
> > > > >
> > > >
> > >
> >
>