+1 for Steven's suggestion. The `newFixedThreadPool` is slightly better
IMHO, but I would be happy with both solutions
On Thu, Sep 26, 2024, 02:06 Steven Wu wrote:
>
> First, we should definitely add Javadoc to `ThreadPools.newWorkerPool` on
> its behavior with a shutdown hook. It is not obvious
I agree that it is reasonable to assume/restrict view definition and
storage table in the same catalog. Hence the storage table reference in the
view metadata can include only namespace and table (excluding the engine
dependent catalog name/alias).
Regarding the question of having lineage metadata
Hi Russell,
Thanks for testing it out! It's a bit unfortunate that we found this issue
after the RC stage. I've made a fix for it:
https://github.com/apache/spark/pull/48257 . I think it should work but
let's confirm it. After it gets merged, we can probably wait for a while to
accumulate more fix
First, we should definitely add Javadoc to `ThreadPools.newWorkerPool` on
its behavior with a shutdown hook. It is not obvious from the method name.
I would actually go further to deprecate `newWorkerPool` with
`newExitingWorkerPool`. `newWorkerPool` method name is easy to cause the
misuage, as the
Checked and extending Delegating Catalog Extension will be quite difficult
or at least cause several breaks in current Iceberg SparkSessionCatalog
implementations. Note this has nothing to do with third party catalogs but
more directly with how Iceberg works with Spark regardless of Catalog
impleme
I do not see release notes for 1.6.1.
Shouldn't https://iceberg.apache.org/releases/ have a section for 1.6.1 and
highlights of the changes? (And for 1.6.1 to show up in the Table of
contents on the right?)
On Wed, Aug 28, 2024 at 8:34 AM Carl Steinbach wrote:
> I'm pleased to announce the rele
I think it should be minimally difficult to switch this around on the
Iceberg side, we only have to move the initialize code out and duplicate
it. Not a huge cost
On Sun, Sep 22, 2024 at 11:39 PM Wenchen Fan wrote:
> It's a buggy behavior that a custom v2 catalog (without extending
> DelegatingC
+1
Sorry for the delay.
Regards
JB
On Fri, Sep 20, 2024 at 11:02 PM Sung Yun wrote:
>
> Hi folks,
>
> I'd like to start this thread to vote on dropping the support for
> Python3.8 in the upcoming 0.8.0 PyIceberg release.
>
> Python3.8 will be End-Of-Life in October 2024, and some of our
> depen
BTW, have recent community sync recordings been uploaded to YouTube? I only
see catalog sync recordings.
On Wed, Sep 25, 2024 at 1:56 AM Sung Yun wrote:
> Thank you Kevin!
>
> Sung
>
> On 2024/09/24 17:51:54 Kevin Liu wrote:
> >
> https://docs.google.com/document/d/1iPGVCIcr-M0XtAiudOguWAvmqIdVg
