Thanks Rayn, then I think I have a path forward. And I will file a feature
request on thread-safe appendManifest on github. Thanks again.

On Thu, Aug 31, 2023 at 10:04 AM Ryan Blue <b...@tabular.io> wrote:

> I think making the operation thread-safe and parallelizing is a good idea.
> It should be pretty easy.
>
> And yes, versions of Iceberg older than the one where that config property
> was added would be the ones where it is unsafe. It's probably safe for most
> people, but we still can't change the default.
>
> On Thu, Aug 31, 2023 at 9:59 AM Pucheng Yang <py...@pinterest.com.invalid>
> wrote:
>
>> Thanks Ryan, what might you consider an "older" version of Iceberg? Is it
>> fair to say any version before
>> https://github.com/apache/iceberg/commit/c3dc9824b381e5e479e356be5e0f4fcf61a9fc37
>> ? If that is the case, my organization controls the Iceberg reader so might
>> be a less concern for me.
>>
>> Another option is to make appendManifests operation thread-safe so that
>> we can parallel the execution when needed (also I think we should add
>> comment to indicate appendManifests API is not thread-safe, this part seems
>> missing, slack discussion:
>> https://apache-iceberg.slack.com/archives/C03LG1D563F/p1693410897758349?thread_ts=1693355747.700759&cid=C03LG1D563F
>> )
>>
>>>
>
> --
> Ryan Blue
> Tabular
>

Reply via email to