We have streaming tables that are written to every 10
> minutes from multiple regions, commits to move the files back to a single
> region, and compaction all happening at the same time. We don't really see
> a significant problem with several writers. The manifest list files are
ave a new PR out to rewrite manifests to take
> advantage of this: https://github.com/apache/incubator-iceberg/pull/200/files
>
> Does that answer your question?
>
>
> On Mon, Jun 3, 2019 at 1:38 PM Erik Wright
> wrote:
> In the process of following up on the "Update
ommits to move the files back to a single
> region, and compaction all happening at the same time. We don't really see
> a significant problem with several writers. The manifest list files are
> generally small enough that it's okay. Definitely better than keeping all
> that inform
icant problem with several writers. The manifest list files are
generally small enough that it's okay. Definitely better than keeping all
that information in the root metadata file.
On Mon, Jun 3, 2019 at 2:13 PM Erik Wright wrote:
> Thanks for the response, Ryan. I can certainly see th
/Deletes/Upserts" thread,
>> I'm re-reading the table spec. I have a question about Manifest List files.
>>
>> If I understand correctly, the manifest list files are separate files
>> that are created prior to attempting to commit a new snapshot. Each
>&g
n about Manifest List files.
>
> If I understand correctly, the manifest list files are separate files that
> are created prior to attempting to commit a new snapshot. Each snapshot may
> have a single manifest list file. The manifest list file references _all_
> manifest files in
In the process of following up on the "Updates/Deletes/Upserts" thread, I'm
re-reading the table spec. I have a question about Manifest List files.
If I understand correctly, the manifest list files are separate files that
are created prior to attempting to commit a new snapshot
