Re: Proposal - Priority based commit ordering on partitions

2022-10-06 Thread Péter Váry
One more thing: - In Hive, we order the data files based on the original fileName and rowNum. This helps when reading the delete files, as we do not need to keep delete file data in memory. Iceberg tables could be sorted, so we either has to keep the already read delete data in memory, or reread th

Re: Proposal - Priority based commit ordering on partitions

2022-10-06 Thread Prashant Singh
Hello all, I was OOO, just saw the mail. Thanks Ryan and Peter for the feedback, will address it and update the doc accordingly. As some of us are not available in the proposed slot, and also need some time to address the feedback. Will move this meeting to next week, and propose some slots acc