Re: Retry ValidationException with concurrent writes to the same partition

2024-12-03 Thread Kevin Liu
Hi Ha, Thanks for the question! Typically, when concurrent writes happen to the same partition, the writer must retry the operation. This is because the state of the table (and the partition) has changed, and the current write cannot be safely applied to the updated state. For example, if the oper

Re: Retry ValidationException with concurrent writes to the same partition

2024-12-03 Thread Yufei Gu
If you’re looking for finer-grained isolation beyond the snapshot level, the closest feature currently *WIP* is *Fine-Grained Commit* in the REST catalog. You can find more details here: Fine-Grained Commit Design Document

Retry ValidationException with concurrent writes to the same partition

2024-12-03 Thread Ha Cao
Hello, I have some concurrent writes to the same partition and they overlap in data files, and the isolation level is snapshot. Expectedly, I get this ValidationException thrown from this line