Hi Vladimir,
As Ryan said, it's not a bug: CREATE OR REPLACE can be seen as "CREATE
AND UPDATE" from table format perspective. Specifically for the
properties, it makes sense to not delete the current properties as it
can be used in several use cases (security, tables grouping, ...).
I'm not sure
Hi Everyone,
Thanks for all the discussion so far. I have created a PR to document the
requirements https://github.com/apache/iceberg/pull/11365. Please feel free
to review it or discuss further in the thread.
Thanks,
Walaa.
On Fri, Oct 11, 2024 at 5:19 PM Walaa Eldin Moustafa
wrote:
> Hi Ben
Hi Jean-Baptiste,
Agreed. REST spec looks good. I am talking about the general spec, where it
might be useful to add a hint to engine developers, that CREATE OR REPLACE
semantics in Iceberg is expected to follow slightly different semantics.
This is already broken in Trino: depending on catalog ty
Hi Vladimir,
This isn't a bug. The behavior of CREATE OR REPLACE is to replace the data
of a table, but to maintain things like other refs, snapshot history,
permissions (if supported by the catalog), and table properties. Table
properties are replaced if they are set in the operation like `b` in
Hey folks,
Thanks, everyone for the discussion, and thanks Ryan for providing the
historical context.
Enforce the `operation` key in Snapshot’s `summary` field
When serializing the `Snapshot` object from JSON, the Java implementation
does not enforce that the `summary` field must contain an `oper
Was it ever valid to have a summary field without the operation key?
No. They were introduced at the same time.
Would it be helpful to create alternative versions of the REST spec
specifically for referencing V1 and V2 tables?
I am strongly against this. The REST spec should be independent of th
Hi,
Consider a REST catalog and a user calls "CREATE OR REPLACE "
command. When processing the command, engines will usually initiate a
"createOrReplace" transaction and add metadata, such as the properties of a
new table.
Users expect a table to be replaced with a new one if it exists,
including
Thanks everyone for your feedback.
The preferred format is hybrid and we have new talks tracks possible.
I will prepare the formal proposal to submit to PMC for approval and
organization.
Thanks again,
Regards
JB
Le ven. 27 sept. 2024 à 19:50, Jean-Baptiste Onofré a
écrit :
> Hi folks,
>
> La
Hi Ryan,
Thanks for the clarification. Yes, I think my confusion was caused by the
fact that many engines treat CREATE OR REPLACE as a semantic equivalent of
DROP + CREATE, which is performed atomically (e.g., Flink [1]). Table
formats add history on top of that, which is expected to be retained,
