Some thoughts...
- In general, many engines want (or may require) a resolved sql field.
This--at minimum--typically includes star expansion since traditional view
behavior is stars are expanded at view creation time (since this is the
only way to guarantee that the view returns the sam
Did not notice that we are also discussing cross-engine interoperability
here, I will add my response in the design doc here.
I would personally prefer cross-engine interoperability as a goal and get
the spec in the right structure in the initial release, because:
1. I believe that cross-engine c
Yep! We discussed this yesterday.
The general plan going forward will be
Phase 1:
Merge Sort based compaction
Allow compaction/rewrite of data files using a space filling curve based sort.
No planning or persisting of metrics.
Phase 2:
Support for Transforms with multiple arguments and possible
Hi Bhavyam,
Has this been discussed on the sync?
Ryan, will it be making into the table metadata spec?
Best,
PF
On Wed, Jul 21, 2021 at 1:50 PM Bhavyam Kamal
wrote:
> Hi Everyone,
>
> I would like to discuss and get feedback on the following proposal for
> Z-Ordering in the Iceberg Sync today:
Hi,
First of all thank you for this discussion and all the view-related work!
I agree that solving cross-engine compatibility problem may not be primary
feature today, I am concerned that not thinking about this from the start
may "tunnel" us into a wrong direction.
Cross-engine compatible views
Hey Anjali,
I am definitely happy to help with implementing 1-3 in your first list once
the spec has been approved by the community. My hope is that the final
version of the view spec will make it easy to re-use existing rollback/time
travel/metadata etc functionalities.
Regarding SQL dialects.My
