Thanks Rahil for driving this and providing the summary of changes since these APIs were originally proposed. I'm +1 (binding) on the spec change.
Thanks, Amogh Jahagirdar On Wed, Sep 4, 2024 at 10:55 AM Chertara, Rahil <rcher...@amazon.com.invalid> wrote: > Thanks Jack and Ryan, I will give an overview of the current design of the > PR[1]. > > > > 1. To avoid duplication between the endpoints, we now have a single > endpoint *planTableScan *which accepts columns projections, a filter, > etc. and initiates a plan. The server will respond back to the client in > either two ways. > 1. Either the server has completed planning and returns *plan-tasks, > file-scan-tasks*. > 2. The server has not completed planning, and returns a *plan-id.* > 2. If the client receives a plan-id, they will call the > *fetchPlanningResult* endpoint which takes in plan-id as input. > Servers will then return a *status* on if planning is completed or > not. The client will continue to call this endpoint until the server > returns a completed status with plan-tasks, file-scan-tasks. > 3. If a client wants to release resources for a given plan, a delete > endpoint was added *cancelPlanning* which takes in plan-id as input. > 4. An endpoint *fetchScanTasks* was added in order for a client to get > the file-scan-tasks associated with a plan-task by providing a plan-task as > input. > > > [1] https://github.com/apache/iceberg/pull/9695 > > > > Regards, > Rahil Chertara > > > > *From: *"rdb...@gmail.com" <rdb...@gmail.com> > *Reply-To: *"dev@iceberg.apache.org" <dev@iceberg.apache.org> > *Date: *Tuesday, September 3, 2024 at 1:24 PM > *To: *"dev@iceberg.apache.org" <dev@iceberg.apache.org> > *Subject: *RE: [VOTE] Merge REST Spec Change To Add New Scan Planning APIs > > > > *CAUTION*: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > +1 > > > > I think it would be good to give an overview of the current proposal since > it has evolved quite a bit from the original like Jack said. > > > > On Tue, Sep 3, 2024 at 9:09 AM Jack Ye <yezhao...@gmail.com> wrote: > > Thanks for keeping pushing for this Rahil. Personally I am +1 (binding) > for this, with just some minor comments in the latest PR. > > > > But I think the initial DISCUSS thread [1] was quite a while ago and a lot > has changed after a lot of comments and reviews. Should we restart another > DISCUSS thread before voting, to make sure people are aware of the latest > design and address any additional comments? > > > > Best, > > Jack Ye > > > > [1] https://lists.apache.org/thread/qq13468x6gk0vxnsckzc5xd02tjlvpkm > > > > > > On Mon, Sep 2, 2024 at 9:22 PM Chertara, Rahil <rcher...@amazon.com.invalid> > wrote: > > Hi all, > > > > I've opened a PR [1] to add REST spec changes for a new protocol around > table scan planning. For context around the design discussions, see the > original google doc proposal [2], the dev list discussion thread [3], and > finally the discussion that has happened on the spec change PR. > > > > Please vote on merging this change. The vote will remain open for at least > 72 hours. > > > > [] +1 > > [] +0 > > [] -1, do not merge because ... > > > > [1] https://github.com/apache/iceberg/pull/9695 > > [2] > https://docs.google.com/document/d/1FdjCnFZM1fNtgyb9-v9fU4FwOX4An-pqEwSaJe8RgUg/edit#heading=h.cftjlkb2wh4h > > [3] https://lists.apache.org/thread/qq13468x6gk0vxnsckzc5xd02tjlvpkm > > > > Thanks, > > Rahil Chertara > >
