HI David, I will be taking over the work for Flight SQL. Last week, I created a draft PR to Arrow Main <https://github.com/apache/arrow/pull/9368> that addresses the original comments on this PR <https://github.com/ryannicholson/arrow/pull/2>. It would be amazing if you could take a look at the draft PR. The work on Flight SQL is temporarily paused on my end, I will be resuming the work next week. I am also going to send out an email to the dev mailing list once the work on FLight SQL resumes next week. Hopefully I will be able to concentrate on Flight SQL from next week onwards. Exciting times ahead!
Cheers, Tiffany Lam (She/Her) On Mon, Feb 8, 2021 at 10:16 AM David Li <lidav...@apache.org> wrote: > Hi Ryan et. al., > > Following up here- I saw Dremio gave a presentation about this project > recently, and also, over the past several months, the authentication work > has been proceeding. Given that, what does the status of the Flight SQL > project itself look like now? I see there are some unanswered comments in > the doc and in the draft PR, would another round of comments be useful if > those were addressed? And the talk mentioned a release in the next few > months, are there any details you can share there? > > Thanks, > David > > On Thu, Oct 8, 2020, at 13:28, Ryan Nicholson wrote: > > Thanks Wes, > > > > Apologies for the delay. The next steps which we were going to pursue > once > > the authentication redesign direction was solid included a session > concept. > > The initial proposal we would put forth would likely include these > concepts > > via use of headers. I agree that with sessions and databases in mind, a > > close operation would be beneficial. In thinking through this earlier it > > wasn't clear to me if it would make sense to implement "close" as an > Action > > but it also didn't seem to fit as a new first class Flight RPC > altogether. > > > > Ryan > > > > > > > > On Thu, Oct 1, 2020 at 3:44 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > I just reviewed the FlightSQL.proto on > > > https://github.com/ryannicholson/arrow/pull/2. It might be a good idea > > > to document somewhere how some of the various SQL concepts (like > > > sessions) are intended to be handled in Flight (which doesn't have > > > these concepts built in). Does it make sense to have > > > session-lifetime-related RPCs (start/close session, check session > > > status, include a session bytes token with requests, etc.)? > > > > > > On Tue, Sep 29, 2020 at 6:53 PM Ryan Nicholson <rnichol...@gmail.com> > > > wrote: > > > > > > > > Hello Andy and Wes, > > > > > > > > Thanks for reaching out. > > > > > > > > Later this week I have time freeing up to start on Flight-SQL. Would > > > either > > > > of you mind examining the approach in the POC [1] as described in the > > > > "Proof of Concept" section of the proposal document [2]? > > > > > > > > I would certainly be happy to chat about this with anyone interested > in > > > > participating too. > > > > > > > > Thank you, > > > > Ryan > > > > > > > > [1] https://github.com/ryannicholson/arrow/pull/2 > > > > [2] > > > > > > > > https://docs.google.com/document/d/1WQz32bDF06GgMdEYyzhakqUigBZkALFwDF2y1x3DTAI > > > > > > > > > > > > On Tue, Sep 29, 2020 at 8:52 AM Andy Grove <andygrov...@gmail.com> > > > wrote: > > > > > > > > > I am also interested in helping with this effort. > > > > > > > > > > Andy. > > > > > > > > > > On Mon, Sep 28, 2020, 1:58 PM Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > > > > > > > > > hi Ryan, > > > > > > > > > > > > I'm interested in enabling this effort to move forward -- is > there > > > any > > > > > > part where feedback or other help could be useful? > > > > > > > > > > > > Thanks, > > > > > > Wes > > > > > > > > > > > > On Wed, Sep 2, 2020 at 4:45 PM Wes McKinney <wesmck...@gmail.com > > > > > wrote: > > > > > > > > > > > > > > I haven't been able to scrutinize the details greatly, but I am > > > > > > > supportive of this effort, otherwise we may end up with many > > > database > > > > > > > systems and vendors implementing Flight in different ways. It > > > would be > > > > > > > nice if there was a built-in SQL client layer that we ship in > C++ > > > (and > > > > > > > its bindings), Java, and others. > > > > > > > > > > > > > > On Wed, Sep 2, 2020 at 10:24 AM Ryan Nicholson < > > > rnichol...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > Hello everyone, > > > > > > > > > > > > > > > > Here is a link to the slides > > > > > > > > < > > > > > > > > > > > > > > > https://docs.google.com/presentation/d/1JcOKA1yKYTNodQl5Z1BtK3dhJLx-Vu_RSAqWF_wiJH8/edit?usp=sharing > > > > > > > > > > > > > > > I would like to discuss this morning. > > > > > > > > > > > > > > > > Take care, > > > > > > > > Ryan Nicholson > > > > > > > > > > > > > > > > On Tue, Sep 1, 2020 at 4:02 PM Ryan Nicholson < > > > rnichol...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > Thank you to all who have viewed and given feedback in the > > > posted > > > > > > material > > > > > > > > > so far. I am working on responses as well as preparing some > > > > > material > > > > > > which > > > > > > > > > I would like to share for discussion in the sync call > > > tomorrow. I > > > > > > will post > > > > > > > > > the material here ahead of time. > > > > > > > > > > > > > > > > > > Take care, > > > > > > > > > Ryan Nicholson > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 31, 2020 at 2:21 PM Ryan Nicholson < > > > > > rnichol...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Hello everyone, > > > > > > > > >> > > > > > > > > >> I would like to propose the following specification to > ease > > > > > > adoption of > > > > > > > > >> SQL based clients and backends while leveraging data > streams > > > for > > > > > > > > >> transporting data. This specification entails a series of > > > protobuf > > > > > > messages > > > > > > > > >> to be used in the opaque messaging framework payloads to > > > enable > > > > > > browsing of > > > > > > > > >> database metadata and statement style operations with > > > scaffolding > > > > > > available > > > > > > > > >> to help implementers provide logic for their backend. > > > > > > > > >> > > > > > > > > >> Please take a look at the following material: > > > > > > > > >> > > > > > > > > >> - ARROW-9825 < > > > https://issues.apache.org/jira/browse/ARROW-9825 > > > > > > > > > > > > has > > > > > > > > >> more details around the goals of the proposal. > > > > > > > > >> - Flight SQL Proposal > > > > > > > > >> < > > > > > > > > > > > > > > > https://docs.google.com/document/d/1WQz32bDF06GgMdEYyzhakqUigBZkALFwDF2y1x3DTAI/edit?usp=sharing > > > > > > > > > > > > is > > > > > > > > >> a more detailed document describing new messages and > how > > > they > > > > > > would be used. > > > > > > > > >> - https://github.com/ryannicholson/arrow/pull/2 on my > > > Arrow > > > > > fork > > > > > > > > >> shows a sample implementation with a catalog browsing > > > action > > > > > and > > > > > > a prepared > > > > > > > > >> statement query workflow. There are details in the > proposal > > > > > > linking to > > > > > > > > >> specific areas of the PR. > > > > > > > > >> > > > > > > > > >> Thank you for your time, > > > > > > > > >> Ryan Nicholson > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > >
