Just following up on this and if there are any thoughts. The purpose would be to standardize how we specify access to some named logical grouping of data. This would make it easy to model catalog/schema semantics in Flight SQL.
Having this be part of the connection URI makes it similar to specifying a resource in an HTTP URL (ie an endpoint) which should make it easy for end users to work with and modify. On Fri, Nov 18, 2022 at 3:17 PM James Duong <jam...@bitquilltech.com> wrote: > As for surfacing catalogs itself, perhaps we allow the URI take in a path > and treat that as a way of specifying a multi-level resource that which the > FlightClient is connecting to: > > eg a connection URI of the form: > scheme://<host>:<port>/path-1/path-2/.../path-n > > The FlightClient could send this path as either a header or a session > property (with a neutral name like 'resource-path'). Flight SQL Producers > could interpret this as a catalog or schema. > eg > grpc://<host>:<port>/catalog/schema > > On Fri, Nov 11, 2022 at 2:07 AM James Henderson <j...@juxt.pro> wrote: > >> Sounds good to me. >> >> > Are you interested in writing up a (sketch of a) proposal? >> >> Yep, can do - I'm OoO over the next couple of weeks so might be a bit >> intermittent. >> >> On Thu, 10 Nov 2022 at 15:28, David Li <lidav...@apache.org> wrote: >> >> > Hey James H., >> > >> > That would make sense to me. So it sounds like we'd want >> > >> > - Formal specification of using cookies/headers to mark a 'session' (I >> > guess this will be a little inconsistent with transactions, though) >> > - Adding RPCs to query session values >> > - Adding RPCs to set session values >> > - Listing standard values and types >> > >> > Some things may require more consideration, e.g. transaction isolation >> > might be better off as part of the transaction RPCs than an ambient >> > property. Are you interested in writing up a (sketch of a) proposal? >> > >> > -David >> > >> > On Thu, Nov 10, 2022, at 10:09, James Henderson wrote: >> > > Similarly, we're also currently considering how best to implement >> some of >> > > the SQL standard session variables in our Flight SQL server - things >> like >> > > current transaction isolation level, access mode, time zone etc, which >> > seem >> > > to have similar properties to the (traditional) connection's current >> > > catalog. We'd (perhaps naively) looked at solutions involving the >> Flight >> > > client's `ClientCookieMiddleware`, but might these have standardised >> > > support within Flight SQL itself eventually? >> > > >> > > Cheers, >> > > >> > > James >> > > >> > > On Wed, 9 Nov 2022 at 21:51, David Li <lidav...@apache.org> wrote: >> > > >> > >> I think having better support for this makes sense, but perhaps we >> can >> > >> find a way to make it not tied to the connection itself? For >> instance, >> > in >> > >> the same way transactions were implemented (as a handle). Or rather, >> > >> instead of adding connection statefulness to Flight RPC, I'd rather >> try >> > to >> > >> work within the gRPC/RPC paradigm. >> > >> >> > >> On Wed, Nov 9, 2022, at 16:47, James Duong wrote: >> > >> > Databases such as SQL Server and PostgreSQL have the concept of >> > catalogs >> > >> as >> > >> > containers of database schemas. Users can usually specify an >> initial >> > >> > catalog during the connection process, list catalogs, and sometimes >> > >> change >> > >> > catalogs during the session. >> > >> > >> > >> > Currently catalog support in Flight SQL is somewhat limited. The >> > protocol >> > >> > provides a way to list catalogs as well as metadata in SqlTypeInfo >> for >> > >> > reporting how catalogs are supported from a syntax perspective. >> > >> > >> > >> > ODBC and JDBC provide API calls for changing the catalog. >> > Additionally, >> > >> > Flight SQL doesn't really provide the concept of "initial" >> connection >> > >> > properties (such as a starting catalog) since Flight itself is >> > stateless >> > >> > from a connection perspective. >> > >> > >> > >> > To support catalogs properly, I'd imagine we need to make some >> > changes to >> > >> > the Flight SQL protocol: >> > >> > - Introduce the concept of connection-time properties (perhaps an >> > >> optional >> > >> > RPC for Flight SQL applications that need this) >> > >> > - Related to the above, expand the connection URL and Java builder >> to >> > >> allow >> > >> > arbitrary application-specific properties. >> > >> > - Add optional RPCs for changing the catalog and relevant error >> codes >> > if >> > >> > this is not permitted. >> > >> > >> > >> > >> > >> > -- >> > >> > >> > >> > *James Duong* >> > >> > Lead Software Developer >> > >> > Bit Quill Technologies Inc. >> > >> > Direct: +1.604.562.6082 | jam...@bitquilltech.com >> > >> > https://www.bitquilltech.com >> > >> > >> > >> > This email message is for the sole use of the intended recipient(s) >> > and >> > >> may >> > >> > contain confidential and privileged information. Any unauthorized >> > >> review, >> > >> > use, disclosure, or distribution is prohibited. If you are not the >> > >> > intended recipient, please contact the sender by reply email and >> > destroy >> > >> > all copies of the original message. Thank you. >> > >> >> > > >> > > >> > > -- >> > > *James Henderson* >> > > XTDB Developer at *JUXT* >> > > Email j...@juxt.pro >> > > Website https://juxt.pro >> > > >> > > [image: photo] >> > >> >> >> -- >> *James Henderson* >> XTDB Developer at *JUXT* >> Email j...@juxt.pro >> Website https://juxt.pro >> >> [image: photo] >> > > > -- > > *James Duong* > Lead Software Developer > Bit Quill Technologies Inc. > Direct: +1.604.562.6082 | jam...@bitquilltech.com > https://www.bitquilltech.com > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > -- *James Duong* Lead Software Developer Bit Quill Technologies Inc. Direct: +1.604.562.6082 | jam...@bitquilltech.com https://www.bitquilltech.com This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Thank you.
