Re: [RFC] Schema Evolution

2022-11-10 Thread Weston Pace
Sorry about that. I've enabled it now. On Wed, Nov 9, 2022, 9:34 PM Micah Kornfield wrote: > It doesn't look like comment access is enabled? > > On Wed, Nov 9, 2022 at 5:16 PM Weston Pace wrote: > > > I've created a document[1] that both describes the general idea of > > schema evolution as we

Re: DISCUSS: [FlightSQL] Catalog support

2022-11-10 Thread James Henderson
Similarly, we're also currently considering how best to implement some of the SQL standard session variables in our Flight SQL server - things like current transaction isolation level, access mode, time zone etc, which seem to have similar properties to the (traditional) connection's current catalo

Re: DISCUSS: [FlightSQL] Catalog support

2022-11-10 Thread David Li
Hey James H., That would make sense to me. So it sounds like we'd want - Formal specification of using cookies/headers to mark a 'session' (I guess this will be a little inconsistent with transactions, though) - Adding RPCs to query session values - Adding RPCs to set session values - Listing st

Re: Struct evolution

2022-11-10 Thread Weston Pace
> I’ve done something like this in the past. It was two parts - first figure > out the desired schema and then when reading files make them conform to > that schema. Good point. So far I've just been focusing on the second part. There is a dataset discovery step that will try and do the first pa

Re: Struct evolution

2022-11-10 Thread David Li
IIRC the discovery step does already try to unify the schemas, it's just that right now, schema unification is basically not implemented. There's a long-standing Jira/PR [1] that might be good for someone to pick up and push over the finish line. [1]: https://github.com/apache/arrow/pull/12000

Re: Arrow sync call November 9 at 12:00 US/Eastern, 17:00 UTC

2022-11-10 Thread Todd Farmer
Hello, As a quick update on the issue tracking migration: - Basic extract from Jira and conversion/import to GitHub issues works well. Some tweaking is needed around content. - Updating source Jira issues now works [1] - The biggest outstanding issue is how to represent related issues [2] - Issue

Re: [DISCUSS] Maintenance policy

2022-11-10 Thread Sutou Kouhei
Hi, Sorry for not replying to this thread... I was working on post release tasks. It's not completed yet but almost done. I want to get consensus about this to complete the last post release task, "Remove old artifacts". I want to remove unmaintained artifacts from https://dist.apache.org/repos/di

Re: [DISCUSS] Maintenance policy

2022-11-10 Thread Sutou Kouhei
Hi, > 3. Which class of defects should trigger a maintenance release once a fix > is made to the branch? How about the following? - We release X.Y.1 when we have at least one fix in a maintenance branch. - If we have another fix while we're preparing X.Y.1, we also include the another fix. -

Re: [DISCUSS] Maintenance policy

2022-11-10 Thread Sutou Kouhei
Hi, > Also, I know we have high release costs for new versions, but is that also > true for backporting fixes? Unlike new releases, if we were creating a > bugfix release, we are presumably starting from a much more stable point, > right? Right. Additionally, I've improved/added shell scripts fo