Hi, The conversation around more frequent minor releases and version split per component has been a long one.
I am in favour of these changes for the Go implementation because we have several maintainers. It might be difficult to release other implementations that do not have the same amount of maintainers. I am not sure what our plan is if one of the split implementations has less maintainers and there's a requirement for a release (i.e. security fix) but that might be something to consider in the future. > I would defer to Raul and Jacob to corroborate this, but because > changes to the CI configuration and release verification scripts don't > affect other implementations, I have been able to maintain that > infrastructure myself without too much effort and don't have to lean > on them for anything except reviews. I think releasing and maintaining release scripts / verifications per component is much easier than for the mono repo. We currently have over 200 nightly CI jobs in the mono repo that are required to pass before releasing. Moving some of those to its own repo helps maintainability. Kou, is your plan also counting on moving the specific nightlies there and removing them from the main repo? I would be in favour of doing a new major release (v18) once the repo and the changes are in-place to update the import path to something like: github.com/apache/arrow-go/v18 This would avoid confusion with previous releases. We can then follow up with patch/minor/major as required. I am also happy to help with the releases and infrastructure if necessary as I've done with the main Arrow one (I can also help on nanoarrow, adbc if necessary). Kind regards, Raul > > [1] https://github.com/apache/arrow-nanoarrow/pull/557 > > On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zotthewiz...@gmail.com> wrote: > > > > Part of the goal of splitting out the release processes is that we'd be > > able to do minor version releases more frequently instead of major version > > releases. > > > > The general convention in the Go community is to include a major version > > "v#" in the import path for all major versions past v1 so that if there's a > > breaking change, it's explicit and prevents potential issues from different > > major versions being used simultaneously. Being able to do minor version > > releases more frequently would lead to not having to change the import > > paths every 3-6 months, but only if we actually do a breaking change. > > > > On Thu, Jul 18, 2024, 3:55 PM George Godik <ggo...@gmail.com> wrote: > > > > > > If we shift the Go lib to a new/different import > > > path we'll end up with the same problem where people will rely on older > > > versions and an incorrect path. > > > > > > Major version upgrades already require changing the import paths by > > > increasing the version. The proposed change would require everyone to go > > > through a similar process one last time. > > > > > > > More to the point, there would be the question of whether or not we > > > should port over the same major version > > > number, i.e. `github.com/apache/arrow-go/v17` > > > <http://github.com/apache/arrow-go/v17> > > > <http://github.com/apache/arrow-go/v17> or something to that end? Or > > > do we restart back at v1 (which I think would be confusing)? > > > > > > My vote - for whatever it's worth - would be to do away with the > > > version-in-path naming convention and relying on the go version/package > > > system for major upgrades. > > > > > > Benefits: I don't have to change import paths every 3-6months > > > > > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zotthewiz...@gmail.com> wrote: > > > > > > > My thoughts: > > > > > > > > > * Go doesn't depend on other components such as C++ > > > > > * Go has some active PMC member (Matt) and committer (Joel) > > > > > * Could you become a release manager for Go? > > > > > > > > I'd happily be the release manager for the Go implementation. > > > > > > > > > Here is my idea how to proceed this: > > > > > > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like > > > > > apache/arrow-rs > > > > > * Filter go/ related commits from apache/arrow and create > > > > > apache/arrow-go with them like we did for apache/arrow-rs > > > > > * Remove go/ related codes from apache/arrow > > > > > 2. Prepare integration test CI like apache/arrow-rs does: > > > > > > > > > > > > > > > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml > > > > > 3. Prepare release script based on apache/arrow-julia, > > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql > > > > > > > > Personally I would prefer that we do not extract it to its own separate > > > > repository purely because I don't want to change the import path for > > > users > > > > again. We already have this issue from before we introduced the major > > > > version into the import path and shifted it up to allow for the Parquet > > > lib > > > > in the same repository. If you look at [1] you see that there's still > > > over > > > > 100 projects that never upgraded to v6 or higher because they are still > > > > using the old import path. If we shift the Go lib to a new/different > > > import > > > > path we'll end up with the same problem where people will rely on older > > > > versions and an incorrect path. > > > > > > > > If we as a community decide that splitting out the implementations all > > > into > > > > separate repositories is the best way forward, I won't hold it up by > > > > strictly hammering on this. I'm just concerned about the realities and > > > > difficulties of communicating the import path change, ensuring we don't > > > > break any consumers, and ensuring that users still end up being able to > > > > upgrade easily. > > > > > > > > > The import path could be "github.com/apache/arrow-go" instead of " > > > > github.com/apache/arrow-go/arrow". Since go will allow users to use > > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go` > > > <http://github.com/apache/arrow-go> > > > > <http://github.com/apache/arrow-go> > > > > <http://github.com/apache/arrow-go>. > > > > > > > > The import path would still have to be ` > > > github.com/apache/arrow-go/arrow` > > > <http://github.com/apache/arrow-go/arrow> > > > > <http://github.com/apache/arrow-go/arrow> > > > > since it would also contain the parquet implementation in ` > > > > github.com/apache/arrow-go/parquet` > > > <http://github.com/apache/arrow-go/parquet> > > > > <http://github.com/apache/arrow-go/parquet>. More to the point, there > > > > would be the > > > > question of whether or not we should port over the same major version > > > > number, i.e. `github.com/apache/arrow-go/v17` > > > <http://github.com/apache/arrow-go/v17> > > > > <http://github.com/apache/arrow-go/v17> or something to that end? Or > > > > do we restart back at v1 (which I think would be confusing)? > > > > > > > > --Matt > > > > > > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow > > > > > > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <anto...@python.org> > > > wrote: > > > > > > > > > > > > > > Hi Kou, > > > > > > > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit : > > > > > > > > > > > > Here is my idea how to proceed this: > > > > > > > > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like > > > > > > apache/arrow-rs > > > > > > * Filter go/ related commits from apache/arrow and create > > > > > > apache/arrow-go with them like we did for apache/arrow-rs > > > > > > * Remove go/ related codes from apache/arrow > > > > > > 2. Prepare integration test CI like apache/arrow-rs does: > > > > > > > > > > > > > > > > > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml > > > > > > 3. Prepare release script based on apache/arrow-julia, > > > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql > > > > > > > > > > I think this is a good idea, but I'm not part of the Go maintainers. > > > > > > > > > > > Cons of this idea: > > > > > > > > > > > > * This is a backward incompatible change > > > > > > * Users need to change their "import" to > > > > > > "github.com/apache/arrow-go/arrow" from > > > > > > "github.com/apache/arrow/go/arrow" > > > > > > > > > > Is there no way to leave some kind of alias or redirection in the > > > > > apache/arrow repository? > > > > > > > > > > Regards > > > > > > > > > > Antoine. > > > > > > > > > > > >
