Just a comment from an outsider (but one that peeks at the communication and I am very much interested as PyArrow binary wheel release is one of the important prerequisites for Apache Airflow official 3.11 support - mostly as transitive dependencies for other dependencies we use).
I am not sure whether the changes included any code that should find its way into the official source package on the https://downloads.apache.org/arrow/ - but if yes, then there is not much choice from the Apache By-laws but the convenience/compiled packages (and this is what PyPI packages are) should always be built basing on the officially released sources https://www.apache.org/legal/release-policy.html#compiled-packages > In all such cases, the binary/bytecode package MUST have the same version > number as the source release and MUST only add binary/bytecode files that are > the result of compiling that version of the source code release and its > dependencies. If there are no changes to the released software (and just packaging/commands etc.), there is always an option to do a "post" release https://peps.python.org/pep-0440/#post-releases - but it is generally discouraged for anything different than for example fixing release documentation embedded in the package. Also comment from the "user" perspective. This is not a deal breaker to wait for 2 months, but it is an inconvenience. We had ~ 6 months delay after 3.10 was released, and our users will wait as well. However PyArrow is a bit special because it holds others a bit. We have a few dependencies that put their 3.11 migration on hold waiting for PyArrow binary wheels and I see PyArrow as an important "domino" - once PyArrow is out there, the others will follow, which means that only then they will discover what "else" needs to be done. I did a very detailed survey and got a PR in Airflow where I have an inventory of what needs to be done (and I could exclude everything that needs PyArrow in our CI - until it's ready): https://github.com/apache/airflow/pull/27264 but many of the other dependencies were stopped at installation time and they have not made any further checks. This is what caused such a long delay in 3.10 - because the "3.10" support had to bubble up through a number of "layers" of dependencies. So if this is just a matter of releasing, that would enable a number of other teams to start working on their 3.11 support much faster and hopefully we will be there much faster. If there is anything I can help with testing the release - I am happy to do so. We have quite a good set of tests for a number of our dependencies - maybe not a complete and comprehensive (Airflow is mostly about orchestration). We do not have automated "pyarrow functionality" tests, but if you have an RC of pyarrow, I am happy to modify the PR I have and test at least the installation part (and possibly also flag to projects that I find still problematic and ask them to run tests with the RC). J. On Mon, Nov 7, 2022 at 11:50 AM Raúl Cumplido <raulcumpl...@gmail.com> wrote: > > Hi, > > As you might be aware, the release of Python 3.11 happened around the time > we were releasing Apache Arrow 10.0.0. > There seems to be quite a lot of users that would like to use Pyarrow on > the new Python 3.11 version, this can be seen on the amount of comments on > the PR that added wheels for Python 3.11 [1], several GitHub issues or Jira > tickets being opened [2][3][4][5][6] and some stackoverflow questions [7]. > > As the next release is more than 2 months away I wanted to propose to > create a minor release adding wheels for pyarrow for Python 3.11. > > Currently wheels are part of our Release so I am not sure if there's any > other possibility apart from creating a minor release. As the wheels will > have to be published I am not sure if we could do something similar to what > we did for 8.0.1 for the go security patch release where we did not publish > other binaries or we would have to go through a full release. > > Thanks, > Raúl > > [1] https://github.com/apache/arrow/pull/14499 > [2] https://issues.apache.org/jira/browse/ARROW-17487 > [3] https://issues.apache.org/jira/browse/ARROW-18154 > [4] https://issues.apache.org/jira/browse/ARROW-18245 > [5] https://issues.apache.org/jira/browse/ARROW-18153 > [6] https://github.com/apache/arrow/issues/14572 > [7] > https://stackoverflow.com/questions/74296856/install-pyarrow-in-vs-code-for-windows
