During the Arrow community meeting today, we decided to include the fix for the wide-dataframe performance regression due to its severity. The ETA is end-of-the-week for the patch. You can review the meeting notes here[1].
[1] https://docs.google.com/document/d/1xrji8fc6_24TVmKiHJB4ECX1Zy2sy2eRbBjpVJMnPmk/edit#heading=h.k1ts4kvvl8jq On Wed, Aug 2, 2023 at 12:43 PM Austin Dickey <aus...@voltrondata.com.invalid> wrote: > Hi all, > > We have generated a new report comparing the Python and R benchmark results > from the contender 13.0.0 RC2 to the baseline 12.0.1 RC1 on the > ursa-i9-9960x machine: > > > http://crossbow.voltrondata.com/release_reports/arrow-release-report-13.0.0-rc2.html > > To dive deeper into the data, here is the same comparison on Conbench: > > > https://conbench.ursa.dev/compare/runs/b5871a9145e545409dd169ec30f17384...833d1302bd98477caaaf2fa428db0512/ > > The main notable performance difference is in the "wide-dataframe > use_legacy_dataset=false" benchmark case, which looks to exhibit a > performance regression, as Raúl mentioned above: > > > https://conbench.ursa.dev/benchmark-results/064ca542414d71f5800054929f91b4a1/ > > Thanks, > Austin > > > On Wed, Aug 2, 2023 at 4:55 AM Raúl Cumplido <rau...@apache.org> wrote: > > > Hi, > > > > As already shared on both Zulip and the Mailing list after creating RC > > 1 we found a couple of regressions on performance. One has been fixed > > and added as part of RC2 but the other is still in progress: > > https://github.com/apache/arrow/issues/36892 > > > > Based on conversations on Zulip it doesn't seem to be a blocker but > > please raise your concerns if you think we should wait for a new RC to > > add it. > > > > I have triggered a new benchmark run on the RC PR but it is still in > > progress: > > https://github.com/apache/arrow/pull/36985#issuecomment-1661559507 > > > > I'll share in this discussion the comparison between the baseline > > (12.0.1) and the current RC once it is available. > > > > Thanks and happy holidays for the ones that are on vacation :) > > Raúl > > > > El mié, 2 ago 2023 a las 11:46, Raúl Cumplido (<rau...@apache.org>) > > escribió: > > > > > > Hi, > > > > > > I would like to propose the following release candidate (RC2) of Apache > > > Arrow version 13.0.0. This is a release consisting of 435 > > > resolved JIRA issues[1]. > > > > > > This release candidate is based on commit: > > > c604084e7c62747780f91d6f8419c47feb4b20fb [2] > > > > > > The source release rc2 is hosted at [3]. > > > The binary artifacts are hosted at [4][5][6][7][8][9][10][11]. > > > The changelog is located at [12]. > > > > > > Please download, verify checksums and signatures, run the unit tests, > > > and vote on the release. See [13] for how to validate a release > > candidate. > > > > > > See also a verification result on GitHub pull request [14]. > > > > > > The vote will be open for at least 72 hours. > > > > > > [ ] +1 Release this as Apache Arrow 13.0.0 > > > [ ] +0 > > > [ ] -1 Do not release this as Apache Arrow 13.0.0 because... > > > > > > [1]: > > > https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A13.0.0+is%3Aclosed > > > [2]: > > > https://github.com/apache/arrow/tree/c604084e7c62747780f91d6f8419c47feb4b20fb > > > [3]: > > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-13.0.0-rc2 > > > [4]: https://apache.jfrog.io/artifactory/arrow/almalinux-rc/ > > > [5]: https://apache.jfrog.io/artifactory/arrow/amazon-linux-rc/ > > > [6]: https://apache.jfrog.io/artifactory/arrow/centos-rc/ > > > [7]: https://apache.jfrog.io/artifactory/arrow/debian-rc/ > > > [8]: https://apache.jfrog.io/artifactory/arrow/java-rc/13.0.0-rc2 > > > [9]: https://apache.jfrog.io/artifactory/arrow/nuget-rc/13.0.0-rc2 > > > [10]: https://apache.jfrog.io/artifactory/arrow/python-rc/13.0.0-rc2 > > > [11]: https://apache.jfrog.io/artifactory/arrow/ubuntu-rc/ > > > [12]: > > > https://github.com/apache/arrow/blob/c604084e7c62747780f91d6f8419c47feb4b20fb/CHANGELOG.md > > > [13]: > > > https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates > > > [14]: https://github.com/apache/arrow/pull/36985 > > >
