I agree with Guillaume and lean towards a -1. IMHO we should not produce a release with incorrect bytecode (even if this is only detectable by other tools like ASM).
Of course, we should try to find the root cause (although it seems quite elusive); but if there is a simple workaround to generate a RC avoiding this issue, I think we should go for it. Personally, I did not create a Jira ticket at the time because apparently there is nothing wrong on our side: it seemed either an ASM bug (disproved in [1]), or some kind of problem with the environment of the 1.36 release manager (OS? JDK?). I honestly thought it would be a one-time-only problem, but now we have a new release by a different RM with the same issue, so it seems a bit more serious than I originally expected. Best, Ruben [1] https://gitlab.ow2.org/asm/asm/-/issues/318008 On Tue, Apr 30, 2024 at 10:41 AM Francis Chuang <francischu...@apache.org> wrote: > My vote is: +1 (binding) > > - Verified GPG signature - OK > - Verified SHA512 - OK > - Diffed source release and git repository - OK > - Checked release notes on tag > ( > https://github.com/apache/calcite/blob/calcite-1.37.0-rc4/site/_docs/history.md) > > - OK > - Checked year and versions in NOTICE, README and HOWTO - OK > - Ran tests (gradle check) - OK > - Spot checked Nexus artifacts - OK > > Environments: > Eclipse-temurin:8 docker container in WSL2 (Ubuntu 22.04.4) on Windows > 11 23h2 > > Eclipse-temurin:19 docker container in WSL2 (Ubuntu 22.04.4) on Windows > 11 23h2 > > $ docker version > Client: > Cloud integration: v1.0.35+desktop.13 > Version: 26.0.0 > API version: 1.45 > Go version: go1.21.8 > Git commit: 2ae903e > Built: Wed Mar 20 15:16:45 2024 > OS/Arch: linux/amd64 > Context: default > > Server: Docker Desktop > Engine: > Version: 26.0.0 > API version: 1.45 (minimum version 1.24) > Go version: go1.21.8 > Git commit: 8b79278 > Built: Wed Mar 20 15:18:01 2024 > OS/Arch: linux/amd64 > Experimental: false > containerd: > Version: 1.6.28 > GitCommit: ae07eda36dd25f8a1b98dfbf587313b99c0190bb > runc: > Version: 1.1.12 > GitCommit: v1.1.12-0-g51d5e94 > docker-init: > Version: 0.19.0 > GitCommit: de40ad0 > > $ gradle -v > > ------------------------------------------------------------ > Gradle 7.6.1 > ------------------------------------------------------------ > > Build time: 2023-02-24 13:54:42 UTC > Revision: 3905fe8ac072bbd925c70ddbddddf4463341f4b4 > > Kotlin: 1.7.10 > Groovy: 3.0.13 > Ant: Apache Ant(TM) version 1.10.11 compiled on July 10 2021 > JVM: 19.0.2 (Eclipse Adoptium 19.0.2+7) > OS: Linux 5.15.146.1-microsoft-standard-WSL2 amd64 > > $ java -version > openjdk version "1.8.0_402" > OpenJDK Runtime Environment (Temurin)(build 1.8.0_402-b06) > OpenJDK 64-Bit Server VM (Temurin)(build 25.402-b06, mixed mode) > > $ java -version > openjdk version "19.0.2" 2023-01-17 > OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7) > OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode, > sharing) > > Francis > > On 30/04/2024 6:35 pm, Stamatis Zampetakis wrote: > > Ubuntu 20.04.6 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 7.6.1 > > > > * Checked signatures and checksums OK > > * Checked diff between repo and artifacts OK > > * Checked README, NOTICE, LICENSE OK > > * All source files have ASF headers OK ( grep -RiL "Licensed to the > > Apache Software Foundation") > > * No unexpected binary files OK (find . -type f -exec file {} \; | > > grep -v text) > > * Checked structure of staged maven repo for calcite-core [1] OK > > * Checked LICENSE, NOTICE, signature, and checksum for > > calcite-core-1.37.0.jar from staged maven repo [1] OK > > * Built from git tag and run tests (./gradlew build) OK > > * Built from source artifacts and run tests (gradle build) OK > > > > +1 (binding) > > > > For the ASM problem, I would suggest opening a JIRA ticket and > > continuing the discussion there. I found the previous discussion [2] > > but not a ticket for this. We should try to keep vote threads as clean > > as possible and keep technical exchanges on separate threads/tickets. > > The main deliverable of this vote is the source package distribution > > so I don't consider this a blocker for the release especially since > > this has popped up before and was left untreated. > > > > Best, > > Stamatis > > > > [1] > https://repository.apache.org/content/repositories/orgapachecalcite-1230/org/apache/calcite/calcite-core/1.37.0/ > > [2] https://lists.apache.org/thread/o736wz4qnr4l285bj5gv073cy0qll9t0 > > > > On Tue, Apr 30, 2024 at 9:50 AM Ruben Q L <rube...@gmail.com> wrote: > >> > >> Thanks Guillaume for checking this! > >> When I looked at this issue on 1.36, I had the impression that > assertions > >> might have a role to play in the error, but I could not confirm this > >> hypothesis. > >> IMO this smells like a JDK bug (or at least it looks like other resolved > >> issues); note that it seems we had some similar problems in the past > with > >> older Java8 versions [1]. > >> > >> Independently of the root cause, would it unblock the release process > if we > >> just remove the problematic asserts, or if we substitute them with an > >> equivalent `if (...) throw new IllegalStateException("...");` ? > >> > >> Best, > >> Ruben > >> > >> [1] > >> > https://github.com/apache/calcite/blob/1506857f404037b63dfd8a11880393b767bd1544/build.gradle.kts#L98 > >> > >> > >> > >> On Mon, Apr 29, 2024 at 11:26 PM Sergey Nuyanzin <snuyan...@gmail.com> > >> wrote: > >> > >>> Hi Guilluame, > >>> > >>> I played a bit more and I realised that if from commit above I just > remove > >>> one line with assert (assert map != null) > >>> then ArrayIndexOutOfBoundsException disappears > >>> > >>> same for current main branch, if I remove all lines from SqlFunctions > with > >>> assert (now there are 3 such lines) then > >>> ArrayIndexOutOfBoundsException disappears > >>> > >>> Thus, it does not look like a Calcite issue, more like a problem on > ASM > >>> side > >>> please correct me if I'm wrong > >>> > >>> On Mon, Apr 29, 2024 at 10:33 PM Sergey Nuyanzin <snuyan...@gmail.com> > >>> wrote: > >>> > >>>> i follow the procedure described here > >>>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate > >>>> started > >>>> btw I played a bit with git bisect and it shows that the issue > >>>> > >>>> java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of > bounds > >>> for length 297 at > >>> org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2695) > at > >>> org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2711) > >>>> > >>>> started appearing after this commit > >>>> > >>>> > >>> > https://github.com/apache/calcite/commit/bcf6bd8577b25c563b1c597c70704594a18ca1a3 > >>>> > >>>> On Mon, Apr 29, 2024 at 10:01 PM Guillaume Masse > >>>> <masse.guilla...@narrative.io.invalid> wrote: > >>>> > >>>>> Hi Sergey, > >>>>> > >>>>> thanks for trying that update > >>>>> > >>>>> I confirm I have the same issue with your release at > >>>>> > >>>>> > >>> > https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/ > >>>>> > >>>>> > >>>>> > >>> > https://github.com/MasseGuillaume/asm-remapper-bug/commit/852e4cd246d278db8acf5e997a54619bc4f85fc7 > >>>>> > >>>>> This rules out one of my hypothesis on the java build version. > >>>>> > >>>>> Can you point me to the release procedure you are using? I saw > >>>>> https://calcite.apache.org/develop/#contributing is there more > precise > >>>>> steps? > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Apr 29, 2024 at 3:27 PM Sergey Nuyanzin <snuyan...@gmail.com > > > >>>>> wrote: > >>>>> > >>>>>> I repeated same procedure with jdk1.8u412 > >>>>>> here you can find the jars > >>>>>> > >>>>>> > >>>>> > >>> > https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/ > >>>>>> it looks like asm-remapper gives the same result... > >>>>>> > >>>>>> On Mon, Apr 29, 2024 at 8:41 PM Guillaume Masse > >>>>>> <masse.guilla...@narrative.io.invalid> wrote: > >>>>>> > >>>>>>> If you take a look at > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > https://www.openlogic.com/openjdk-downloads?field_java_parent_version_target_id=416&field_operating_system_target_id=426&field_architecture_target_id=391&field_java_package_target_id=All > >>>>>>> > >>>>>>> The most recent release is 8u412-b08 > >>>>>>> > >>>>>>> If it follows more or less the openjdk schedule: > >>>>>>> > >>>>>>> https://wiki.openjdk.org/display/jdk8u/Main > >>>>>>> > >>>>>>> *Most recent and past release details:* > >>>>>>> > >>>>>>> - 8u412-b08 (GA), April 16th 2024 [Release < > >>>>>> https://bit.ly/openjdk8u412 > >>>>>>>> ] > >>>>>>> [Tag < > https://github.com/openjdk/jdk8u/releases/tag/jdk8u412-ga > >>>> ] > >>>>> [ > >>>>>>> Binaries > >>>>>>> < > >>>>>>> > >>>>> > https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u412-b08 > >>>> > >>>>>>> ] > >>>>>>> - 8u402-b06 (GA), January 16th 2024 [Release > >>>>>>> <https://bit.ly/openjdk8u402>] [Tag > >>>>>>> <https://github.com/openjdk/jdk8u/releases/tag/jdk8u402-ga>] > >>>>>> [Binaries > >>>>>>> < > >>>>>>> > >>>>> > https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u402-b06 > >>>> > >>>>>>> ] > >>>>>>> - 8u392-b08 (GA), October 17th 2023 [Release > >>>>>>> <https://bit.ly/openjdk8u392>] [Tag > >>>>>>> <https://github.com/openjdk/jdk8u/releases/tag/jdk8u392-ga>] > >>>>>> [Binaries > >>>>>>> < > >>>>>>> > >>>>> > https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u392-b08 > >>>> > >>>>>>> ] > >>>>>>> - 8u382-b05 (GA), July 18th 2023 [Release < > >>>>>> https://bit.ly/openjdk8u382 > >>>>>>>> ] > >>>>>>> [Tag < > https://github.com/openjdk/jdk8u/releases/tag/jdk8u382-ga > >>>> ] > >>>>> [ > >>>>>>> Binaries > >>>>>>> < > >>>>>>> > >>>>> > https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u382-b05 > >>>> > >>>>>>> ] > >>>>>>> - 8u372-b07 (GA), April 18th 2023 [Release < > >>>>>> https://bit.ly/openjdk8u372 > >>>>>>>> ] > >>>>>>> [Tag < > https://github.com/openjdk/jdk8u/releases/tag/jdk8u372-ga > >>>> ] > >>>>> [ > >>>>>>> Binaries > >>>>>>> < > >>>>>>> > >>>>> > https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u372-b07 > >>>> > >>>>>>> ] > >>>>>>> > >>>>>>> > >>>>>>> This means your java version is from April 2023. Can you try to > >>> update > >>>>>> your > >>>>>>> system to the latest build 8u412-b08 and publish the jarfile > >>>>> somewhere I > >>>>>>> can test ? > >>>>>>> > >>>>>>> > >>>>>>> On Mon, Apr 29, 2024 at 2:25 PM Sergey Nuyanzin < > >>> snuyan...@gmail.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Thanks for testing Guillaume > >>>>>>>> > >>>>>>>>> Sergey can you give me all the details about your build > >>> environment > >>>>>> so I > >>>>>>>>> can reproduce that bytecode? > >>>>>>>>> OS / java --version > >>>>>>>> yep, sure > >>>>>>>> Ubuntu 22.04.4 LTS, > >>>>>>>> 1.8.0_372-372 (OpenLogic-OpenJDK 25.372-b07) > >>>>>>>> > >>>>>>>> I will also try to look at it with my set up and the info you've > >>>>>>> provided > >>>>>>>> > >>>>>>>> On Mon, Apr 29, 2024 at 8:03 PM Guillaume Masse > >>>>>>>> <masse.guilla...@narrative.io.invalid> wrote: > >>>>>>>> > >>>>>>>>> -1 (not binding) > >>>>>>>>> > >>>>>>>>> The bytecode of classfile > >>>>>> org/apache/calcite/runtime/SqlFunctions.class > >>>>>>>> is > >>>>>>>>> invalid (and only this classfile) > >>>>>>>>> > >>>>>>>>> I tested all class files from the following artifacts: > >>>>>>>>> calcite-core-1.37.0.jar > >>>>>>>>> calcite-linq4j-1.37.0.jar > >>>>>>>>> avatica-core-1.25.0.jar > >>>>>>>>> avatica-metrics-1.25.0.jar > >>>>>>>>> > >>>>>>>>> This will prevent bytecode tools like asm (https://asm.ow2.io/) > >>>>> from > >>>>>>>>> transforming calcite bytecode (for example shading a > >>> dependency). > >>>>>>>>> > >>>>>>>>> See > >>>>>>>>> https://gitlab.ow2.org/asm/asm/-/issues/318008 > >>>>>>>>> > >>> https://lists.apache.org/thread/nrt4ysoc14p20sq23z744jyfqh1bznyh > >>>>>>>>> > >>>>>>>>> To reproduce > >>>>>>>>> https://github.com/MasseGuillaume/asm-remapper-bug > >>>>>>>>> > >>>>>>>>> Sergey can you give me all the details about your build > >>>>> environment > >>>>>> so > >>>>>>> I > >>>>>>>>> can reproduce that bytecode? > >>>>>>>>> OS / java --version > >>>>>>>>> > >>>>>>>>> I do not have the root cause of the issue. > >>>>>>>>> I suspect it's related to import > >>>>>>>>> org.checkerframework.checker.nullness.qual.Nullable > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Apr 29, 2024 at 10:56 AM Sergey Nuyanzin < > >>>>>> snuyan...@gmail.com> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> The issue CALCITE-6390 with failing of Arrow Adapter tests on > >>>>>> Windows > >>>>>>>>> while > >>>>>>>>>> building from source is fixed. > >>>>>>>>>> Thanks a lot to Caican Can for highlighting the issue > >>>>>>>>>> Stamatis for quick PR > >>>>>>>>>> and Ruben and Alessandro for the review! > >>>>>>>>>> > >>>>>>>>>> I have created a build for Apache Calcite 1.37.0, release > >>>>>>>>>> candidate 4. > >>>>>>>>>> > >>>>>>>>>> Thanks to everyone who has contributed to this release. > >>>>>>>>>> > >>>>>>>>>> You can read the release notes here: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > https://github.com/apache/calcite/blob/calcite-1.37.0-rc4/site/_docs/history.md > >>>>>>>>>> > >>>>>>>>>> The commit to be voted upon: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=044db3d72ee53c1a82ce68b0c2a9b4f0bed81f18 > >>>>>>>>>> > >>>>>>>>>> Its hash is 044db3d72ee53c1a82ce68b0c2a9b4f0bed81f18 > >>>>>>>>>> > >>>>>>>>>> Tag: > >>>>>>>>>> https://github.com/apache/calcite/tree/calcite-1.37.0-rc4 > >>>>>>>>>> > >>>>>>>>>> The artifacts to be voted on are located here: > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >>> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.37.0-rc4 > >>>>>>>>>> (revision 68857) > >>>>>>>>>> > >>>>>>>>>> The hashes of the artifacts are as follows: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > a6ae7ea1ebbab3e5b723122c9517412d74b6bccba9fb41d360f7c2262ed7f26e5a4cc861a711d4813d27d72a0ffd086ae3a9175abc3c874b079137c84d13f83a > >>>>>>>>>> *apache-calcite-1.37.0-src.tar.gz > >>>>>>>>>> > >>>>>>>>>> A staged Maven repository is available for review at: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > https://repository.apache.org/content/repositories/orgapachecalcite-1230/org/apache/calcite/ > >>>>>>>>>> > >>>>>>>>>> Release artifacts are signed with the following key: > >>>>>>>>>> https://people.apache.org/keys/committer/snuyanzin.asc > >>>>>>>>>> https://www.apache.org/dist/calcite/KEYS > >>>>>>>>>> > >>>>>>>>>> To create the jars and test Apache Calcite: "gradle build" > >>>>>>>>>> (requires an appropriate Gradle/JDK installation) > >>>>>>>>>> > >>>>>>>>>> Please vote on releasing this package as Apache Calcite > >>> 1.37.0. > >>>>>>>>>> > >>>>>>>>>> The vote is open for the next 72 hours and passes if a > >>> majority > >>>>> of > >>>>>> at > >>>>>>>>>> least three +1 PMC votes are cast. > >>>>>>>>>> > >>>>>>>>>> [ ] +1 Release this package as Apache Calcite 1.37.0 > >>>>>>>>>> [ ] 0 I don't feel strongly about it, but I'm okay with the > >>>>>> release > >>>>>>>>>> [ ] -1 Do not release this package because... > >>>>>>>>>> > >>>>>>>>>> Here is my vote: > >>>>>>>>>> > >>>>>>>>>> +1 (binding) > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Best regards, > >>>>>>>>>> Sergey > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Best regards, > >>>>>>>> Sergey > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best regards, > >>>>>> Sergey > >>>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Best regards, > >>>> Sergey > >>>> > >>> > >>> > >>> -- > >>> Best regards, > >>> Sergey > >>> >
