> The best solution is to make the build repeatable. IIUC, that will let us build all of the artifacts using CI instead of personal machines, which removes this class of errors.
I agree! But I don't know if Apache rules allow this operation, and if so, I recommend using the CI to publish our project. Thanks, Zixuan Zike Yang <z...@apache.org> 于2023年6月1日周四 18:45写道： > > > The best solution is to make the build repeatable. IIUC, that will let > us build all of the artifacts using CI instead of personal machines, > which removes this class of errors. > > I don't think this issue is related to where the docker image was > built. It has nothing to do with caching and nothing to do with the > machine on which it is built. It's just because the build uses the > `latest` tag, which points to an older version. > I'm + 0.5 for this solution. I am currently wondering how much reward > it will bring. > > > If we are able to prepare the docker image from the RELEASED tarbal > and connectors then I am fine with it. > > We currently build our images using compiled tarball rather than > rebuilding from sources. You can check the code here: > https://github.com/apache/pulsar/blob/5e6e6cebcdbeec32ed49729f658f2d5cd0d98347/docker/pulsar/Dockerfile#L25-L26 > > BR, > Zike Yang > > On Thu, Jun 1, 2023 at 4:52 PM Asaf Mesika <asaf.mes...@gmail.com> wrote: > > > > I understand that you can't build a release process due to Apache > > foundation rules, that makes it mandatory to release something you built on > > your own machine. > > > > > > On Wed, May 31, 2023 at 5:54 PM Enrico Olivelli <eolive...@gmail.com> wrote: > > > > > Il giorno mer 31 mag 2023 alle ore 16:50 Michael Marshall > > > <mmarsh...@apache.org> ha scritto: > > > > > > > > The best solution is to make the build repeatable. IIUC, that will let > > > > us build all of the artifacts using CI instead of personal machines, > > > > which removes this class of errors. > > > > > > > > That being said, I don't know how much effort it would be to achieve. > > > > > > > > Thanks, > > > > Michael > > > > > > > > On Tue, May 30, 2023 at 2:24 AM Zike Yang <z...@apache.org> wrote: > > > > > > > > > > Hi, Enrico > > > > > > > > > > > When we ran the VOTE and we provided the docker images, were they > > > > > already broken ? > > > > > > > > > > Actually, they are not broken unless we use the new features of Pulsar > > > 3.0.0. > > > > > > It is not Pulsar 3 but Pulsar 2.11. > > > So for users this is a problem > > > > > > If we are able to prepare the docker image from the RELEASED tarbal > > > and connectors then I am fine with it. > > > If you have to rebuild from the sources I strongly believe that we > > > cannot do it without a proper release process > > > > > > Enrico > > > > > > > > > > > > > > > > > > > I think we need something like verification test scripts, to verify > > > > > the release candidate. For example, we use the image provided in the > > > > > RC to run the integration tests. And we need to make sure that we have > > > > > tested the newly added feature for the RC docker image. > > > > > > > > > > What do you think? > > > > > > > > > > BR, > > > > > Zike Yang > > > > > > > > > > On Tue, May 30, 2023 at 3:14 PM Zike Yang <z...@apache.org> wrote: > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > How do you suggest we prevent it from happening next time? > > > > > > > > > > > > I have pushed a PR to fix it: > > > https://github.com/apache/pulsar/pull/20435 > > > > > > This PR specifies the correct image name for `pulsar` image to build > > > pulsar-all. > > > > > > > > > > > > Note that, in the release of Pulsar 3.0, we build the docker image > > > > > > by > > > > > > executing the following command instead of the `docker/build.sh`: > > > > > > ``` > > > > > > mvn install -DUBUNTU_MIRROR=http://azure.archive.ubuntu.com/ubuntu/ > > > \ > > > > > > -DskipTests \ > > > > > > -Pdocker -Pdocker-push \ > > > > > > -Ddocker.platforms=linux/amd64,linux/arm64 \ > > > > > > -Ddocker.organization=snzkyang \ > > > > > > -pl docker/pulsar,docker/pulsar-all > > > > > > ``` > > > > > > I think to take it a step further, we could fix these > > > scripts(build.sh > > > > > > and publish.sh) and use the shell scripts to build the image. > > > > > > > > > > > > I have verified the PR, and it works well. Please see more detail in > > > > > > the PR description. > > > > > > > > > > > > Thanks, > > > > > > Zike Yang > > > > > > > > > > > > On Mon, May 29, 2023 at 9:50 PM Enrico Olivelli > > > > > > <eolive...@gmail.com> > > > wrote: > > > > > > > > > > > > > > I am really worried about the process. > > > > > > > > > > > > > > When we ran the VOTE and we provided the docker images, were they > > > > > > > already broken ? > > > > > > > > > > > > > > In any case we cannot overwrite those images, they have been > > > > > > > cached > > > > > > > all over the world now. > > > > > > > > > > > > > > It is safer to cut a new 3.0.1 release and run a VOTE. > > > > > > > > > > > > > > Maybe we can remove the old images, forever > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > Il giorno lun 29 mag 2023 alle ore 13:55 Asaf Mesika > > > > > > > <asaf.mes...@gmail.com> ha scritto: > > > > > > > > > > > > > > > > Good catch! > > > > > > > > > > > > > > > > How do you suggest we prevent it from happening next time? > > > > > > > > > > > > > > > > On Mon, May 29, 2023 at 1:34 PM Zike Yang <z...@apache.org> > > > wrote: > > > > > > > > > > > > > > > > > Hi, all > > > > > > > > > > > > > > > > > > Recently, we found an issue with the `pulsar-all:3.0.0` image. > > > The > > > > > > > > > pulsar library included in `pulsar-all:3.0.0` is the version > > > > > > > > > of > > > > > > > > > 2.11.0: > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > docker run apachepulsar/pulsar-all:3.0.0 ls lib/ | grep > > > pulsar-broker > > > > > > > > > > > > > > > > > > org.apache.pulsar-pulsar-broker-2.11.0.jar > > > > > > > > > org.apache.pulsar-pulsar-broker-auth-sasl-2.11.0.jar > > > > > > > > > org.apache.pulsar-pulsar-broker-common-2.11.0.jar > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > The root cause is that we use `apachepulsar/pulsar:latest` to > > > build > > > > > > > > > the `pulsar-all` image. But at the time of building Pulsar > > > 3.0.0, > > > > > > > > > `apachepulsar/pulsar:latest` was pointing to version 2.11.0. > > > > > > > > > > > > > > > > > > Therefore, the `pulsar-all:3.0.0` is actually a version 2.11.0 > > > of > > > > > > > > > Pulsar but with 3.0.0 connectors and offloaders. > > > > > > > > > > > > > > > > > > Please see more detail in this issue: > > > > > > > > > https://github.com/apache/pulsar/issues/20420 > > > > > > > > > > > > > > > > > > I have rebuilt the `pulsar-all:3.0.0` image: > > > > > > > > > > > > > > > > > > > > > https://hub.docker.com/layers/snzkyang/pulsar-all/3.0.0/images/sha256-833ea988bce8c704b179cc4c9c38fac8980e108b0bc67454e06c22927990b169?context=explore > > > > > > > > > > > > > > > > > > Please help and verify it. And check if there are any other > > > problems > > > > > > > > > with the image. > > > > > > > > > > > > > > > > > > I'm going to publish the image to the `apachepulsar` > > > organization to > > > > > > > > > replace the old one. But before we do that, do we need a Vote > > > or other > > > > > > > > > ways to reach a consensus? Is there any problem if we replace > > > the old > > > > > > > > > image? > > > > > > > > > > > > > > > > > > Besides, I will also fix the docker build script to avoid > > > similar issues. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Zike Yang > > > > > > > > > > > >
