On Thu, 2017-08-24 at 11:22 -0700, Sijie Guo wrote: On Tue, Aug 22, 2017 at 12:30 AM, Francesco Caliumi - Diennea < francesco.cali...@diennea.com<mailto:francesco.cali...@diennea.com>> wrote:
I'm not sure if I explained well my point. No docker hub involved, just a tool for every developer who wants to build an image by its own. Maybe to test the stability of a feature branch instead of master, for instance. If we don't push to docker hub, how can this image be used for continuous testing. The idea is to have a nightly image, so this image can be deployed to somewhere to do integration test, jepsen test and other things. I'm not familiar with this use case, but I think the image can be builded before starting integration tests, without any need of docker hub at all (but it adds a couple of minutes to overall test time). This way integration tests could be executed on master and on branches as well. Maybe it could help to well define the use case scenario and the target user. If target user is a developer or tester that needs to test a bk ensemble, having available only the one day delayed build of the master branch could be limiting. On Tue, 2017-08-22 at 14:38 +0800, Jia Zhai wrote: Hi Sijie, For releases version, we have no permissions to do "docker push" images into wanted place(neither under apache <https://hub.docker.com/u/apache/> nor "Official <https://docs.docker.com/docker-hub/official_repos/>"). Thanks. -Jia On Tue, Aug 22, 2017 at 12:55 PM, Sijie Guo <guosi...@gmail.com<mailto:guosi...@gmail.com><mailto:guo si...@gmail.com<mailto:si...@gmail.com>>> wrote: On Aug 21, 2017 12:56 AM, "Francesco Caliumi - Diennea" < francesco.cali...@diennea.com<mailto:francesco.cali...@diennea.com><mailto:francesco.cali...@diennea.com>> wrote: My view for these topics: 1) I'm a big fan of official docker images, so it's ok for me. 2) For latest build, both Jia and Enrico solutions are ok, but I think Enrico's maven target one is the most flexible. Who usually need the nightly build are developers, who do not care very much to have an official or almost-official image, but they like the flexibility to build the image of the commit / branch they want to test. Having a maven target should create a workflow like this: - do some work or git checkout {<branch> / <commit>} - mvn clean package docker:build - (optional: only if working with multiple versions) docker tag <generated_image> <dev_docker_user>/bookkeeper:mytag - (optional: only if the image should be accessibile by others devs / machines) docker push <dev_docker_user>/bookkeeper:mytag - test the image On functionality side, this image will be slightly different from the current one, because it will not need to download the package but it will find it directly in the build dir. Personally I'd like to have a single way to build images for nightly and releases. It is going to be hard to maintain two solutions in long term. On Tue, 2017-08-15 at 00:49 -0700, Sijie Guo wrote: On Tue, Aug 15, 2017 at 12:38 AM, Enrico Olivelli <eolive...@gmail.com<mailto:eolive...@gmail.com> <mailto:eolive...@gmail.com> <mailto:eolive...@gmail.com>> wrote: Il mar 15 ago 2017, 09:16 Sijie Guo <guosi...@gmail.com<mailto:guosi...@gmail.com><mailto:guo si...@gmail.com<mailto:si...@gmail.com>><mailto:guo si...@gmail.com<mailto:si...@gmail.com><mailto:si...@gmail.com>>> ha scritto: On Aug 14, 2017 11:18 PM, "Enrico Olivelli" <eolive...@gmail.com<mailto:eolive...@gmail.com><mailto:eo live...@gmail.com<mailto:live...@gmail.com>><mailto:eo live...@gmail.com<mailto:live...@gmail.com><mailto:live...@gmail.com>>> wrote: Il mar 15 ago 2017, 03:29 Jia Zhai <zhaiji...@gmail.com<mailto:zhaiji...@gmail.com><mailto:zh aiji...@gmail.com<mailto:aiji...@gmail.com>><mailto:zh aiji...@gmail.com<mailto:aiji...@gmail.com><mailto:aiji...@gmail.com>>> ha scritto: Hi Sijie, From my view, the approach problem is whether we have permissions to "docker push" images into wanted place. Following the way putting images under "apache", we seems not have permission, While following the "Official <https://docs.docker.com/docker-hub/official_repos/>" way, seems neither. Since nightly build is mainly for our development, It maybe OK to manage and maintain a dockerhub account by our community to hold the nightly one(maybe also for the release images). On Tue, Aug 15, 2017 at 8:19 AM, Sijie Guo <guosi...@gmail.com<mailto:guosi...@gmail.com><mailto:guo si...@gmail.com<mailto:si...@gmail.com>><mailto:guo si...@gmail.com<mailto:si...@gmail.com><mailto:si...@gmail.com>>> wrote: On Mon, Aug 14, 2017 at 3:47 AM, Jia Zhai <zhaiji...@gmail.com<mailto:zhaiji...@gmail.com><mailto:zh aiji...@gmail.com<mailto:aiji...@gmail.com>><mailto:zh aiji...@gmail.com<mailto:aiji...@gmail.com><mailto:aiji...@gmail.com>>> wrote: Thanks Sijie for raising these good topics up. Regarding 1) official image, the Flink one is similar as Zookeeper one, which we have discussed before. If the currently approach could not make bookkeeper docker image official, we should go this way. Regarding 2) nightly build, there is already an issue <https://github.com/apache/bookkeeper/issues/289> opened. Seems the issue is where to put the nightly build images, since for both way of official image, there is limited access to the dockerhub, The first thought in my head is to place a nightly build somewhere, such as ( https://dist.apache.org/repos/dist/dev/bookkeeper/), and the current docker file will not changed too much, seems only some env var need change. currently we don't have any process to produce any nightly built packages. If we are planning to use dist/dev for hosting the nightly build, we need to figure out how to get the credentials to do that. Because the dist/dev is a svn repo, during a release, the release manager uses its own credentials to commit the new packages to the svn repo. Regarding 3), the build failure <https://hub.docker.com/r/apache/bookkeeper/builds/ bvzft3fsnpmmj5i8jnpk3fl/> is caused by connection(from dockerhub to gpg key server) issue. It is also one reason to come out PR420 <https://github.com/apache/ bookkeeper/pull/420>, which wanted to download local KEY to avoid gpg server connection, But we agree that it is not very security. I think there are a few drawbacks that I can see in current approach: - building the packages and building the docker images are managed by two different systems. - the `latest` image isn't really a tag pointing to the image of latest release; the latest is actually building from master. any changes pushed to master unnecessarily trigger auto build in docker hub, even the package itself isn't changed. "latest" is some kind of place holder, I thought to use it to point to the nightly version, seems I was mis-understanding of it. My two cents IMHO latest should be v the latest stable version Yes that's my thought as well. Then it should be just a tag for the image of latest release. However in current approach, it keeps regenerating a image everytime there is a change on master. This doesn't make any sense to me. We have to touch the latest tag only as part of the release process. Currently latest is not just a tag of 4.5.0. they are separate images generated by docker auto builds. One step back, what is the purpose of having this image? I mean an image of the master branch. It can be used in a CD (continuous deployment) pipeline, you can deploy the image to a dockerized environment to verify if it is working or not. Yup, so it is like the maven snapshots. From the licensing point in that case all the constraints are relaxed, we can really use a PMC managed repository If you only need to have a working bookkeeper server for docker we can use the maven docker plugin which is well integreated with maven based environments. The problem of where to put is is still here. Sometime ago for a project I had just created a free account on docker hub and pushed there the image directly from the build. Jia already created a BookKeeper account, we could use it. Credentials will be managed by PMC Enrico - building a nightly package is not trivial. If we step back and revisit my comment in https://github.com/apache/bookkeeper/pull/197#issuecomment-317831799 , if we use `docker push` approach and let jenkins build and push docker images, it seems to be much easier to address the above issues. Thoughts? - Sijie -- -- Enrico Olivelli -- -- Enrico Olivelli -- Francesco Caliumi Developer @ Diennea - MagNews Tel.: (+39) 0546 066100 - Int. 266 Viale G.Marconi 30/14 - 48018 Faenza (RA) [Magnews.it]<http://www.magnews.it/it> [Linkedin]<http://www.linkedin.com/company/diennea---magnews> [Twitter] <http://twitter.com/DienneaMagNews> [Facebook] < http://www.facebook.com/pages/MagNews/197617841797> [Newsletter] < http://www.magnews.it/it/iscriviti-alla-newsletter> ________________________________ Iscriviti alla nostra newsletter per rimanere aggiornato su digital ed email marketing! http://www.magnews.it/newsletter/ The information in this email is confidential and may be legally privileged. If you are not the intended recipient please notify the sender immediately and destroy this email. Any unauthorized, direct or indirect, disclosure, copying, storage, distribution or other use is strictly forbidden. -- Francesco Caliumi Developer @ Diennea - MagNews Tel.: (+39) 0546 066100 - Int. 266 Viale G.Marconi 30/14 - 48018 Faenza (RA) [Magnews.it]<http://www.magnews.it/it> [Linkedin]<http://www.linkedin.com/company/diennea---magnews> [Twitter] <http://twitter.com/DienneaMagNews> [Facebook] < http://www.facebook.com/pages/MagNews/197617841797> [Newsletter] < http://www.magnews.it/it/iscriviti-alla-newsletter> ________________________________ Iscriviti alla nostra newsletter per rimanere aggiornato su digital ed email marketing! http://www.magnews.it/newsletter/ The information in this email is confidential and may be legally privileged. If you are not the intended recipient please notify the sender immediately and destroy this email. Any unauthorized, direct or indirect, disclosure, copying, storage, distribution or other use is strictly forbidden. -- Francesco Caliumi Developer @ Diennea - MagNews Tel.: (+39) 0546 066100 - Int. 266 Viale G.Marconi 30/14 - 48018 Faenza (RA) [Magnews.it]<http://www.magnews.it/it> [Linkedin]<http://www.linkedin.com/company/diennea---magnews> [Twitter] <http://twitter.com/DienneaMagNews> [Facebook] <http://www.facebook.com/pages/MagNews/197617841797> [Newsletter] <http://www.magnews.it/it/iscriviti-alla-newsletter> ________________________________ Iscriviti alla nostra newsletter per rimanere aggiornato su digital ed email marketing! http://www.magnews.it/newsletter/ The information in this email is confidential and may be legally privileged. If you are not the intended recipient please notify the sender immediately and destroy this email. Any unauthorized, direct or indirect, disclosure, copying, storage, distribution or other use is strictly forbidden.
