Re: F37 Change: Ansible 5 (Self-Contained Change proposal)
On Sat, Oct 16, 2021 at 8:04 AM Nico Kadel-Garcia wrote:
> On Fri, Oct 15, 2021 at 11:39 PM Kevin Fenzi wrote:
> >
> > On Fri, Oct 15, 2021 at 07:44:12PM -0400, Nico Kadel-Garcia wrote:
> > > On Fri, Oct 15, 2021 at 7:29 PM Nico Kadel-Garcia
> wrote:
> > > >
> > > > On Fri, Oct 15, 2021 at 4:32 PM Ben Cotton
> wrote:
> > > > >
> > > > > https://fedoraproject.org/wiki/Changes/Ansible5
> > > > >
> > > > > == Summary ==
> > > > >
> > > > > The ansible project has re-organized how they release and
> distribute
> > > > > ansible. This change moves Fedora to be in sync with those changes
> and
> > > > > retires the old 'ansible classic/2.9.x' package in favor of a
> > > > > 'ansible' package that pulls in ansible-core (the engine) and
> includes
> > > > > all the collections in upstream ansible releases.
> > > >
> > > > I wrote to the various upstream bugtrackers about this already. The
> > > > re-org upstream is confusing and unwelcome, and creates a stack of
> > > > problems.
> >
> > Yeah, it's been confusing to people for sure, but it does also help out
> > a lot with other problems. :(
>
> it could have made good sense, and still would, for the "ansible"
> package to be what is now being colloquially referred to as
> "ansible-core", but for which the published upstream git repo is still
> https://github.com/ansible/ansible, and which is and will remain
> accessible as a github release tarball with the old numbering. The
> pypi.org published "ansible-core" is a republication of that repo with
> a new name duck-taped on it. Fragmenting out the bulky and potentially
> dynamic set of tools that are now in the "galaxy collections" suite
> makes some sense, but the result is that to get any of the core
> modules like "ansible.posix" we wind up including 573 Megabytes of
> unneeded and unwelcome debris in
> /usr/lib/python3.6/site-packages/ansible_collections. Very few of us
> need more than 10% of the list
>
> There is no specific source repository for the "ansible_collections"
> tarball, as best I can tell. The list of modules selected from the
> galaxy collection is very large, but incomplete and I've not seen any
> criteria for what goes in that tarball and what does not. Have you
> seen any?
>
> I'd suggest that discarding the pypi.org renaming and instead using
> the raw tarballs from github as sources for individual galaxy
> collection modules helps resolve. This would keep the numbering of the
> "ansible" RPM consistent, and its core functionality. The modules
> which have been shifted to the galaxy collection, such as the "ec2"
> and "cisco" modules, could and should be bundled individually as
> Fedora has been doing quite effectively. I did some tests: an RPM of
> the current pypi.org tarball mislabeled as "ansible" occupies more
> than 570 MBytes of local disk. If you want a lightweight Ansible
> setup, say for applying some playbooks to your localhost, it's an
> unreasonable burden.
>
Why not have both?
* In my experience and anecdotally, most Ansible users want to do a `dnf
install ansible` and get the entire 570 MB worth of plugins. Therefore, it
makes sense for Fedora to provide such an RPM named `ansible`. And this is
what other distributions are doing.
* But as you point out, there are many use cases (small device, edge, low
bandwidth, low disk space) where having an `ansible-core` package + the
ability to install just the needed collections is preferable.
I think the `ansible-core` package should not `Provides: ansible` because
ansible-core is most definitely not the historical `ansible` package. It
is misleading at best, dangerous at worst.
> Those numbers do not include the documentation: The sphinx build of
> the HTML docs is something I've tried, but so far is not working with
> my test SRPM. As it is, I've had to rename doc or license files that
> have " " in the filename because the '%doc' and '%license' macros do
> not like whitespace in filenames, and split them up because a
> '%license' or '%doc' that have so many hundreds of entries overwhelms
> SRPM scripting. Packaging up the individual modules or sets of modules
> individually avoids this unwelcome burden.
>
> > > > I would publish ansible-core as just that, with a "Provides: ansible
> > > > %{version{-%{release}" and even "Obsoletes: ansible >= %{version}".
> >
> > That would radically diverge from upstream and cause _more_ confusion.
>
> Upstream already confused people. I don't think it justifies repeating
> their bundling mistakes, mistakes that are not reflected in the
> upstream source code for the software but only in intermediate
> repackaging and which Fedora has so far effectively avoided.
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_g
Re: F37 Change: Ansible 5 (Self-Contained Change proposal)
On Sat, Oct 16, 2021 at 10:39 PM Nico Kadel-Garcia wrote: > On Sat, Oct 16, 2021 at 10:22 PM Kevin Fenzi wrote: > > > > On Sat, Oct 16, 2021 at 08:51:46PM -0500, Maxwell G via devel wrote: > > > > > > Hi everyone, > > > > > > I have a couple comments/questions about this change. > > > > > > How will this effect EPEL? Is the plan to keep Ansible 2.9 there for > > > now? > > > > For now yes, but 2.9 will go EOL at the end of the year (last I heard). > > I think someone is being very, very optimistic. It's especially tricky > because EPEL does not keep previously released versions of software in > their primary repos. If ansible the github tagged ansible 2.10, > rebundled as ansible-core > > > > Could we also consider making Ansible a modular package on both Fedora > > > and EPEL? Then, it would be possible to install any of the currently > > > supported versions of the Ansible core/engine (ansible 2.9, ansible- > > > base 2.10, and ansible-core 2.11). > > > > No thanks. It would be a ton of effort, and those will all EOL soon > > or already have... > > Don't count on it. The python 3.8 requirement for ansible-core 2.12 > and ansible 5.0, as described at Installing Ansible — Ansible > Documentation is going to case difficulties porting it to RHEL > especially RHEL 7. There are no "python38" or "python310" packages > I've found for RHEL 7, except the SCLO packages which are awkward to > use. Many people have been very, very reluctant to update to RHEL 8 > and CentOS 8 for various reasons, so I expect to see some demand for > ansible to roll back that pending required python update, or perhaps > EPEL to organize a python310 suite of python components. > > > (from the ansible 4.7.0 release announcement: > > > > "* Except for ansible-2.9.x, older versions of ansible are no longer > > seeing maintenance releases." > > > > > Will the new `ansible` package have virtual provides for the > > > collections it provides? While there is not a good reason to, it will > > > still be possible to install both the new `ansible` package and any of > > > the ansible-collection-* packages, right? > > > > I don't think we will do provides, that would cause problems installing > > the stand alone collections. So, yeah, we want to be able to install the > > stand alone collections along with or in addition to the bundle. > > I'd prefer "instead of". That might require a more complete set of > collection components, to avoid installing "ansible" and potentially > conflicting. > How about this: * Have a single ansible dist git repo with a single spec file * The source would be https://pypi.org/project/ansible/ * Produce multiple RPM packages from this spec file * ansible-core * one RPM package for each collection in the pypi ansible package - these could be created programatically in the spec file using LUA or other spec file automation to create separate package, documentation, files, Requires, etc. for each collection package * a package called "ansible" which is essentially a meta-package which simply does a "Requires: ansible-core" and also "Requires" every collection package That way, there is a single source for any/all combinations of ansible-core + collection rpms, or the entire ansible containing everything. One downside is that this restricts the ability to produce fixes or upgrades for individual collections independently of the rest of the Ansible packages. But I think that's only a problem if * the independent collections change frequently * the https://pypi.org/project/ansible lags behind and doesn't keep up with the collections > > > Also, I would be happy to help with Ansible packaging in Fedora; > > > however, I am not yet part of the packager group and still need a > > > sponsor. > > You'll also need a stiff drink. The process is slow because of the > size of the monolithic collection, and the cleanups needed to get all > the docs and licenses packaged. > > > Thanks. I appreciate the PR's... :) > > > > kevin > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: h
Re: F37 Change: Ansible 5 (Self-Contained Change proposal)
On Tue, Oct 19, 2021 at 3:36 PM David Moreau-Simard wrote: > (replying from hyperkitty as I'm not subscribed to fedora-devel, hopefully > this works) > > > How about this: > > > > * Have a single ansible dist git repo with a single spec file > > * The source would be https://pypi.org/project/ansible/ > > There couldn't be a single source because 'ansible' and 'ansible-core' are > two distinct packages: > - https://pypi.org/project/ansible/ > - https://pypi.org/project/ansible-core/ > > Since >=2.10.x, 'ansible' is mostly composed of the ansible_collections > directory with a selection of included collections whereas 'ansible-core' > provides the CLI, runtime, builtin modules/plugins, etc. > > > * Produce multiple RPM packages from this spec file > > * ansible-core > > * one RPM package for each collection in the pypi ansible package - > > these could be created programatically in the spec file using LUA or > other > > spec file automation to create separate package, documentation, files, > > Requires, etc. for each collection package > > * a package called "ansible" which is essentially a meta-package > which > > simply does a "Requires: ansible-core" and also "Requires" every > > collection > > package > > > > That way, there is a single source for any/all combinations of > ansible-core > > + collection rpms, or the entire ansible containing everything. > > > > One downside is that this restricts the ability to produce fixes or > > upgrades for individual collections independently of the rest of the > > Ansible packages. But I think that's only a problem if > > > > * the independent collections change frequently > > * the https://pypi.org/project/ansible lags behind and doesn't keep up > > with the collections > > In fact, we first considered proposing something that was similar to what > you suggest -- an ansible package that pulls in individually packaged > collections as well as ansible-core - before settling on the change at > https://fedoraproject.org/wiki/Changes/Ansible5. > > We concluded that it would be hard to do in practice because: > - Ansible 5 will ship with ~95 individual collections ( > https://github.com/ansible-community/ansible-build-data/blob/main/5/ansible.in > ) > - Of these 95 collections, there's only about a dozen that are already > packaged (I wrote down some notes here: > https://hackmd.io/iAloYlTQQ3e-yi99w_o54A) > - The versions of individually packaged collections could diverge from > those shipped inside the 'ansible' package (ex: > https://github.com/ansible-community/ansible-build-data/blob/main/5/ansible-5.0.0a2.deps > ) > - Even considering we develop tooling and automation around the process, > it would still be error prone and a lot of work without mentioning the > administrative overhead of maintaining 95+ packages > > In short: it would require a non-negligible amount of effort to package > each individual collection, keep them updated while in sync with the > versions that the upstream package ships without conflicts. > ok > The change that is currently proposed is somewhat of a middle ground to > unblock the situation and allow us to move forward: > - Users who aren't interested in the "batteries included" or "kitchen > sink" package can install ansible-core (note that ansible-core by itself > contains far less modules/plugins than ansible 2.9.x as those were split > out >=2.10.x) > Indeed - https://docs.ansible.com/ansible-core/2.11/collections/ansible/builtin/index.html (but note that this list doesn't include filters, or jinja2 built-ins) - Users of the 'ansible-core' package can install collections from Ansible > galaxy or from individually packaged collections > - Users of the 'ansible' package can install collections from Ansible > galaxy or from individually packaged collections and they will have > precedence over the ones installed by 'ansible' by default > - It doesn't prevent collections from being individually packaged if a > maintainer wants to do so > > Hopefully that makes sense and I'm happy to answer questions. > Yes, this makes sense. ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject
