Hi Pierre On 10/4/24 10:25 PM, PIERRE AUGIER wrote:
I agree that for a big repo like mercurial-devel, it makes sense to avoid the full conversion. However, for small repo, the Git mirroring is still useful, in particular for visibility.
I agree with that, and I know you've been waiting for this feature for a long time. I mean, done in a proper way, not only hitchhiking on the legacy internal conversions to Git that were the way Heptapod worked in its early propotype phase, as I believe you are doing with your fluiddyn projects.
However, I am not confident to be able to do it reliably enough in the near future, as there is also much work to be done to maintain the viability of the solution. The issue I have started nicknaming the "refgeddon" [1] is an example of that, in the category of purely technical constraints. That is why I am more in favour of the external repo approach. I see with your later email on this list that you've made more progress in that direction.
However, what I can and will do is to start some of the groundwork towards Git-push mirrors as part of the "refgeddon": securing the Git repo so that it is not converted to a format that we cannot handle and making the conversion asynchronous, so that it stops being a price to pay on each push.
Cheers, [1]: https://foss.heptapod.net/heptapod/heptapod/-/issues/1764
For Mercurial, we "just" need: - to have a job a Heptapod CI job that produces the sdist and pushes it somewhere where it can be downloaded. It is better because the sdist is produced from the hg repo. - a tiny repo on Githut (basically only a README, license and workflow files) with a workflow triggered by foss.heptapod.net which downloads the sdist, builds the wheels and tests them. We need to figure out how cibuildwheel can build wheels from a sdist and not from a repo but that should be quite easy. I can check that with a sdist from PyPI. We need to figure out how a job on foss.heptapod.net can trigger a job on Github, and if it is possible to send a bit of data (where the sdist has to be downloaded and if it is a simple commit or a release). For the release we can even download the sdist from PyPI. And we will even be able to use this workflow on a Mercurial version already released, to upload wheels for 6.8.1. For the rust part, Mercurial core dev has to choose. Is it still necessary to upload wheels without Rust or can we only upload wheels with Rust (so that all users using Mercurial installed from PyPI would have by default the Rust extensions)? If it is still necessary for some cases to use Mercurial without Rust, I really think Rust extensions should be moved in a separate PyPI project. ----- Mail original -----De: "Georges Racinet" <georges.raci...@cloudcrane.io> À: "mercurial" <mercurial@lists.mercurial-scm.org> Envoyé: Vendredi 4 Octobre 2024 16:13:04 Objet: Re: Producing Mercurial wheels from Github Actions + modernize build process Hi Pierre ! On 10/4/24 2:17 PM, PIERRE AUGIER wrote: - @gracinet do you think heptapod/heptapod#355 could be implemented quite soon so that we can get a good mirror of [ https://foss.heptapod.net/mercurial/mercurial-devel | https://foss.heptapod.net/mercurial/mercurial-devel ] on Github? I may have a better idea, but let me answer the direct question first: On one hand, I do not have time for much these days. On the other hand, nice wheels would certainly save me (if with Rust) and my coworkers quite a bunch of time, so I can consider it (probably not for the upcoming 17.5). Notably in maintenance of CI images, such wheels would make much easier to change version from the one provided in distro packages. I would probably go the virtualenv way everywhere and be happy with it. I wouldn't make a UI though, as it is too time consuming and not necessary to accommodate Mercurial. I suppose that the maintainers would not have any problem setting it up with the API :-) A simpler solution, perhaps more robust and less time-consuming / risky for me : Why not create a Git project on GitHub dedicated to building Mercurial only (not a mirror). Let's call it `hg-bin-build` for the time being. ts CI/CD actions could then just grab the relevant tarball from foss.heptapod.net (or pull from mercurial-scm.org) and perform the binary builds. I don't mean moving the actual build recipes in there, just to execute them. It would thus have only the GitHub action definitions a `VERSION` file. There are many advantages: no need to rely on hg-git etc to do that (and I do not have to dodge heptapod#1764). One could even imagine a CI job on foss.heptapod.net that triggers it in tag pipelines by committing and pushing (with Git). There can be variations on the same idea. Here's a more self-hosted variant: have the `hg-bin-build` Git repo on foss.h.n (somewhat easier to use the proper tokens to push on it from a `mercurial-devel` CI job) and activate Git push mirroring to GitHub. If noone has objections with such a plan, I would prefer it. Switching back to using Mercurial-to-Git push mirroring (heptapod#355) would still be possible in the future if people find it more satisfying. In terms of PR, one could say that Mercurial is depending on Git, and that is to be avoided. My take is that this use case is well bounded and actually should be considered on the positive side as a demo that using Mercurial in a Git-centric world is perfectly possible. It's not possible to have different flavors for wheels on PyPI. The right way (from the point of view of the PyPA) to distribute with PyPI "Mercurial with C extensions" and "Mercurial with Rust extensions" is to have a package mercurial (with C extensions) and another package hg-rust, which could be an optional dependency of mercurial so that things like pipx install mercurial[rust] works. One would need in mercurial's pyproject.toml [project.optional-dependencies] rust = ["hg-rust"] This would be fine by me, and indeed a huge progress. A practical example: in the Heptapod Development Kit, managing of dependencies is a real pain (the whole setup is been done by a collection of Makefiles, not my choice) and it happens that packages depending on Mercurial get installed hence also Mercurial (without Rust) before the target that is supposed to install Mercurial (with Rust), and there is no way to replace that with the proper version other than uninstalling the wrong one. Of course I tend not to notice immediately, only when some tests start failing because the repos have the nodemap. Worse, there is sometimes a cached wheel (without Rust). I would be really happy to just depend on `hg-rust` or whatever the end name happens to be. Thank you for your efforts on this subject, it seems we are not so far from a solution. Side note: well yes I would be happy if we could have Windows and MacOS runners on foss.heptapod.net. Our past experience with the former is that they are awfully time-consuming. Last time I checked, Heptapod Runner worked on Darwin, but I have no reasonable place to host it. Best, -- Georges Racinet [ https://orbeet.io/a-propos/cloudcrane | https://orbeet.io/a-propos/cloudcrane ] , [ https://heptapod.net/ | https://heptapod.net ] GPG: 09719FFE0B476DC26923F8EEB8EB20361976F291 _______________________________________________ Mercurial mailing list Mercurial@lists.mercurial-scm.org https://lists.mercurial-scm.org/mailman/listinfo/mercurial_______________________________________________ Mercurial mailing list Mercurial@lists.mercurial-scm.org https://lists.mercurial-scm.org/mailman/listinfo/mercurial
-- Georges Racinet https://orbeet.io/a-propos/cloudcrane, https://heptapod.net GPG: 09719FFE0B476DC26923F8EEB8EB20361976F291
OpenPGP_0xB8EB20361976F291.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
_______________________________________________ Mercurial mailing list Mercurial@lists.mercurial-scm.org https://lists.mercurial-scm.org/mailman/listinfo/mercurial
