On Fri, Aug 29, 2025 at 9:50 AM Ryan Carsten Schmidt < ryandes...@macports.org> wrote:
> On Aug 29, 2025, at 09:39, bl...@netjibbing.com wrote: > > > >> On Aug 29, 2025, at 6:45 AM, Ryan Carsten Schmidt wrote: > >> > >> Users and I have noted selfupdate is slow. Our ports collection has > grown, and for many years we no longer just rsync the ports tree, which was > fast, but instead rsync a tarball which we then decompress, which involves > throwing away and recreating the tens of thousands of files that make up > the entire ports tree each time. > >> > >> Can we speed it up and reduce local disk space usage by not unpacking > the tarball at sync time? Maybe we could keep the portindexes and > _resources folder on disk but leave everything else in the tarball until > it's asked for, e.g. untar the port's directory into the work directory > when the user asks to install that port. > >> > >> Or, any other ideas for speeding up selfupdate? > > > > For me, the download of the tarball is still limited to under 1 MB/s > when the source is fastly here in Seattle. I finally figured out how to > force Macports to use a mirror in California. That significantly speeds up > self-update for me. The other part that takes forever is the port sync. > > Mm, I meant port sync. I want to speed up the daily task of updating the > ports tree. > > The usually much less frequent task of updating base was already recently > improved to use https instead of rsync though that does as you note mean it > is subject to the slowness of the CDN that I never got to the bottom of. > What is the problem with just going back to rsync and hosting the uncompressed port tree? Then only the changed files would be downloaded for port sync. You could also offer a complete tarball available as an alternative, for new installs and full rebuilds. I feel like I am missing something here.
