Re: lazy loading of filelists.xml to speed up dnf

2018-08-08 Thread Björn Persson
Michael Schroeder wrote: > I don't think this is hard to implement, but there's a little detail > that needs to be discussed: what should happen if the filelists.xml > download fails? This can happen because the metadata has been rewritten > in the meantime. Doesn't that race condition already ex

Re: lazy loading of filelists.xml to speed up dnf

2018-08-07 Thread Jeff Johnson
Sure extending the whitelist -- either through hard coded patterns or an actual list -- obviates the need for downloading bloated file lists. Those techniques simply move "useful" data out of file lists and into another file, either primary.xml or a whitelist. Patterns (and Provides: /path) ha

Re: lazy loading of filelists.xml to speed up dnf

2018-08-07 Thread Jeff Johnson
At the risk of again being banned from Fedora, I point out this thread from 2011, discussing/measuring various techniques for reducing the size of metadata containing many file paths for Mandriva http://rpm5.org/community/rpm-devel/.html ___ devel

Re: lazy loading of filelists.xml to speed up dnf

2018-08-07 Thread Jeff Johnson
Bloom filters have false positives that can lead to difficult to predict behavior. The core proble(s) being reported with file dependencies is the size of the download, the cost of the load, and the size/number of copies one for each location managed by dnf. These are all problems that are stra

Re: lazy loading of filelists.xml to speed up dnf

2018-08-07 Thread Michael Schroeder
On Mon, Aug 06, 2018 at 04:36:07PM +, Zbigniew J??drzejewski-Szmek wrote: > this mail is a continuation of an FPC [1] and a FESCo [2] tickets. > > A proposal was made is to disallow packages in Fedora from using file > deps, and to optimize dnf to not load filelists.xml. File deps would > stil

Re: lazy loading of filelists.xml to speed up dnf

2018-08-06 Thread John Reiser
On 08/06/2018 04:42 PM, Jeff Johnson wrote: Whether dnf does what yum does misses the point: Lazy downloading of file dependencies did not work then and does not work now. Here's why: The fundamental decision made by depsolvers is what packages are "newer" and need to be upgraded. When _ANY_

Re: lazy loading of filelists.xml to speed up dnf

2018-08-06 Thread Jeff Johnson
Whether dnf does what yum does misses the point: Lazy downloading of file dependencies did not work then and does not work now. Here's why: The fundamental decision made by depsolvers is what packages are "newer" and need to be upgraded. When _ANY_ commonly used package has a file dependency o

Re: lazy loading of filelists.xml to speed up dnf

2018-08-06 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Aug 06, 2018 at 07:52:03PM +0200, Robert-André Mauchin wrote: > On lundi 6 août 2018 18:36:07 CEST Zbigniew Jędrzejewski-Szmek wrote: > > Hi dnf and libsolv developers, > > > > this mail is a continuation of an FPC [1] and a FESCo [2] tickets. > > > > A proposal was made is to disallow pa

Re: lazy loading of filelists.xml to speed up dnf

2018-08-06 Thread Robert-André Mauchin
On lundi 6 août 2018 18:36:07 CEST Zbigniew Jędrzejewski-Szmek wrote: > Hi dnf and libsolv developers, > > this mail is a continuation of an FPC [1] and a FESCo [2] tickets. > > A proposal was made is to disallow packages in Fedora from using file > deps, and to optimize dnf to not load filelists