Re: rpmbuild is very slow with large files

2022-07-15 Thread Marius Schwarz
Am 13.07.22 um 16:30 schrieb John Reiser: On 7/11/22 Marius Schwarz wrote: I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 GB rpm and found, that rpmbuild is an extrem bottleneck. IMHO, this is caused by a fileread function which reads files in 32k blocks, which is very

Re: rpmbuild is very slow with large files

2022-07-13 Thread John Reiser
On 7/11/22 Marius Schwarz wrote: I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 GB rpm and found, that rpmbuild is an extrem bottleneck. IMHO, this is caused by a fileread function which reads files in 32k blocks, which is very slow and extrem IO intensive.  The result

Re: rpmbuild is very slow with large files

2022-07-12 Thread Stephen Smoogen
On Mon, 11 Jul 2022 at 18:52, Marius Schwarz wrote: > > Hi, > > I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 > GB rpm and found, that rpmbuild is an extrem bottleneck. > > At that size, is RPM actually a good fit for the data inside it? Building it is going to be slow an

Re: rpmbuild is very slow with large files

2022-07-12 Thread Sérgio Basto
The only coincidence is "large files" the rest is different but may worth check , if you are talking about rawhide. https://pagure.io/copr/copr/issue/2241 On Tue, 2022-07-12 at 00:52 +0200, Marius Schwarz wrote: > > Hi, > > I have just create(d/ not finished yet, started 15 minutes ago) a

Re: rpmbuild is very slow with large files

2022-07-12 Thread Florian Weimer
* Miroslav Lichvar: > On Tue, Jul 12, 2022 at 09:26:14AM +0200, Florian Weimer wrote: >> * Marius Schwarz: >> > and I don't think, this tasks needs to read the clock that often too. >> >> strace shouldn't see a system call here because clock_gettime should be >> handled in the vDSO. This suggest

Re: rpmbuild is very slow with large files

2022-07-12 Thread Florian Festi
On 7/12/22 11:02, Marius Schwarz wrote: > Am 12.07.22 um 10:55 schrieb Marius Schwarz: >> >> The rpmbuild process for this one rpm was single thread. With a >> lsof-loop,  I could see "bytes" getting attached to the resulting file >> with an awful slow progression rate. Which is very frustrating to

Re: rpmbuild is very slow with large files

2022-07-12 Thread Marius Schwarz
Am 12.07.22 um 10:55 schrieb Marius Schwarz: The rpmbuild process for this one rpm was single thread. With a lsof-loop,  I could see "bytes" getting attached to the resulting file with an awful slow progression rate. Which is very frustrating to see on a 8 core system. The thing is, I do te

Re: rpmbuild is very slow with large files

2022-07-12 Thread Marius Schwarz
Am 12.07.22 um 09:47 schrieb Kamil Dudka: On Tuesday, July 12, 2022 12:52:13 AM CEST Marius Schwarz wrote: Multicore use would also be helpful i.e. while packing the files. What do you mean by packing? Creation of the resulting RPM packages? I believe this phase already runs in parallel in cas

Re: rpmbuild is very slow with large files

2022-07-12 Thread Marius Schwarz
Am 12.07.22 um 10:30 schrieb Miroslav Lichvar: be selected by the kernel, or it could be a VM which doesn't have one that would work with migrations, etc. I think your right, it's a vm on xen base. best regards, Marius ___ devel mailing list -- devel

Re: rpmbuild is very slow with large files

2022-07-12 Thread Marius Schwarz
Am 12.07.22 um 09:26 schrieb Florian Weimer: * Marius Schwarz: I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 GB rpm and found, that rpmbuild is an extrem bottleneck. IMHO, this is caused by a fileread function which reads files in 32k blocks, which is very slow and ext

Re: rpmbuild is very slow with large files

2022-07-12 Thread Miroslav Lichvar
On Tue, Jul 12, 2022 at 09:26:14AM +0200, Florian Weimer wrote: > * Marius Schwarz: > > and I don't think, this tasks needs to read the clock that often too. > > strace shouldn't see a system call here because clock_gettime should be > handled in the vDSO. This suggests something is wrong with th

Re: rpmbuild is very slow with large files

2022-07-12 Thread Kamil Dudka
On Tuesday, July 12, 2022 12:52:13 AM CEST Marius Schwarz wrote: > > Hi, > > I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 > GB rpm and found, that rpmbuild is an extrem bottleneck. > > IMHO, this is caused by a fileread function which reads files in 32k > blocks, whic

Re: rpmbuild is very slow with large files

2022-07-12 Thread Florian Weimer
* Marius Schwarz: > I have just create(d/ not finished yet, started 15 minutes ago) a ~2.5 > GB rpm and found, that rpmbuild is an extrem bottleneck. > > IMHO, this is caused by a fileread function which reads files in 32k > blocks, which is very slow and extrem IO intensive.  The result is a > t