Re: fast-import slowness when importing large files with small differences

2018-07-03 Thread Mike Hommey
On Tue, Jul 03, 2018 at 06:05:16PM +0200, Ævar Arnfjörð Bjarmason wrote: > > On Fri, Jun 29 2018, Mike Hommey wrote: > > > On Sat, Jun 30, 2018 at 12:10:24AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > >> On Fri, Jun 29 2018, Mike Hommey wrote: > >> > >> > I noticed some slowness when fast-impor

Re: fast-import slowness when importing large files with small differences

2018-07-03 Thread Ævar Arnfjörð Bjarmason
On Fri, Jun 29 2018, Mike Hommey wrote: > On Sat, Jun 30, 2018 at 12:10:24AM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> On Fri, Jun 29 2018, Mike Hommey wrote: >> >> > I noticed some slowness when fast-importing data from the Firefox mercurial >> > repository, where fast-import spends more than

Re: fast-import slowness when importing large files with small differences

2018-06-29 Thread Mike Hommey
On Sat, Jun 30, 2018 at 12:10:24AM +0200, Ævar Arnfjörð Bjarmason wrote: > > On Fri, Jun 29 2018, Mike Hommey wrote: > > > I noticed some slowness when fast-importing data from the Firefox mercurial > > repository, where fast-import spends more than 5 minutes importing ~2000 > > revisions of one

Re: fast-import slowness when importing large files with small differences

2018-06-29 Thread Ævar Arnfjörð Bjarmason
On Fri, Jun 29 2018, Mike Hommey wrote: > I noticed some slowness when fast-importing data from the Firefox mercurial > repository, where fast-import spends more than 5 minutes importing ~2000 > revisions of one particular file. I reduced a testcase while still > using real data. One could synth

Re: fast-import slowness when importing large files with small differences

2018-06-29 Thread Stefan Beller
+cc Jun Wu, original author of these patches On Fri, Jun 29, 2018 at 1:39 PM Jeff King wrote: > > Interesting pieces regarding performance: > > > > c420792217c8 xdiff: reduce indent heuristic overhead > > https://phab.mercurial-scm.org/rHGc420792217c89622482005c99e959b9071c109c5 Going by the ma

Re: fast-import slowness when importing large files with small differences

2018-06-29 Thread Jeff King
On Fri, Jun 29, 2018 at 01:14:52PM -0700, Stefan Beller wrote: > Interesting pieces regarding performance: > > c420792217c8 xdiff: reduce indent heuristic overhead > https://phab.mercurial-scm.org/rHGc420792217c89622482005c99e959b9071c109c5 > > f33a87cf60cc xdiff: add a preprocessing step that t

Re: fast-import slowness when importing large files with small differences

2018-06-29 Thread Stefan Beller
On Fri, Jun 29, 2018 at 3:18 AM Mike Hommey wrote: > > Hi, > > I noticed some slowness when fast-importing data from the Firefox mercurial > repository, where fast-import spends more than 5 minutes importing ~2000 > revisions of one particular file. I reduced a testcase while still > using real da