On Thu, Jan 24, 2019 at 02:18:36PM -0500, Jeff King wrote:
> > I did some benchmarking, using cat as the clean filter:
> > [...]
> > From this, it looks like the file has to be quite large before the
> > preallocation makes a sizable improvement to runtime, and the
> > smudge/clean filters have to
On Thu, Jan 24, 2019 at 02:38:10PM -0400, Joey Hess wrote:
> > Just off the top of my head, something like:
> >
> > /* guess that the filtered output will be the same size as the original */
> > hint = len;
> >
> > /* allocate 10% extra in case the clean size is slightly larger */
> > hi
Jeff King wrote:
> Looking at apply_single_file_filter(), it's not the _original_ file that
> it's trying to store, but rather the data coming back from the filter.
> It's just that we use the original file size as a hint!
Thanks much for working that out!
> In other words, I think this patch fix
Jeff King wrote:
> I didn't experiment with the smudge side, but I think it uses the same
> apply_filter() code. Which means that yes, it would try to store the
> 11GB in memory before writing it out. And I agree writing it out to a
> file and moving it directly into place is the sanest option ther
On Thu, Jan 24, 2019 at 7:11 PM Jeff King wrote:
>
> On Tue, Jan 22, 2019 at 06:07:14PM -0400, Joey Hess wrote:
>
> > joey@darkstar:~/tmp/t> ls -l big-file
> > -rw-r--r-- 1 joey joey 11811160064 Jan 22 17:48 big-file
> > joey@darkstar:~/tmp/t> git status
> > fatal: Out of memory, realloc failed
>
On Thu, Jan 24, 2019 at 12:39:49AM +, brian m. carlson wrote:
> > [pid 6573] mmap(NULL, 11811164160, PROT_READ|PROT_WRITE,
> > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> >
> > Why status needs to mmap a large file that is not modified
> > and that is configured
On Tue, Jan 22, 2019 at 06:07:14PM -0400, Joey Hess wrote:
> joey@darkstar:~/tmp/t> ls -l big-file
> -rw-r--r-- 1 joey joey 11811160064 Jan 22 17:48 big-file
> joey@darkstar:~/tmp/t> git status
> fatal: Out of memory, realloc failed
>
> This file is checked into git, but using a smudge/clean filt
On Tue, Jan 22, 2019 at 06:07:14PM -0400, Joey Hess wrote:
> joey@darkstar:~/tmp/t> ls -l big-file
> -rw-r--r-- 1 joey joey 11811160064 Jan 22 17:48 big-file
> joey@darkstar:~/tmp/t> git status
> fatal: Out of memory, realloc failed
>
> This file is checked into git, but using a smudge/clean filte
8 matches
Mail list logo
