On 13/12/10 17:50, Paul Eggert wrote:
> On 12/13/10 02:09, Pádraig Brady wrote:
>> * Keep allocation to multiples of BUFSIZ which may be more efficient
>
> I don't see how the patch accomplishes this.
> If alloc starts off being BUFSIZ, and the assignment
> "alloc = alloc + alloc / 2" does not kee
On 12/13/10 02:09, Pádraig Brady wrote:
> * Keep allocation to multiples of BUFSIZ which may be more efficient
I don't see how the patch accomplishes this.
If alloc starts off being BUFSIZ, and the assignment
"alloc = alloc + alloc / 2" does not keep the allocation
to a multiple of BUFSIZ. Also,
On 25/10/10 01:26, Pádraig Brady wrote:
> I noticed large realloc()s still being done by fread_file() for the
> common case of a regular file. As well as being inefficient,
> it could needlessly fail for large files.
>
> $ truncate -s1M t.f
> $ ltrace ./a.out < t.f
> malloc(1048577)
On 10/24/2010 05:26 PM, Pádraig Brady wrote:
> On a related note I was wondering if we should fall back
> to increasing the buffer by +=BUFSIZ rather than *=2
> when we get ENOMEM?
I wouldn't bother, as it would lead to O(N**2) behavior,
which could well be worse than the problem that it'd cure.
I noticed large realloc()s still being done by fread_file() for the
common case of a regular file. As well as being inefficient,
it could needlessly fail for large files.
$ truncate -s1M t.f
$ ltrace ./a.out < t.f
__libc_start_main(0x8048624, 1, 0xbfd86c34, 0x8048940, 0x8048930
fileno(0x7b7420)
Eric Blake wrote:
> I see a few changes to squash in:
I see some more changes to squash in:
> +#include
This requires the use of module 'sys_stat'. (See in
doc/posix-headers/sys_stat.texi; we use S_REG.)
> +/* Get SIZE_MAX. */
> +#include
This requires the use of module 'stdint'.
> /* Get
On 08/18/2010 05:43 AM, Giuseppe Scrivano wrote:
PING^2
are there more problems with this patch?
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depends-on): Add ftello and malloc-posix.
* lib/read-file.c: Include,,
,,.
(fread_file): With regula
PING^2
are there more problems with this patch?
Thanks,
Giuseppe
> From da6e41d8ca204903cc088444b882d904db5e649e Mon Sep 17 00:00:00 2001
> From: Giuseppe Scrivano
> Date: Tue, 3 Aug 2010 15:40:19 +0200
> Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
>
> * modules
Ping.
I have rebased the patch.
Cheers,
Giuseppe
>From da6e41d8ca204903cc088444b882d904db5e649e Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depends-on): A
Paul Eggert writes:
> It would be clearer without the casts. (Casts are often
> overkill in C; they disable too much checking.) Also, I'm still
> dubious about going ahead with a file that's too
> large to fit into memory.
Here is another version, it fails with ENOMEM on files that don't fit
i
On 08/03/10 17:00, Giuseppe Scrivano wrote:
> + if (alloc_off - (off_t) (size_t) alloc_off
> + || (size_t) alloc_off + 1 < (size_t) alloc_off)
> +break;
It would be clearer without the casts. (Casts are often
overkill in C; they disable too much checking.) Also, I'm still
Bruno, Paul,
I have followed your suggestions and amended them in the patch.
It rollbacks to the previous code and reads from the stream on
overflows.
Also, as suggested, I have added another check for overflow in the
existing code.
Thanks,
Giuseppe
>From c0f45017bded0d958ec430a54b1fb1b29098
On 08/03/10 15:17, Bruno Haible wrote:
>> > and errno should be set to ENOMEM if overflow occurs.
> I disagree. In this case there's likely something fishy with the fstat
> results, and it's better to start reading from the stream, like for
> non-regular files.
Hmm, well, it could happen when read
Hi Giuseppe,
> + long pos = ftell (stream);
> + if (pos < 0)
> +return NULL;
Not every regular file is seekable (think of files under /proc
on Linux). Therefore, of 'ftello' fails, it's better to not pre-allocate
a buffer but start reading from the stream, like for non-regular f
On 08/03/10 13:05, Giuseppe Scrivano wrote:
> + long pos = ftell (stream);
This isn't reliable on modern 32-bit hosts. Please use
ftello rather than ftell, as ftell is obsolete. You'll
need to alter the module to depend on the ftello module.
> + alloc = st.st_size - pos + 1;
The assi
Hi Paul,
Paul Eggert writes:
> Surely this would be much, much slower if the file seeks slowly,
> for example, if it is a tape drive.
>
> It might be helpful to use fstat to find the file's type and size,
> and to subtract the current file offset from its size
> (for file types where st_size is
On 08/03/10 06:50, Giuseppe Scrivano wrote:
> if ((pos = fseek (stream, 0, SEEK_CUR)) == 0)
Surely this would be much, much slower if the file seeks slowly,
for example, if it is a tape drive.
It might be helpful to use fstat to find the file's type and size,
and to subtract the current file offs
ano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory reallocations with seekable files.
* lib/read-file.c (fread_file): With seekable files, use the file
length as the initial buffer size.
---
ChangeLog |6 ++
lib/read-file.c | 18 ++
18 matches
Mail list logo
