Re: Reducing iconv-induced memory usage

2011-05-07 Thread Ludovic Courtès
Hello! l...@gnu.org (Ludovic Courtès) writes: > Hello, > > l...@gnu.org (Ludovic Courtès) writes: > >> Here’s an updated patch that strictly checks for ill-formed UTF-8 >> sequences, as Mark pointed out. It passes all the tests I recently >> added to ports.test. > > I committed it, though Mark r

Re: Reducing iconv-induced memory usage

2011-05-06 Thread Ludovic Courtès
Hello, l...@gnu.org (Ludovic Courtès) writes: > Here’s an updated patch that strictly checks for ill-formed UTF-8 > sequences, as Mark pointed out. It passes all the tests I recently > added to ports.test. I committed it, though Mark rightfully noted on IRC a non-conformance issue. I’ve added

Re: Reducing iconv-induced memory usage

2011-05-05 Thread Ludovic Courtès
Hello! Here’s an updated patch that strictly checks for ill-formed UTF-8 sequences, as Mark pointed out. It passes all the tests I recently added to ports.test. I’d like to commit it soon, when Mark approves. :-) Thanks, Ludo’. diff --git a/libguile/ports.c b/libguile/ports.c index b5ad95e..2

Re: Reducing iconv-induced memory usage

2011-04-27 Thread Ludovic Courtès
Hi Mark, Mark H Weaver writes: > l...@gnu.org (Ludovic Courtès) writes: >> So, here’s the patch. >> >> It also makes UTF-8 input ~30% faster according to ports.bm (which >> doesn’t benchmark output): > > Thanks for working on this. I haven't yet had time to fully review this > patch, but here I

Re: Reducing iconv-induced memory usage

2011-04-26 Thread Mark H Weaver
Hi Ludovic! l...@gnu.org (Ludovic Courtès) writes: > So, here’s the patch. > > It also makes UTF-8 input ~30% faster according to ports.bm (which > doesn’t benchmark output): Thanks for working on this. I haven't yet had time to fully review this patch, but here I will document the problems I se

Re: Reducing iconv-induced memory usage

2011-04-26 Thread Ludovic Courtès
Hi! So, here’s the patch. It also makes UTF-8 input ~30% faster according to ports.bm (which doesn’t benchmark output): * before: ("ports.bm: peek-char: latin-1 port" 70 user 0.36) ("ports.bm: peek-char: utf-8 port, ascii character" 70 user 0.35) ("ports.bm: peek-char: utf

Reducing iconv-induced memory usage

2011-04-26 Thread Ludovic Courtès
Hello! As Andy noted in the past, iconv conversion descriptors associated with ports take up a lot of malloc’d memory, that only gets freed when finalizers are run. On GNU/Linux, a UTF-8 → UTF-8 C.D., which does nothing, mallocs 180 KiB (!), according to the program attached. So the problem is a