On 30/06/11 00:05, Pádraig Brady wrote: > On 29/06/11 21:59, Bruno Haible wrote: >> Hi, >> >> The glibc 'iconv' program buffers its input, and some people don't like this. >> I thought that the 'stdbuf' program could remove the buffering, but it does >> not >> work. > > The following shows I think that iconv is bypassing stdio and buffering > internally? > > (echo; sleep 3; echo) | ltrace iconv -f ASCII > > The stdbuf man page notes that: > > NOTE: If COMMAND adjusts the buffering of its standard streams (`tee' > does for e.g.) then that will override corresponding settings changed > by `stdbuf'. Also some filters (like `dd' and `cat' etc.) don't use > streams for I/O, and are thus unaffected by `stdbuf' settings.
In fact iconv seems to buffer for ever and so it not scalable, as demonstrated with this consuming all of memory: yes | iconv It would be OK to treat '\n' simply in all unibyte encodings and utf8 for example, but that would introduce an inconsistency I suppose. Though maybe iconv could employ more scalable buffering at least, for all encodings? cheers, Pádraig.
