On Aug 15, 2010, at 12:49 PM, Dimitry Andric wrote: > So my first quick fix attempt was to replace the home-grown grep_fgetln > with fgetln(3), which is in libc. This does not support gzip and bzip2 > files, but just to prove the point, it is enough. It gave the following > profiling result:
FYI: libarchive has some pretty heavily-optimized bulk I/O routines and handles automatic decompression (including gzip, bzip2, lzma, xz, lzip, compress, and soon uuencode). There's a trick supported in libarchive now that will let you just use it's automatic decompression features on non-archive files (via "format_raw"). Unfortunately, it provides binary blocks of data; there's no nice line-reader interface. There's an effort afoot to refactor libarchive so that the stream I/O and compression/decompression support is actually a separate library that should be very useful for this sort of usage. As part of that, we plan to add some line-oriented I/O features that should be noticeably more efficient than stdio. Cheers, Tim _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
