Package: analog
Version: 2:6.0-17~lenny1
Severity: normal
Tags: patch
Multi-stream bz2 files are created if you concatenate two "normal" bz2
files. Analog's current code stops at the end of the first stream. This
happens without error, and simply shows up as an unexplained failure to
read to the end of the log file.
The attached patch checks if the end-of-stream is also the end-of-file,
and if not, it opens the next stream and continues on.
A workaround is to use
UNCOMPRESS *.bz2 "bzcat"
in your configuration file, thus overriding the built-in bzip code.
-- System Information:
Debian Release: 5.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Kernel: Linux 2.6.26-1-686 (SMP w/2 CPU cores)
Locale: LANG=en_IE.UTF-8, LC_CTYPE=en_IE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages analog depends on:
ii debconf [debconf-2.0] 1.5.24 Debian configuration management sy
ii libbz2-1.0 1.0.5-1 high-quality block-sorting file co
ii libc6 2.7-18 GNU C Library: Shared libraries
ii libgd2-xpm 2.0.36~rc1~dfsg-3 GD Graphics Library version 2
ii libjpeg62 6b-14 The Independent JPEG Group's JPEG
ii libpcre3 7.6-2.1 Perl 5 Compatible Regular Expressi
ii libpng12-0 1.2.27-2+lenny2 PNG library - runtime
ii perl 5.10.0-19 Larry Wall's Practical Extraction
ii zlib1g 1:1.2.3.3.dfsg-12 compression library - runtime
analog recommends no packages.
Versions of packages analog suggests:
ii apache2 2.2.9-10+lenny2 Apache HTTP Server metapackage
ii apache2-mpm-prefork [htt 2.2.9-10+lenny2 Apache HTTP Server - traditional n
pn rmagic <none> (no description available)
-- debconf information excluded
--- input.c.old 2004-12-19 13:51:30.000000000 +0000
+++ input.c 2009-04-02 10:07:38.000000000 +0100
@@ -377,6 +377,9 @@
choice getmoredata(Logfile *lf, char *start, size_t length) {
#ifndef LINE_PARSER
int bze;
+ void* unusedPtr;
+ char* unusedData;
+ int unusedCount;
#endif
int n;
@@ -397,9 +400,53 @@
n = 0; /* unzReadCurrentFile() returns -1 on error */
break;
case LF_BZ2:
- n = BZ2_bzRead(&bze, (BZFILE *)(lf->file2), (void *)start, length);
- if (bze != BZ_OK && bze != BZ_STREAM_END)
- n = 0; /* return value from BZ2_bzRead() undefined on error */
+ n = 0;
+ // Multiple BZ2 streams may occur in a single file. If we reach the end of
+ // a stream before we have enough bytes, open the next stream and go again
+ while (n < length) {
+ n = n + BZ2_bzRead(&bze, (BZFILE *)(lf->file2), (void *)start + n, length - n);
+
+ if (bze != BZ_OK && bze != BZ_STREAM_END) {
+ n = 0; /* return value from BZ2_bzRead() undefined on error */
+ break;
+ }
+
+ if (bze == BZ_STREAM_END) {
+ // Is this really the end of the file, or is there another stream?
+ // Check for "unused" data (see the bzip docs for what this is)
+ BZ2_bzReadGetUnused(&bze, (BZFILE *)(lf->file2), &unusedPtr, &unusedCount);
+ if (bze != BZ_OK) {
+ n = 0; // Something's wrong, give up
+ break;
+ }
+
+ if (unusedCount == 0 && feof((FILE *)(lf->file))) {
+ break; // Real end-of-file
+ }
+
+ // Put the unused data somewhere safe for a moment
+ unusedData = malloc(unusedCount);
+ memcpy(unusedData, unusedPtr, unusedCount);
+
+ // Close the old stream
+ BZ2_bzReadClose(&bze, (BZFILE *)(lf->file2));
+ if (bze != BZ_OK) {
+ n = 0; // Something's wrong, give up
+ break;
+ }
+
+ // Open the new stream
+ lf->file2 = (void *)BZ2_bzReadOpen(&bze, (FILE *)(lf->file),
+ 0, 0, unusedData, unusedCount);
+ if (bze != BZ_OK || (BZFILE *)(lf->file2)==NULL) {
+ n = 0; // Something's wrong, give up
+ break;
+ }
+
+ // Unused data is copied immediately by bzReadOpen
+ free(unusedData);
+ }
+ }
break;
#endif
default:
