Hi all, first off, many thanks for your efforts here, Paul!
On 06.06.19 15:54, Paul Gevers wrote: > On 06-06-2019 04:23, Marc Dequènes (duck) wrote: >> I'm not committing to this plan for the above stated reasons. I also >> feels uncomfortable uploading with a know security problem, so unless >> upstream or our security team says it's low risk, I'm not taking such >> responsibility. > > Sorry, I am mising something here. Can you please point me to it again? AFAICT duck had this in his original unblock request: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923885#5: > Please unblock package libmspack. 0.9.1-1 should have made it > but somehow the build failed on big-endian systems (see #914794), > maybe gcc changes in the meanwhile; anyway upstream kindly fixed > it but it took some time. > > 0.10.1 is a maintenance release with only few fixes, among which > fixing chmd_read_headers() closed a security problem (see upstream > ChangeLog in debdiff). There is also a simple one-liner fix for > the doc (compared to 0.9.1-1, which had ample time to be tested). > I would also note that 0.9.1-1 fixed a regression (see #912687). > Thus I believe 0.10.1-1 is a much better fit for release and hope > you would conclude the same. The referenced ChangeLog was: 2019-02-18 Stuart Caie <ky...@cabextract.org.uk>: > chmd_read_headers(): CHM files can declare their chunks are any > size up to 4GB, and libmspack will attempt to allocate that to > read the file. > > This is not a security issue; libmspack doesn't promise how > much memory it'll use to unpack files. You can set your own > limits by returning NULL in a custom mspack_system.alloc() > implementation. > > However, it would be good to validate chunk size further. With > no offical specification, only empirical data is available. All > files created by hhc.exe have a chunk size of 4096 bytes, and > this is matched by all the files I've found in the wild, except > for one which has a chunk size of 8192 bytes, which was created > by someone developing a CHM file creator 15 years ago, and they > appear to have abandoned it, so it seems 4096 is a de-facto > standard. > > I've changed the "chunk size is not a power of two" warning to > "chunk size is not 4096", and now only allow chunk sizes between > 22 and 8192 bytes. If you have CHM files with a larger chunk > size, please send them to me and I'll increase this upper limit. > > Thanks to ADLab of Venustech for the report. On 06.06.19 15:54, Paul Gevers wrote: > On 06-06-2019 04:23, Marc Dequènes (duck) wrote: >> On 2019-06-04 03:53, Paul Gevers wrote: >> Currently the current version has been sitting in unstable for three >> months without any single bug reported, this feels like a good progress >> towards saying this version is safe. > > It's a valid argument for sure. And the previous 0.9.1-1 is even 7 months old now, having been blocked only by the big endian issue. Personally I doubt an in-depth code review is really helpful here. Anyway, I'm attaching 2 new debdiffs: Upstream did a "tab to spaces". A "debdiff --ignore-space" reduces the diff from 11176 to 4819 lines. I think this is also a point for 0.10.1-1 in buster, because it will make eventual security updates easier. I further removed the diff of generated, examples and test files (I marked those files in the diffstat). Yay, the diff had 678 lines improving the tests! I did this both for each 0.8-1 (in the archive since 2018-10-24) and 0.9.1-1 (2018-11-06) compared to 0.10.1-1 (2019-03-05). So if at all, I'd suggest to only have a look at the last one of these diffs: $ wc -l debdiff_libmspack_0.* 11176 debdiff_libmspack_0.8-1_0.10.1-1.diff 4819 debdiff_libmspack_0.8-1_0.10.1-1_ignore-space.diff 1724 debdiff_libmspack_0.8-1_0.10.1-1_ignore-space_edited.diff 1067 debdiff_libmspack_0.9.1-1_0.10.1-1_ignore-space.diff 731 debdiff_libmspack_0.9.1-1_0.10.1-1_ignore-space_edited.diff Thanks again and greets jre
diffstat for libmspack-0.8 libmspack-0.10.1 ChangeLog | 107 + Makefile.am | 66 -Makefile.in | 737 +++++++--- README | 17 acinclude.m4 | 12 config.h.in | 27 -configure | 402 +++-- configure.ac | 20 debian/changelog | 18 debian/control | 2 debian/copyright | 2 debian/libmspack-doc.docs | 12 debian/rules | 2 doc/.gitignore | 1 -doc/Doxyfile | 17 doc/Doxyfile.in | 22 -doc/Makefile | 16 doc/Makefile.in | 14 -examples/cabrip.c | 85 + -examples/chmextract.c | 121 + -examples/msexpand.c | 48 -examples/oabextract.c | 41 -libmscabd.la (Generated by libtool) | 41 -libmschmd.la (Generated by libtool) | 41 -libmspack.la (Generated by libtool) | 41 mspack/cab.h | 6 mspack/cabd.c | 67 mspack/chmd.c | 60 mspack/mspack.h | 31 mspack/oab.h | 1 mspack/oabd.c | 85 - mspack/system.c | 5 mspack/system.h | 54 mspack/szddd.c | 4 src/cabrip.c | 85 - src/chmextract.c | 122 - src/error.h | 22 src/msexpand.c | 48 src/oabextract.c | 41 -test-driver | 148 ++ -test/cabd_test.c | 297 ++-- -test/chmd_test.c | 66 -test/chminfo.c | 7 -test/kwajd_test.c | 144 - -test/md5.c | 7 -test/md5.h | 9 -test/test_files/cabd/mszip_lzx_qtm.cab |binary -test/test_files/cabd/normal_2files_2folders.cab |binary -test/test_files/chmd/cve-2015-4467-reset-interval-zero.chm.LZXC-is-lzxc |binary -test/test_files/chmd/cve-2015-4467-reset-interval-zero.chm.xor |binary -test/test_files/chmd/short-system-filenames.chm |binary 51 files changed, 1995 insertions(+), 1226 deletions(-) diff -Nru -w libmspack-0.8/acinclude.m4 libmspack-0.10.1/acinclude.m4 --- libmspack-0.8/acinclude.m4 2017-08-13 21:38:18.000000000 +0200 +++ libmspack-0.10.1/acinclude.m4 2018-10-30 11:56:24.000000000 +0100 @@ -1,5 +1,5 @@ # =========================================================================== -# http://www.gnu.org/software/autoconf-archive/ax_func_mkdir.html +# https://www.gnu.org/software/autoconf-archive/ax_func_mkdir.html # =========================================================================== # # SYNOPSIS @@ -43,7 +43,7 @@ # Public License for more details. # # You should have received a copy of the GNU General Public License along -# with this program. If not, see <http://www.gnu.org/licenses/>. +# with this program. If not, see <https://www.gnu.org/licenses/>. # # As a special exception, the respective Autoconf Macro's copyright owner # gives unlimited permission to copy, distribute and modify the configure @@ -58,19 +58,19 @@ # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. -#serial 4 +#serial 6 AU_ALIAS([AC_FUNC_MKDIR], [AX_FUNC_MKDIR]) AC_DEFUN([AX_FUNC_MKDIR], [AC_CHECK_FUNCS([mkdir _mkdir]) AC_CACHE_CHECK([whether mkdir takes one argument], [ac_cv_mkdir_takes_one_arg], -[AC_COMPILE_IFELSE([AC_LANG_PROGRAM([ +[AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #include <sys/stat.h> #if HAVE_UNISTD_H # include <unistd.h> #endif -], [mkdir (".");])], +]], [[mkdir (".");]])], [ac_cv_mkdir_takes_one_arg=yes], [ac_cv_mkdir_takes_one_arg=no])]) if test x"$ac_cv_mkdir_takes_one_arg" = xyes; then AC_DEFINE([MKDIR_TAKES_ONE_ARG], 1, @@ -91,7 +91,7 @@ dnl | dnl |Alexandre: dnl | Would it be sufficient to check for these headers and #include -dnl | them in the AC_TRY_COMPILE block? (and is AC_HEADER_DIRENT +dnl | them in the AC_COMPILE_IFELSE block? (and is AC_HEADER_DIRENT dnl | suitable for this?) dnl | dnl |Thomas: diff -Nru -w libmspack-0.8/ChangeLog libmspack-0.10.1/ChangeLog --- libmspack-0.8/ChangeLog 2018-10-20 20:04:29.000000000 +0200 +++ libmspack-0.10.1/ChangeLog 2019-02-18 21:01:59.000000000 +0100 @@ -1,3 +1,110 @@ +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * chmd_read_headers(): a CHM file name beginning "::" but shorter + than 33 bytes will lead to reading past the freshly-allocated name + buffer - checks for specific control filenames didn't take length + into account. Thanks to ADLab of Venustech for the report and + proof of concept. + +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * chmd_read_headers(): CHM files can declare their chunks are any + size up to 4GB, and libmspack will attempt to allocate that to + read the file. + + This is not a security issue; libmspack doesn't promise how much + memory it'll use to unpack files. You can set your own limits by + returning NULL in a custom mspack_system.alloc() implementation. + + However, it would be good to validate chunk size further. With no + offical specification, only empirical data is available. All files + created by hhc.exe have a chunk size of 4096 bytes, and this is + matched by all the files I've found in the wild, except for one + which has a chunk size of 8192 bytes, which was created by someone + developing a CHM file creator 15 years ago, and they appear to + have abandoned it, so it seems 4096 is a de-facto standard. + + I've changed the "chunk size is not a power of two" warning to + "chunk size is not 4096", and now only allow chunk sizes between + 22 and 8192 bytes. If you have CHM files with a larger chunk size, + please send them to me and I'll increase this upper limit. + + Thanks to ADLab of Venustech for the report. + +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * oabd.c: replaced one-shot copying of uncompressed blocks (which + requires allocating a buffer of the size declared in the header, + which can be 4GB) with a fixed-size buffer. The buffer size is + user-controllable with the new msoab_decompressor::set_param() + method (check you have version 2 of the OAB decompressor), and + also controls the input buffer used for OAB's LZX decompression. + + Reminder: compression formats can dictate how much memory is + needed to decompress them. If memory usage is a security concern + to you, write a custom mspack_system.alloc() that returns NULL + if "too much" memory is requested. Do not rely on libmspack adding + special heuristics to know not to request "too much". + + Thanks to ADLab of Venustech for the report. + +2018-11-03 Stuart Caie <ky...@cabextract.org.uk> + + * configure.ac, doc/Makefile.in, doc/Doxyfile.in: remove these + template files and replace with static files. You can still build + the documentation with make -C doc + +2018-11-03 Stuart Caie <ky...@cabextract.org.uk> + + * Makefile.am, src: move the "useful" programs in src/ to examples/ + and don't auto-install them. Even though they're useful, they are + intended as examples and aren't productised (no commmand-line + options, no man pages, etc.) -- if you disagree, feel free to + send in a patch + +2018-11-01 Stuart Caie <ky...@cabextract.org.uk> + + * cabd_extract(): would not do decompression for random-access + offsets if the folder type was LZX. This is a fairly major bug, + and affects any decompression where you skip directly to a file, + or decompress data out-of-order. Thanks to austin987 for alerting + me to this. + + This bug was introduced by the recent 'salvage mode' patch. Even + though I'd reviewed all the differences in clamav's copy of + libmspack and said "wtf" to this particular change, I didn't + notice it was still in the resulting patch I merged. Mea culpa :) + + * test/cabd_test.c: now has a regression test to cover this + +2018-10-31 Stuart Caie <ky...@cabextract.org.uk> + + * Makefile.am, test/*_test.c: use the automake test-suite system + with the test-suite programs (cabd_test, chmd_test, kwajd_test). + This also fixes a longstanding bugbear that these programs don't + access their test files using an absolute path. Now this is passed + to them and you can run them from any directory. Thanks to Richard + Jones for requesting this. + +2018-10-31 Stuart Caie <ky...@cabextract.org.uk> + + * configure.ac: require at least automake 1.11, use AM_SILENT_RULES + unconditionally + +2018-10-30 Stuart Caie <ky...@cabextract.org.uk> + + * configure.ac: remove obsolescent C library tests. AC_HEADER_STDC is + removed, and so are most checks for standard C headers. libmspack now + makes these assumptions: + - <ctype.h> <limits.h> <stdlib.h> <string.h> exist + - <ctype.h> defines tolower() + - <string.h> defines memset(), memcmp(), strlen() + - if towlower() exists, it's defined in <wctype.h> + +2018-10-22 Stuart Caie <ky...@cabextract.org.uk> + + * cabd.c: remove the only use of assert() + 2018-10-20 Stuart Caie <ky...@cabextract.org.uk> * src/chmextract.c: add anti "../" and leading slash protection to diff -Nru -w libmspack-0.8/config.h.in libmspack-0.10.1/config.h.in --- libmspack-0.8/config.h.in 2018-10-21 17:51:13.000000000 +0200 +++ libmspack-0.10.1/config.h.in 2019-03-04 10:39:16.000000000 +0100 @@ -1,11 +1,11 @@ /* config.h.in. Generated from configure.ac by autoheader. */ +/* Define if building universal (internal helper macro) */ +#undef AC_APPLE_UNIVERSAL_BUILD + /* Turn debugging mode on? */ #undef DEBUG -/* Define to 1 if you have the <ctype.h> header file. */ -#undef HAVE_CTYPE_H - /* Define to 1 if you have the <dlfcn.h> header file. */ #undef HAVE_DLFCN_H @@ -15,9 +15,6 @@ /* Define to 1 if you have the <inttypes.h> header file. */ #undef HAVE_INTTYPES_H -/* Define to 1 if you have the <limits.h> header file. */ -#undef HAVE_LIMITS_H - /* Define to 1 if you have the <memory.h> header file. */ #undef HAVE_MEMORY_H @@ -42,18 +39,12 @@ /* Define to 1 if you have the <sys/types.h> header file. */ #undef HAVE_SYS_TYPES_H -/* Define to 1 if you have the `tolower' function. */ -#undef HAVE_TOLOWER - /* Define to 1 if you have the `towlower' function. */ #undef HAVE_TOWLOWER /* Define to 1 if you have the <unistd.h> header file. */ #undef HAVE_UNISTD_H -/* Define to 1 if you have the <wctype.h> header file. */ -#undef HAVE_WCTYPE_H - /* Define to 1 if you have the `_mkdir' function. */ #undef HAVE__MKDIR @@ -93,6 +84,18 @@ /* Version number of package */ #undef VERSION +/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most + significant byte first (like Motorola and SPARC, unlike Intel). */ +#if defined AC_APPLE_UNIVERSAL_BUILD +# if defined __BIG_ENDIAN__ +# define WORDS_BIGENDIAN 1 +# endif +#else +# ifndef WORDS_BIGENDIAN +# undef WORDS_BIGENDIAN +# endif +#endif + /* Enable large inode numbers on Mac OS X 10.5. */ #ifndef _DARWIN_USE_64_BIT_INODE # define _DARWIN_USE_64_BIT_INODE 1 diff -Nru -w libmspack-0.8/configure.ac libmspack-0.10.1/configure.ac --- libmspack-0.8/configure.ac 2018-10-21 17:49:47.000000000 +0200 +++ libmspack-0.10.1/configure.ac 2019-03-04 10:36:51.000000000 +0100 @@ -1,16 +1,13 @@ # -*- Autoconf -*- # Process this file with autoconf to produce a configure script. AC_PREREQ(2.59) -AC_INIT([libmspack],[0.8alpha],[ky...@cabextract.org.uk]) +AC_INIT([libmspack],[0.10.1alpha],[ky...@cabextract.org.uk]) AC_CONFIG_MACRO_DIR([m4]) -AM_INIT_AUTOMAKE +AM_INIT_AUTOMAKE([1.11]) +AM_SILENT_RULES([yes]) AC_CONFIG_SRCDIR([mspack/mspack.h]) AC_CONFIG_HEADER([config.h]) -dnl Enable silent rules by default (if yet support in automake) -dnl use 'make V=1' to look at verbose commandline -m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) - # --enable-debug option AC_ARG_ENABLE(debug, AS_HELP_STRING(--enable-debug,enable debugging), @@ -19,7 +16,6 @@ if test x$enable_debug = xyes; then AC_DEFINE(DEBUG, 1, [Turn debugging mode on?]) fi -AM_CONDITIONAL(DEBUG, test x$enable_debug = 'xyes') # Checks for programs. AC_PROG_CC @@ -27,15 +23,14 @@ AM_PROG_AR AC_PROG_INSTALL LT_INIT -AC_EXEEXT # Checks for header files. -AC_HEADER_STDC -AC_CHECK_HEADERS([ctype.h inttypes.h limits.h stdlib.h wctype.h]) +AC_CHECK_HEADERS([inttypes.h]) # Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_C_INLINE +AC_C_BIGENDIAN AC_TYPE_MODE_T AC_TYPE_OFF_T AC_TYPE_SIZE_T @@ -43,12 +38,11 @@ # Checks for library functions AX_FUNC_MKDIR -AC_CHECK_FUNCS([tolower towlower]) +AC_CHECK_FUNCS([towlower]) # largefile support AC_SYS_LARGEFILE AC_FUNC_FSEEKO -# Checks for library functions. -AC_CONFIG_FILES([Makefile doc/Makefile doc/Doxyfile libmspack.pc]) +AC_CONFIG_FILES([Makefile libmspack.pc]) AC_OUTPUT diff -Nru -w libmspack-0.8/debian/changelog libmspack-0.10.1/debian/changelog --- libmspack-0.8/debian/changelog 2018-10-24 03:03:13.000000000 +0200 +++ libmspack-0.10.1/debian/changelog 2019-03-05 11:03:29.000000000 +0100 @@ -1,3 +1,21 @@ +libmspack (0.10.1-1) unstable; urgency=medium + + * New upstream release: + + fix build on big-endian systems (Closes: #914794) + * Add missing JS files for documentation menu and search functions. + + -- Marc Dequènes (Duck) <d...@duckcorp.org> Tue, 05 Mar 2019 19:03:29 +0900 + +libmspack (0.9.1-1) unstable; urgency=medium + + * New upstream release: + + fix regression when extracting cabinets using -F option + (Closes: #912687) + * Bump Standards-Version to 4.2.1. + * Adapt to documentation now generated in 'doc/html'. + + -- Marc Dequènes (Duck) <d...@duckcorp.org> Tue, 06 Nov 2018 22:38:49 +0900 + libmspack (0.8-1) unstable; urgency=medium * New upstream release: diff -Nru -w libmspack-0.8/debian/control libmspack-0.10.1/debian/control --- libmspack-0.8/debian/control 2018-04-12 12:20:00.000000000 +0200 +++ libmspack-0.10.1/debian/control 2019-03-05 07:24:16.000000000 +0100 @@ -2,7 +2,7 @@ Section: libs Priority: optional Maintainer: Marc Dequènes (Duck) <d...@duckcorp.org> -Standards-Version: 4.1.4 +Standards-Version: 4.2.1 Build-Depends: dpkg-dev (>= 1.16.1.1), debhelper (>= 11) Build-Depends-indep: doxygen, graphviz Vcs-Browser: https://salsa.debian.org/debian/libmspack diff -Nru -w libmspack-0.8/debian/copyright libmspack-0.10.1/debian/copyright --- libmspack-0.8/debian/copyright 2018-04-05 05:16:07.000000000 +0200 +++ libmspack-0.10.1/debian/copyright 2019-03-05 07:24:35.000000000 +0100 @@ -2,6 +2,8 @@ Upstream-Name: libmspack Upstream-Contact: Stuart Caie <ky...@4u.net> Source: https://www.cabextract.org.uk/libmspack/ +# doxygen-generated doc which is embedded in the upstream tarball; regenerated in the Debian build +Files-excluded: doc/html Files: * diff -Nru -w libmspack-0.8/debian/libmspack-doc.docs libmspack-0.10.1/debian/libmspack-doc.docs --- libmspack-0.8/debian/libmspack-doc.docs 2018-04-05 05:16:07.000000000 +0200 +++ libmspack-0.10.1/debian/libmspack-doc.docs 2019-03-05 10:52:56.000000000 +0100 @@ -1,4 +1,8 @@ -doc/*.html -doc/*.css -doc/*.png -doc/search +doc/html/*.html +doc/html/*.css +doc/html/*.png +# jquery.js is generated by doxygen +# there is no easy way to replace it with a symlink to a system common version +# rationale: /usr/share/doc/doxygen/README.jquery +doc/html/*.js +doc/html/search diff -Nru -w libmspack-0.8/debian/rules libmspack-0.10.1/debian/rules --- libmspack-0.8/debian/rules 2018-04-12 12:24:08.000000000 +0200 +++ libmspack-0.10.1/debian/rules 2019-03-05 10:50:48.000000000 +0100 @@ -15,8 +15,6 @@ override_dh_auto_clean: [ -f doc/Makefile ] && $(MAKE) -C doc clean - # not enough - rm -rf doc/menu.js doc/menudata.js dh_auto_clean override_dh_strip: diff -Nru -w libmspack-0.8/doc/Doxyfile.in libmspack-0.10.1/doc/Doxyfile.in --- libmspack-0.8/doc/Doxyfile.in 2015-06-05 10:10:54.000000000 +0200 +++ libmspack-0.10.1/doc/Doxyfile.in 1970-01-01 01:00:00.000000000 +0100 @@ -1,22 +0,0 @@ -PROJECT_NAME = libmspack -OUTPUT_DIRECTORY = . -EXTRACT_ALL = YES -EXTRACT_LOCAL_CLASSES = YES -HIDE_UNDOC_MEMBERS = YES -SHOW_INCLUDE_FILES = YES -JAVADOC_AUTOBRIEF = YES -OPTIMIZE_OUTPUT_FOR_C = YES -SHOW_USED_FILES = YES -INPUT = @top_srcdir@/mspack/mspack.h -FULL_PATH_NAMES = NO -GENERATE_HTML = YES -HTML_OUTPUT = . -HTML_FILE_EXTENSION = .html -HTML_TIMESTAMP = NO -GENERATE_HTMLHELP = NO -GENERATE_LATEX = NO -GENERATE_RTF = NO -GENERATE_MAN = NO -GENERATE_XML = NO -GENERATE_AUTOGEN_DEF = NO -CLASS_DIAGRAMS = NO diff -Nru -w libmspack-0.8/doc/.gitignore libmspack-0.10.1/doc/.gitignore --- libmspack-0.8/doc/.gitignore 1970-01-01 01:00:00.000000000 +0100 +++ libmspack-0.10.1/doc/.gitignore 2018-11-04 00:53:53.000000000 +0100 @@ -0,0 +1,1 @@ +html diff -Nru -w libmspack-0.8/doc/Makefile.in libmspack-0.10.1/doc/Makefile.in --- libmspack-0.8/doc/Makefile.in 2017-11-26 12:58:55.000000000 +0100 +++ libmspack-0.10.1/doc/Makefile.in 1970-01-01 01:00:00.000000000 +0100 @@ -1,14 +0,0 @@ -DOCS = annotated.html classes.html dir_*.html doxygen.* dynsections.js \ - files.html functions*.html globals*.html graph_legend.html index.html \ - jquery.js mspack*.html search struct*.html tab* *.map *.md5 *.png - -all: index.html - -clean: - -rm -rf $(DOCS) installdox - -index.html: @top_srcdir@/mspack/mspack.h Doxyfile - doxygen - -mspack_docs.zip: index.html - zip -9r $@ $(DOCS) szdd_kwaj_format.html diff -Nru -w libmspack-0.8/Makefile.am libmspack-0.10.1/Makefile.am --- libmspack-0.8/Makefile.am 2018-10-16 12:07:28.000000000 +0200 +++ libmspack-0.10.1/Makefile.am 2018-11-06 12:14:30.000000000 +0100 @@ -1,30 +1,26 @@ AUTOMAKE_OPTIONS = subdir-objects -EXTRA_DIST = $(srcdir)/debian/* $(srcdir)/doc/* $(srcdir)/test/test_files/*/* +EXTRA_DIST = doc test/test_files pkgconfigdir = $(libdir)/pkgconfig -pkgconfig_DATA = libmspack.pc - -dist-hook: - -rm -f $(distdir)/*.la +TESTS = $(check_PROGRAMS) ACLOCAL_AMFLAGS = -I m4 AM_CFLAGS = # add "-DMSPACK_NO_DEFAULT_SYSTEM" to remove default mspack_system -if DEBUG -AM_CFLAGS += -DDEBUG -endif if GCC AM_CFLAGS += -Wall -Wextra -Wno-unused-parameter -Wno-unused-result endif -AM_CPPFLAGS = -I$(top_srcdir)/mspack -I$(top_srcdir)/test +AM_CPPFLAGS = -I$(srcdir)/mspack -I$(srcdir)/test -bin_PROGRAMS = src/cabrip src/chmextract src/msexpand src/oabextract include_HEADERS = mspack/mspack.h lib_LTLIBRARIES = libmspack.la +pkgconfig_DATA = libmspack.pc noinst_LTLIBRARIES = libmscabd.la libmschmd.la -noinst_PROGRAMS = examples/cabd_memory examples/multifh test/cabd_md5 \ - test/cabd_test test/chmd_find test/chmd_md5 \ - test/chmd_order test/chmd_test test/chminfo test/kwajd_test +noinst_PROGRAMS = examples/cabd_memory examples/cabrip examples/chmextract \ + examples/msexpand examples/multifh examples/oabextract \ + test/cabd_md5 test/chmd_find test/chmd_md5 test/chmd_order \ + test/chminfo +check_PROGRAMS = test/cabd_test test/chmd_test test/kwajd_test libmspack_la_SOURCES = mspack/mspack.h \ mspack/system.h mspack/system.c \ @@ -42,7 +38,7 @@ mspack/lzss.h mspack/lzssd.c \ mspack/des.h mspack/sha.h \ mspack/crc32.c mspack/crc32.h -libmspack_la_LDFLAGS = -export-symbols-regex '^mspack_' -version-info 1:0:1 +libmspack_la_LDFLAGS = -export-symbols-regex '^mspack_' -version-info 1:0:1 -no-undefined libmscabd_la_SOURCES = mspack/mspack.h \ mspack/system.h mspack/system.c \ @@ -62,38 +58,34 @@ examples_cabd_memory_SOURCES = examples/cabd_memory.c libmscabd.la examples_cabd_memory_LDADD = libmscabd.la +examples_cabrip_SOURCES = examples/cabrip.c libmspack.la +examples_cabrip_LDADD = libmspack.la +examples_chmextract_SOURCES = examples/chmextract.c test/error.h libmspack.la +examples_chmextract_LDADD = libmspack.la +examples_msexpand_SOURCES = examples/msexpand.c test/error.h libmspack.la +examples_msexpand_LDADD = libmspack.la examples_multifh_SOURCES = examples/multifh.c libmscabd.la examples_multifh_LDADD = libmscabd.la +examples_oabextract_SOURCES = examples/oabextract.c test/error.h libmspack.la +examples_oabextract_LDADD = libmspack.la -src_cabrip_SOURCES = src/cabrip.c libmspack.la -src_cabrip_LDADD = libmspack.la -src_chmextract_SOURCES = src/chmextract.c src/error.h libmspack.la -src_chmextract_LDADD = libmspack.la -src_msexpand_SOURCES = src/msexpand.c src/error.h libmspack.la -src_msexpand_LDADD = libmspack.la -src_oabextract_SOURCES = src/oabextract.c src/error.h libmspack.la -src_oabextract_LDADD = libmspack.la - -test_cabd_md5_SOURCES = test/cabd_md5.c test/md5.c test/md5.h \ - test/md5_fh.h test/error.h libmscabd.la +test_cabd_md5_SOURCES = test/cabd_md5.c test/md5.c test/md5.h test/md5_fh.h test/error.h libmscabd.la test_cabd_md5_LDADD = libmscabd.la -test_cabd_test_SOURCES= test/cabd_test.c libmscabd.la -test_cabd_test_LDADD = libmscabd.la test_chmd_find_SOURCES = test/chmd_find.c test/error.h libmschmd.la test_chmd_find_LDADD = libmschmd.la -test_chmd_md5_SOURCES = test/chmd_md5.c test/md5.c test/md5.h \ - test/md5_fh.h test/error.h libmschmd.la +test_chmd_md5_SOURCES = test/chmd_md5.c test/md5.c test/md5.h test/md5_fh.h test/error.h libmschmd.la test_chmd_md5_LDADD = libmschmd.la -test_chmd_order_SOURCES = test/chmd_order.c test/md5.c test/md5.h \ - test/md5_fh.h test/error.h libmschmd.la +test_chmd_order_SOURCES = test/chmd_order.c test/md5.c test/md5.h test/md5_fh.h test/error.h libmschmd.la test_chmd_order_LDADD = libmschmd.la -test_chmd_test_SOURCES = test/chmd_test.c libmschmd.la -test_chmd_test_LDADD = libmschmd.la -test_chmd_test_DEPENDENCIES = libmschmd.la test/test_files/chmd/cve-2015-4467-reset-interval-zero.chm test_chminfo_SOURCES = test/chminfo.c libmschmd.la test_chminfo_LDADD = libmschmd.la + +test_cabd_test_SOURCES = test/cabd_test.c test/md5.c test/md5.h test/md5_fh.h libmscabd.la +test_cabd_test_CPPFLAGS = $(AM_CPPFLAGS) -DTEST_FILES=$(abs_srcdir)/test/test_files/cabd +test_cabd_test_LDADD = libmscabd.la +test_chmd_test_SOURCES = test/chmd_test.c test/md5.c test/md5.h test/md5_fh.h libmschmd.la +test_chmd_test_CPPFLAGS = $(AM_CPPFLAGS) -DTEST_FILES=$(abs_srcdir)/test/test_files/chmd +test_chmd_test_LDADD = libmschmd.la test_kwajd_test_SOURCES = test/kwajd_test.c libmspack.la +test_kwajd_test_CPPFLAGS = $(AM_CPPFLAGS) -DTEST_FILES=$(abs_srcdir)/test/test_files/kwajd test_kwajd_test_LDADD = libmspack.la - -test/test_files/chmd/cve-2015-4467-reset-interval-zero.chm: test/test_files/chmd/cve-2015-4467-reset-interval-zero.chm.LZXC-is-lzxc - sed s/lzxc/LZXC/ $< >$@ diff -Nru -w libmspack-0.8/mspack/cabd.c libmspack-0.10.1/mspack/cabd.c --- libmspack-0.8/mspack/cabd.c 2018-10-21 17:45:07.000000000 +0200 +++ libmspack-0.10.1/mspack/cabd.c 2019-03-04 02:09:26.000000000 +0100 @@ -23,7 +23,9 @@ #include <system.h> #include <cab.h> -#include <assert.h> +#include <mszip.h> +#include <lzx.h> +#include <qtm.h> /* Notes on compliance with cabinet specification: * @@ -154,10 +156,10 @@ self->d = NULL; self->error = MSPACK_ERR_OK; - self->param[MSCABD_PARAM_SEARCHBUF] = 32768; - self->param[MSCABD_PARAM_FIXMSZIP] = 0; - self->param[MSCABD_PARAM_DECOMPBUF] = 4096; - self->param[MSCABD_PARAM_SALVAGE] = 0; + self->searchbuf_size = 32768; + self->fix_mszip = 0; + self->buf_size = 4096; + self->salvage = 0; } return (struct mscab_decompressor *) self; } @@ -201,7 +203,7 @@ if ((fh = sys->open(sys, filename, MSPACK_SYS_OPEN_READ))) { if ((cab = (struct mscabd_cabinet_p *) sys->alloc(sys, sizeof(struct mscabd_cabinet_p)))) { cab->base.filename = filename; - error = cabd_read_headers(sys, fh, cab, (off_t) 0, self->param[MSCABD_PARAM_SALVAGE], 0); + error = cabd_read_headers(sys, fh, cab, (off_t) 0, self->salvage, 0); if (error) { cabd_close(base, (struct mscabd_cabinet *) cab); cab = NULL; @@ -598,7 +600,7 @@ sys = self->system; /* allocate a search buffer */ - search_buf = (unsigned char *) sys->alloc(sys, (size_t) self->param[MSCABD_PARAM_SEARCHBUF]); + search_buf = (unsigned char *) sys->alloc(sys, (size_t) self->searchbuf_size); if (!search_buf) { self->error = MSPACK_ERR_NOMEMORY; return NULL; @@ -647,7 +649,7 @@ struct mspack_system *sys = self->system; unsigned char *p, *pend, state = 0; unsigned int cablen_u32 = 0, foffset_u32 = 0; - int false_cabs = 0, salvage = self->param[MSCABD_PARAM_SALVAGE]; + int false_cabs = 0; #if !LARGEFILE_SUPPORT /* detect 32-bit off_t overflow */ @@ -662,8 +664,8 @@ /* search length is either the full length of the search buffer, or the * amount of data remaining to the end of the file, whichever is less. */ length = flen - offset; - if (length > self->param[MSCABD_PARAM_SEARCHBUF]) { - length = self->param[MSCABD_PARAM_SEARCHBUF]; + if (length > self->searchbuf_size) { + length = self->searchbuf_size; } /* fill the search buffer with data from disk */ @@ -673,9 +675,8 @@ /* FAQ avoidance strategy */ if ((offset == 0) && (EndGetI32(&buf[0]) == 0x28635349)) { - sys->message(fh, "WARNING; found InstallShield header. " - "This is probably an InstallShield file. " - "Use UNSHIELD from www.synce.org to unpack it."); + sys->message(fh, "WARNING; found InstallShield header. Use unshield " + "(https://github.com/twogood/unshield) to unpack this file"); } /* read through the entire buffer. */ @@ -728,14 +729,14 @@ * mode, don't check the alleged length, allow it to be garbage */ if ((foffset_u32 < cablen_u32) && ((caboff + (off_t) foffset_u32) < (flen + 32)) && - (((caboff + (off_t) cablen_u32) < (flen + 32)) || salvage)) + (((caboff + (off_t) cablen_u32) < (flen + 32)) || self->salvage)) { /* likely cabinet found -- try reading it */ if (!(cab = (struct mscabd_cabinet_p *) sys->alloc(sys, sizeof(struct mscabd_cabinet_p)))) { return MSPACK_ERR_NOMEMORY; } cab->base.filename = filename; - if (cabd_read_headers(sys, fh, cab, caboff, salvage, 1)) { + if (cabd_read_headers(sys, fh, cab, caboff, self->salvage, 1)) { /* destroy the failed cabinet */ cabd_close((struct mscab_decompressor *) self, (struct mscabd_cabinet *) cab); @@ -1029,7 +1030,7 @@ */ filelen = file->length; if (filelen > CAB_LENGTHMAX || (file->offset + filelen) > CAB_LENGTHMAX) { - if (self->param[MSCABD_PARAM_SALVAGE]) { + if (self->salvage) { filelen = CAB_LENGTHMAX - file->offset; } else { @@ -1047,7 +1048,7 @@ /* if file goes beyond what can be decoded, given an error. * In salvage mode, don't assume block sizes, just try decoding */ - if (!self->param[MSCABD_PARAM_SALVAGE]) { + if (!self->salvage) { off_t maxlen = fol->base.num_blocks * CAB_BLOCKMAX; if ((file->offset + filelen) > maxlen) { sys->message(NULL, "ERROR; file \"%s\" cannot be extracted, " @@ -1125,12 +1126,10 @@ * and pass back MSPACK_ERR_READ */ self->d->outfh = NULL; - if ((self->d->comp_type & cffoldCOMPTYPE_MASK) != cffoldCOMPTYPE_LZX) { if ((bytes = file->offset - self->d->offset)) { error = self->d->decompress(self->d->state, bytes); self->error = (error == MSPACK_ERR_READ) ? self->read_error : error; } - } /* if getting to the correct offset was error free, unpack file */ if (!self->error) { @@ -1161,31 +1160,27 @@ { struct mspack_file *fh = (struct mspack_file *) self; - assert(self && self->d); - self->d->comp_type = ct; switch (ct & cffoldCOMPTYPE_MASK) { case cffoldCOMPTYPE_NONE: self->d->decompress = (int (*)(void *, off_t)) &noned_decompress; - self->d->state = noned_init(&self->d->sys, fh, fh, - self->param[MSCABD_PARAM_DECOMPBUF]); + self->d->state = noned_init(&self->d->sys, fh, fh, self->buf_size); break; case cffoldCOMPTYPE_MSZIP: self->d->decompress = (int (*)(void *, off_t)) &mszipd_decompress; - self->d->state = mszipd_init(&self->d->sys, fh, fh, - self->param[MSCABD_PARAM_DECOMPBUF], - self->param[MSCABD_PARAM_FIXMSZIP]); + self->d->state = mszipd_init(&self->d->sys, fh, fh, self->buf_size, + self->fix_mszip); break; case cffoldCOMPTYPE_QUANTUM: self->d->decompress = (int (*)(void *, off_t)) &qtmd_decompress; self->d->state = qtmd_init(&self->d->sys, fh, fh, (int) (ct >> 8) & 0x1f, - self->param[MSCABD_PARAM_DECOMPBUF]); + self->buf_size); break; case cffoldCOMPTYPE_LZX: self->d->decompress = (int (*)(void *, off_t)) &lzxd_decompress; self->d->state = lzxd_init(&self->d->sys, fh, fh, (int) (ct >> 8) & 0x1f, 0, - self->param[MSCABD_PARAM_DECOMPBUF], (off_t)0,0); + self->buf_size, (off_t)0,0); break; default: return self->error = MSPACK_ERR_DATAFORMAT; @@ -1224,10 +1219,10 @@ struct mspack_system *sys = self->system; int avail, todo, outlen, ignore_cksum, ignore_blocksize; - ignore_cksum = self->param[MSCABD_PARAM_SALVAGE] || - (self->param[MSCABD_PARAM_FIXMSZIP] && + ignore_cksum = self->salvage || + (self->fix_mszip && ((self->d->comp_type & cffoldCOMPTYPE_MASK) == cffoldCOMPTYPE_MSZIP)); - ignore_blocksize = self->param[MSCABD_PARAM_SALVAGE]; + ignore_blocksize = self->salvage; todo = bytes; while (todo > 0) { @@ -1247,7 +1242,7 @@ /* check if we're out of input blocks, advance block counter */ if (self->d->block++ >= self->d->folder->base.num_blocks) { - if (!self->param[MSCABD_PARAM_SALVAGE]) { + if (!self->salvage) { self->read_error = MSPACK_ERR_DATAFORMAT; } else { @@ -1484,17 +1479,17 @@ switch (param) { case MSCABD_PARAM_SEARCHBUF: if (value < 4) return MSPACK_ERR_ARGS; - self->param[MSCABD_PARAM_SEARCHBUF] = value; + self->searchbuf_size = value; break; case MSCABD_PARAM_FIXMSZIP: - self->param[MSCABD_PARAM_FIXMSZIP] = value; + self->fix_mszip = value; break; case MSCABD_PARAM_DECOMPBUF: if (value < 4) return MSPACK_ERR_ARGS; - self->param[MSCABD_PARAM_DECOMPBUF] = value; + self->buf_size = value; break; case MSCABD_PARAM_SALVAGE: - self->param[MSCABD_PARAM_SALVAGE] = value; + self->salvage = value; break; default: return MSPACK_ERR_ARGS; diff -Nru -w libmspack-0.8/mspack/cab.h libmspack-0.10.1/mspack/cab.h --- libmspack-0.8/mspack/cab.h 2018-10-21 17:44:47.000000000 +0200 +++ libmspack-0.10.1/mspack/cab.h 2019-03-04 10:09:35.000000000 +0100 @@ -10,10 +10,6 @@ #ifndef MSPACK_CAB_H #define MSPACK_CAB_H 1 -#include <mszip.h> -#include <qtm.h> -#include <lzx.h> - /* generic CAB definitions */ /* structure offsets */ @@ -117,7 +113,7 @@ struct mscab_decompressor base; struct mscabd_decompress_state *d; struct mspack_system *system; - int param[4]; /* !!! MATCH THIS TO NUM OF PARAMS IN MSPACK.H !!! */ + int buf_size, searchbuf_size, fix_mszip, salvage; /* params */ int error, read_error; }; diff -Nru -w libmspack-0.8/mspack/chmd.c libmspack-0.10.1/mspack/chmd.c --- libmspack-0.8/mspack/chmd.c 2018-10-17 12:24:53.000000000 +0200 +++ libmspack-0.10.1/mspack/chmd.c 2019-03-04 02:10:28.000000000 +0100 @@ -44,7 +44,7 @@ struct mschm_decompressor_p *self, struct mschmd_file *file); static int read_reset_table( struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, - int entry, off_t *length_ptr, off_t *offset_ptr); + unsigned int entry, off_t *length_ptr, off_t *offset_ptr); static int read_spaninfo( struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, off_t *length_ptr); @@ -292,7 +292,7 @@ } /* check both header GUIDs */ - if (mspack_memcmp(&buf[chmhead_GUID1], &guids[0], 32L) != 0) { + if (memcmp(&buf[chmhead_GUID1], &guids[0], 32L) != 0) { D(("incorrect GUIDs")) return MSPACK_ERR_SIGNATURE; } @@ -381,14 +381,18 @@ D(("more than 100,000 chunks")) return MSPACK_ERR_DATAFORMAT; } + if (chm->chunk_size > 8192) { + D(("chunk size over 8192 (get in touch if this is valid)")) + return MSPACK_ERR_DATAFORMAT; + } if ((off_t)chm->chunk_size * (off_t)chm->num_chunks > chm->length) { D(("chunks larger than entire file")) return MSPACK_ERR_DATAFORMAT; } /* common sense checks on header section 1 fields */ - if ((chm->chunk_size & (chm->chunk_size - 1)) != 0) { - sys->message(fh, "WARNING; chunk size is not a power of two"); + if (chm->chunk_size != 4096) { + sys->message(fh, "WARNING; chunk size is not 4096"); } if (chm->first_pmgl != 0) { sys->message(fh, "WARNING; first PMGL chunk is not zero"); @@ -435,7 +439,7 @@ sys->message(fh, "WARNING; PMGL quickref area is too small"); } if (EndGetI32(&chunk[pmgl_QuickRefSize]) > - ((int)chm->chunk_size - pmgl_Entries)) + (chm->chunk_size - pmgl_Entries)) { sys->message(fh, "WARNING; PMGL quickref area is too large"); } @@ -483,20 +487,18 @@ if (name[0] == ':' && name[1] == ':') { /* system file */ - if (mspack_memcmp(&name[2], &content_name[2], 31L) == 0) { - if (mspack_memcmp(&name[33], &content_name[33], 8L) == 0) { + if (name_len == 40 && memcmp(name, content_name, 40) == 0) { chm->sec1.content = fi; } - else if (mspack_memcmp(&name[33], &control_name[33], 11L) == 0) { + else if (name_len == 44 && memcmp(name, control_name, 44) == 0) { chm->sec1.control = fi; } - else if (mspack_memcmp(&name[33], &spaninfo_name[33], 8L) == 0) { + else if (name_len == 41 && memcmp(name, spaninfo_name, 41) == 0) { chm->sec1.spaninfo = fi; } - else if (mspack_memcmp(&name[33], &rtable_name[33], 72L) == 0) { + else if (name_len == 105 && memcmp(name, rtable_name, 105) == 0) { chm->sec1.rtable = fi; } - } fi->next = chm->sysfiles; chm->sysfiles = fi; } @@ -536,7 +538,10 @@ struct mschm_decompressor_p *self = (struct mschm_decompressor_p *) base; struct mspack_system *sys; struct mspack_file *fh; - const unsigned char *chunk, *p, *end; + /* p and end are initialised to prevent MSVC warning about "potentially" + * uninitialised usage. This is provably untrue, but MS won't fix: + * https://developercommunity.visualstudio.com/content/problem/363489/c4701-false-positive-warning.html */ + const unsigned char *chunk, *p = NULL, *end = NULL; int err = MSPACK_ERR_OK, result = -1; unsigned int n, sec; @@ -586,7 +591,7 @@ } /* stop simple infinite loops: can't visit the same chunk twice */ - if ((int)n == EndGetI32(&chunk[pmgl_NextChunk])) { + if (n == EndGetI32(&chunk[pmgl_NextChunk])) { break; } } @@ -824,35 +829,11 @@ } #if HAVE_TOWLOWER -# if HAVE_WCTYPE_H # include <wctype.h> -# endif # define TOLOWER(x) towlower(x) -#elif HAVE_TOLOWER -# if HAVE_CTYPE_H +#else # include <ctype.h> -# endif # define TOLOWER(x) tolower(x) -#else -# define TOLOWER(x) (((x)<0||(x)>255)?(x):mspack_tolower_map[(x)]) -/* Map of char -> lowercase char for the first 256 chars. Generated with: - * LC_CTYPE=en_GB.utf-8 perl -Mlocale -le 'print map{ord(lc chr).","} 0..255' - */ -static const unsigned char mspack_tolower_map[256] = { - 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27, - 28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52, - 53,54,55,56,57,58,59,60,61,62,63,64,97,98,99,100,101,102,103,104,105,106, - 107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,91,92,93,94, - 95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114, - 115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133, - 134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152, - 153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171, - 172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190, - 191,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241, - 242,243,244,245,246,215,248,249,250,251,252,253,254,223,224,225,226,227,228, - 229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247, - 248,249,250,251,252,253,254,255 -}; #endif /* decodes a UTF-8 character from s[] into c. Will not read past e. @@ -1175,7 +1156,8 @@ */ static int read_reset_table(struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, - int entry, off_t *length_ptr, off_t *offset_ptr) + unsigned int entry, + off_t *length_ptr, off_t *offset_ptr) { struct mspack_system *sys = self->system; unsigned char *data; diff -Nru -w libmspack-0.8/mspack/mspack.h libmspack-0.10.1/mspack/mspack.h --- libmspack-0.8/mspack/mspack.h 2018-10-16 11:17:09.000000000 +0200 +++ libmspack-0.10.1/mspack/mspack.h 2019-02-18 21:25:16.000000000 +0100 @@ -1,5 +1,5 @@ /* libmspack -- a library for working with Microsoft compression formats. - * (C) 2003-2016 Stuart Caie <ky...@cabextract.org.uk> + * (C) 2003-2019 Stuart Caie <ky...@cabextract.org.uk> * * libmspack is free software; you can redistribute it and/or modify it under * the terms of the GNU Lesser General Public License (LGPL) version 2.1 @@ -1554,7 +1554,7 @@ */ int (*set_param)(struct mschm_compressor *self, int param, - unsigned int value); + int value); /** * Returns the error code set by the most recently called method. @@ -1853,7 +1853,7 @@ */ int (*set_param)(struct msszdd_compressor *self, int param, - unsigned int value); + int value); /** * Returns the error code set by the most recently called method. @@ -2091,7 +2091,7 @@ */ int (*set_param)(struct mskwaj_compressor *self, int param, - unsigned int value); + int value); /** @@ -2353,8 +2353,31 @@ const char *input, const char *base, const char *output); + + /** + * Sets an OAB decompression engine parameter. Available only in OAB + * decompressor version 2 and above. + * + * - #MSOABD_PARAM_DECOMPBUF: How many bytes should be used as an input + * buffer by decompressors? The minimum value is 16. The default value + * is 4096. + * + * @param self a self-referential pointer to the msoab_decompressor + * instance being called + * @param param the parameter to set + * @param value the value to set the parameter to + * @return MSPACK_ERR_OK if all is OK, or MSPACK_ERR_ARGS if there + * is a problem with either parameter or value. + */ + int (*set_param)(struct msoab_decompressor *self, + int param, + int value); + }; +/** msoab_decompressor::set_param() parameter: size of decompression buffer */ +#define MSOABD_PARAM_DECOMPBUF (0) + #ifdef __cplusplus } #endif diff -Nru -w libmspack-0.8/mspack/oabd.c libmspack-0.10.1/mspack/oabd.c --- libmspack-0.8/mspack/oabd.c 2018-08-13 14:21:07.000000000 +0200 +++ libmspack-0.10.1/mspack/oabd.c 2019-03-04 10:11:49.000000000 +0100 @@ -33,6 +33,11 @@ static int oabd_decompress_incremental(struct msoab_decompressor *self, const char *input, const char *base, const char *output); +static int oabd_param(struct msoab_decompressor *base, int param, int value); +static int copy_fh(struct mspack_system *sys, struct mspack_file *infh, + struct mspack_file *outfh, size_t bytes_to_copy, + unsigned char *buf, int buf_size); + struct msoab_decompressor * mspack_create_oab_decompressor(struct mspack_system *sys) @@ -45,7 +50,9 @@ if ((self = (struct msoab_decompressor_p *) sys->alloc(sys, sizeof(struct msoab_decompressor_p)))) { self->base.decompress = &oabd_decompress; self->base.decompress_incremental = &oabd_decompress_incremental; + self->base.set_param = &oabd_param; self->system = sys; + self->buf_size = 4096; } return (struct msoab_decompressor *) self; } @@ -132,17 +139,13 @@ block_max = EndGetI32(&hdrbuf[oabhead_BlockMax]); target_size = EndGetI32(&hdrbuf[oabhead_TargetSize]); - /* We use it for reading block headers too */ - if (block_max < oabblk_SIZEOF) - block_max = oabblk_SIZEOF; - outfh = sys->open(sys, output, MSPACK_SYS_OPEN_WRITE); if (!outfh) { ret = MSPACK_ERR_OPEN; goto out; } - buf = sys->alloc(sys, block_max); + buf = sys->alloc(sys, self->buf_size); if (!buf) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -181,14 +184,8 @@ ret = MSPACK_ERR_DATAFORMAT; goto out; } - if (sys->read(infh, buf, blk_dsize) != (int)blk_dsize) { - ret = MSPACK_ERR_READ; - goto out; - } - if (sys->write(outfh, buf, blk_dsize) != (int)blk_dsize) { - ret = MSPACK_ERR_WRITE; - goto out; - } + ret = copy_fh(sys, infh, outfh, blk_dsize, buf, self->buf_size); + if (ret) goto out; } else { /* LZX compressed block */ window_bits = 17; @@ -200,7 +197,7 @@ out_ofh.crc = 0xffffffff; lzx = lzxd_init(&oabd_sys, (void *)&in_ofh, (void *)&out_ofh, window_bits, - 0, 4096, blk_dsize, 1); + 0, self->buf_size, blk_dsize, 1); if (!lzx) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -214,18 +211,8 @@ lzx = NULL; /* Consume any trailing padding bytes before the next block */ - while (in_ofh.available) { - int count = block_max; - if ((size_t)count > in_ofh.available) - count = in_ofh.available; - - count = sys->read(infh, buf, count); - if (count < 0) { - ret = MSPACK_ERR_READ; - goto out; - } - in_ofh.available -= count; - } + ret = copy_fh(sys, infh, NULL, in_ofh.available, buf, self->buf_size); + if (ret) goto out; if (out_ofh.crc != blk_crc) { ret = MSPACK_ERR_CHECKSUM; @@ -301,7 +288,7 @@ goto out; } - buf = sys->alloc(sys, block_max); + buf = sys->alloc(sys, self->buf_size); if (!buf) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -364,18 +351,8 @@ lzx = NULL; /* Consume any trailing padding bytes before the next block */ - while (in_ofh.available) { - int count = block_max; - if ((size_t)count > in_ofh.available) - count = in_ofh.available; - - count = sys->read(infh, buf, count); - if (count < 0) { - ret = MSPACK_ERR_READ; - goto out; - } - in_ofh.available -= count; - } + ret = copy_fh(sys, infh, NULL, in_ofh.available, buf, self->buf_size); + if (ret) goto out; if (out_ofh.crc != blk_crc) { ret = MSPACK_ERR_CHECKSUM; @@ -394,3 +371,33 @@ return ret; } + +static int copy_fh(struct mspack_system *sys, struct mspack_file *infh, + struct mspack_file *outfh, size_t bytes_to_copy, + unsigned char *buf, int buf_size) +{ + while (bytes_to_copy) { + int run = buf_size; + if ((size_t) run > bytes_to_copy) { + run = (int) bytes_to_copy; + } + if (sys->read(infh, buf, run) != run) { + return MSPACK_ERR_READ; + } + if (outfh && sys->write(outfh, buf, run) != run) { + return MSPACK_ERR_WRITE; + } + bytes_to_copy -= run; + } + return MSPACK_ERR_OK; +} + +static int oabd_param(struct msoab_decompressor *base, int param, int value) { + struct msoab_decompressor_p *self = (struct msoab_decompressor_p *) base; + if (self && param == MSOABD_PARAM_DECOMPBUF && value >= 16) { + /* must be at least 16 bytes (patchblk_SIZEOF, oabblk_SIZEOF) */ + self->buf_size = value; + return MSPACK_ERR_OK; + } + return MSPACK_ERR_ARGS; +} diff -Nru -w libmspack-0.8/mspack/oab.h libmspack-0.10.1/mspack/oab.h --- libmspack-0.8/mspack/oab.h 2017-11-26 15:11:38.000000000 +0100 +++ libmspack-0.10.1/mspack/oab.h 2019-02-18 17:39:40.000000000 +0100 @@ -27,6 +27,7 @@ struct msoab_decompressor_p { struct msoab_decompressor base; struct mspack_system *system; + int buf_size; /* todo */ }; diff -Nru -w libmspack-0.8/mspack/system.c libmspack-0.10.1/mspack/system.c --- libmspack-0.8/mspack/system.c 2018-10-16 11:17:09.000000000 +0200 +++ libmspack-0.10.1/mspack/system.c 2019-02-18 17:26:17.000000000 +0100 @@ -31,12 +31,15 @@ * - added MSCABD_PARAM_SALVAGE */ case MSPACK_VER_MSCABD: + /* OAB decoder version 1 -> 2 changes: + * - added msoab_decompressor::set_param and MSOABD_PARAM_DECOMPBUF + */ + case MSPACK_VER_MSOABD: return 2; case MSPACK_VER_LIBRARY: case MSPACK_VER_SYSTEM: case MSPACK_VER_MSSZDDD: case MSPACK_VER_MSKWAJD: - case MSPACK_VER_MSOABD: return 1; case MSPACK_VER_MSCABC: case MSPACK_VER_MSCHMC: diff -Nru -w libmspack-0.8/mspack/system.h libmspack-0.10.1/mspack/system.h --- libmspack-0.8/mspack/system.h 2018-07-29 09:46:43.000000000 +0200 +++ libmspack-0.10.1/mspack/system.h 2019-03-04 02:10:28.000000000 +0100 @@ -1,5 +1,5 @@ /* This file is part of libmspack. - * (C) 2003-2004 Stuart Caie. + * (C) 2003-2018 Stuart Caie. * * libmspack is free software; you can redistribute it and/or modify it under * the terms of the GNU Lesser General Public License (LGPL) version 2.1 @@ -21,6 +21,9 @@ #include <mspack.h> +/* assume <string.h> exists */ +#include <string.h> + /* fix for problem with GCC 4 and glibc (thanks to Ville Skytta) * http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=150429 */ @@ -54,10 +57,6 @@ * greater than 2GB is detected, an error message indicating the library * can't support the file should be printed. */ -#if HAVE_LIMITS_H -# include <limits.h> -#endif - #if HAVE_INTTYPES_H # include <inttypes.h> #else @@ -67,10 +66,11 @@ # define PRIu32 "lu" #endif +#include <limits.h> #if ((defined(_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS >= 64) || \ (defined(FILESIZEBITS) && FILESIZEBITS >= 64) || \ - (defined(SIZEOF_OFF_T) && SIZEOF_OFF_T >= 8) || \ - defined(_LARGEFILE_SOURCE) || defined(_LARGEFILE64_SOURCE)) + defined(_LARGEFILE_SOURCE) || defined(_LARGEFILE64_SOURCE) || \ + SIZEOF_OFF_T >= 8) # define LARGEFILE_SUPPORT 1 # define LD PRId64 # define LU PRIu64 @@ -81,20 +81,19 @@ #endif /* endian-neutral reading of little-endian data */ -#define __egi32(a,n) ( ((((unsigned char *) a)[n+3]) << 24) | \ - ((((unsigned char *) a)[n+2]) << 16) | \ - ((((unsigned char *) a)[n+1]) << 8) | \ - ((((unsigned char *) a)[n+0]))) -#define EndGetI64(a) ((((unsigned long long int) __egi32(a,4)) << 32) | \ - ((unsigned int) __egi32(a,0))) +#define __egi32(a,n) (((unsigned int) ((unsigned char *)(a))[n+3] << 24) | \ + ((unsigned int) ((unsigned char *)(a))[n+2] << 16) | \ + ((unsigned int) ((unsigned char *)(a))[n+1] << 8) | \ + ((unsigned int) ((unsigned char *)(a))[n])) +#define EndGetI64(a) (((unsigned long long int) __egi32(a,4) << 32) | __egi32(a,0)) #define EndGetI32(a) __egi32(a,0) #define EndGetI16(a) ((((a)[1])<<8)|((a)[0])) /* endian-neutral reading of big-endian data */ -#define EndGetM32(a) (((((unsigned char *) a)[0]) << 24) | \ - ((((unsigned char *) a)[1]) << 16) | \ - ((((unsigned char *) a)[2]) << 8) | \ - ((((unsigned char *) a)[3]))) +#define EndGetM32(a) (((unsigned int) ((unsigned char *)(a))[0] << 24) | \ + ((unsigned int) ((unsigned char *)(a))[1] << 16) | \ + ((unsigned int) ((unsigned char *)(a))[2] << 8) | \ + ((unsigned int) ((unsigned char *)(a))[3])) #define EndGetM16(a) ((((a)[0])<<8)|((a)[1])) extern struct mspack_system *mspack_default_system; @@ -106,27 +105,6 @@ /* validates a system structure */ extern int mspack_valid_system(struct mspack_system *sys); -#if HAVE_STRINGS_H -# include <strings.h> -#endif - -#if HAVE_STRING_H -# include <string.h> -#endif - -#if HAVE_MEMCMP -# define mspack_memcmp memcmp -#else -/* inline memcmp() */ -static inline int mspack_memcmp(const void *s1, const void *s2, size_t n) { - unsigned char *c1 = (unsigned char *) s1; - unsigned char *c2 = (unsigned char *) s2; - if (n == 0) return 0; - while (--n && (*c1 == *c2)) c1++, c2++; - return *c1 - *c2; -} -#endif - #ifdef __cplusplus } #endif diff -Nru -w libmspack-0.8/mspack/szddd.c libmspack-0.10.1/mspack/szddd.c --- libmspack-0.8/mspack/szddd.c 2018-08-13 14:21:53.000000000 +0200 +++ libmspack-0.10.1/mspack/szddd.c 2018-11-05 12:53:09.000000000 +0100 @@ -150,7 +150,7 @@ /* read and check signature */ if (sys->read(fh, buf, 8) != 8) return MSPACK_ERR_READ; - if ((mspack_memcmp(buf, szdd_signature_expand, 8) == 0)) { + if ((memcmp(buf, szdd_signature_expand, 8) == 0)) { /* common SZDD */ hdr->format = MSSZDD_FMT_NORMAL; @@ -160,7 +160,7 @@ hdr->missing_char = buf[1]; hdr->length = EndGetI32(&buf[2]); } - else if ((mspack_memcmp(buf, szdd_signature_qbasic, 8) == 0)) { + else if ((memcmp(buf, szdd_signature_qbasic, 8) == 0)) { /* special QBasic SZDD */ hdr->format = MSSZDD_FMT_QBASIC; if (sys->read(fh, buf, 4) != 4) return MSPACK_ERR_READ; diff -Nru -w libmspack-0.8/README libmspack-0.10.1/README --- libmspack-0.8/README 2018-10-21 17:49:53.000000000 +0200 +++ libmspack-0.10.1/README 2019-03-04 10:36:57.000000000 +0100 @@ -1,4 +1,4 @@ -libmspack 0.8alpha +libmspack 0.10.1alpha The purpose of libmspack is to provide compressors and decompressors, archivers and dearchivers for Microsoft compression formats: CAB, CHM, WIM, @@ -43,7 +43,7 @@ In addition to gcc, you also need the following for building from repository: - at least autoconf 2.57 -- at least automake 1.7 +- at least automake 1.11 - libtool This is an alpha release. Unless you are in a position to package the @@ -84,21 +84,18 @@ in-memory images, raw file descriptors, open file handles and regular disk files -src/cabrip.c - extracts any CAB files embedded in another file -src/chmextract.c - extracts all files in a CHM file to disk -src/msexpand.c - expands an SZDD or KWAJ file -src/oabextract.c - extracts an Exchange Offline Address Book (.LZX) file +examples/cabrip.c - extracts any CAB files embedded in another file +examples/chmextract.c - extracts all files in a CHM file to disk +examples/msexpand.c - expands an SZDD or KWAJ file +examples/oabextract.c - extracts an Exchange Offline Address Book (.LZX) file test/cabd_c10 - tests the CAB decompressor on the C10 collection test/cabd_compare - compares libmspack with Microsoft's EXTRACT.EXE test/cabd_md5 - shows MD5 checksums of all files in a CAB file/set -test/cabd_test.c - regression tests for libmspack's CAB decompression test/chmd_compare - compares libmspack with Microsoft's HH.EXE -test/chmd_find.c - fast-finds a file within a CHM file. +test/chmd_find.c - checks all files in a CHM file can be fast-found test/chmd_md5.c - shows MD5 checksums of all files within a CHM file test/chmd_order.c - extracts files in a CHM file in four different ways -test/chmd_test.c - regression tests for libmspack's CHM decompression -test/kwajd_test.c - regression tests for libmspack's KWAJ decompression test/chminfo.c - prints verbose information about CHM file structures test/msdecompile_md5 - runs Microsoft's HH.EXE -DECOMPILE via WINE test/msextract_md5 - runs Microsoft's EXTRACT.EXE via WINE diff -Nru -w libmspack-0.8/src/cabrip.c libmspack-0.10.1/src/cabrip.c --- libmspack-0.8/src/cabrip.c 2018-10-21 14:26:01.000000000 +0200 +++ libmspack-0.10.1/src/cabrip.c 1970-01-01 01:00:00.000000000 +0100 @@ -1,85 +0,0 @@ -#ifdef HAVE_CONFIG_H -#include <config.h> -#endif - -#include <stdio.h> -#include <stdlib.h> -#include <sys/stat.h> -#include <mspack.h> -#include <system.h> - -#if HAVE_FSEEKO -# define fseek fseeko -#endif - -#define BUF_SIZE (1024*4096) -char buf[BUF_SIZE]; - -void rip(char *fname, off_t offset, unsigned int length) { - static unsigned int counter = 1; - struct stat st_buf; - char outname[13]; - FILE *in = NULL, *out = NULL; - - /* find an unused output filename */ - do { - snprintf(outname, 13, "%08u.cab", counter++); - } while (stat(outname, &st_buf) == 0); - - printf("ripping %s offset %" LD " length %u to %s\n", - fname, offset, length, outname); - - if (!(in = fopen(fname, "rb"))) { - perror(fname); - goto cleanup; - } - if (!(out = fopen(outname, "wb"))) { - perror(outname); - goto cleanup; - } - if (fseek(in, offset, SEEK_SET)) { - fprintf(stderr, "%s: can't seek to cab offset %"LD"\n", fname, offset); - goto cleanup; - } - while (length) { - size_t run = (length > BUF_SIZE) ? BUF_SIZE : length; - size_t actual = fread(&buf[0], 1, run, in); - if (actual < run) { - fprintf(stderr, "%s: file %u bytes shorter than expected\n", - fname, length - (run - actual)); - length = run = actual; - } - if (fwrite(&buf[0], 1, run, out) != run) { - perror(outname); - break; - } - length -= run; - } - -cleanup: - if (in) fclose(in); - if (out) fclose(out); -} - -int main(int argc, char *argv[]) { - struct mscab_decompressor *cabd; - struct mscabd_cabinet *cab, *c; - int err; - - MSPACK_SYS_SELFTEST(err); - if (err) return 0; - - if ((cabd = mspack_create_cab_decompressor(NULL))) { - cabd->set_param(cabd, MSCABD_PARAM_SALVAGE, 1); - for (argv++; *argv; argv++) { - if ((cab = cabd->search(cabd, *argv))) { - for (c = cab; c; c = c->next) { - rip(*argv, c->base_offset, c->length); - } - cabd->close(cabd, cab); - } - } - mspack_destroy_cab_decompressor(cabd); - } - return 0; -} diff -Nru -w libmspack-0.8/src/chmextract.c libmspack-0.10.1/src/chmextract.c --- libmspack-0.8/src/chmextract.c 2018-10-20 20:05:18.000000000 +0200 +++ libmspack-0.10.1/src/chmextract.c 1970-01-01 01:00:00.000000000 +0100 @@ -1,122 +0,0 @@ -#ifdef HAVE_CONFIG_H -#include <config.h> -#endif - -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <mspack.h> -#include <ctype.h> -#include <sys/stat.h> - -#include <error.h> - -#if HAVE_MKDIR -# if MKDIR_TAKES_ONE_ARG -# define mkdir(a, b) mkdir(a) -# endif -#else -# if HAVE__MKDIR -# define mkdir(a, b) _mkdir(a) -# else -# error "Don't know how to create a directory on this system." -# endif -#endif - -mode_t user_umask; - -/** - * Ensures that all directory components in a filepath exist. New directory - * components are created, if necessary. - * - * @param path the filepath to check - * @return non-zero if all directory components in a filepath exist, zero - * if components do not exist and cannot be created - */ -static int ensure_filepath(char *path) { - struct stat st_buf; - char *p; - int ok; - - for (p = &path[1]; *p; p++) { - if (*p != '/') continue; - *p = '\0'; - ok = (stat(path, &st_buf) == 0) && S_ISDIR(st_buf.st_mode); - if (!ok) ok = (mkdir(path, 0777 & ~user_umask) == 0); - *p = '/'; - if (!ok) return 0; - } - return 1; -} - -char *create_output_name(char *fname) { - char *out, *p; - if ((out = malloc(strlen(fname) + 1))) { - /* remove leading slashes */ - while (*fname == '/' || *fname == '\\') fname++; - /* if that removes all characters, just call it "x" */ - strcpy(out, (*fname) ? fname : "x"); - - /* change "../" to "xx/" */ - for (p = out; *p; p++) { - if (p[0] == '.' && p[1] == '.' && (p[2] == '/' || p[2] == '\\')) { - p[0] = p[1] = 'x'; - } - } - } - return out; -} - -static int sortfunc(const void *a, const void *b) { - off_t diff = - ((* ((struct mschmd_file **) a))->offset) - - ((* ((struct mschmd_file **) b))->offset); - return (diff < 0) ? -1 : ((diff > 0) ? 1 : 0); -} - -int main(int argc, char *argv[]) { - struct mschm_decompressor *chmd; - struct mschmd_header *chm; - struct mschmd_file *file, **f; - unsigned int numf, i; - - setbuf(stdout, NULL); - setbuf(stderr, NULL); - user_umask = umask(0); umask(user_umask); - - MSPACK_SYS_SELFTEST(i); - if (i) return 0; - - if ((chmd = mspack_create_chm_decompressor(NULL))) { - for (argv++; *argv; argv++) { - printf("%s\n", *argv); - if ((chm = chmd->open(chmd, *argv))) { - - /* build an ordered list of files for maximum extraction speed */ - for (numf=0, file=chm->files; file; file = file->next) numf++; - if ((f = (struct mschmd_file **) calloc(numf, sizeof(struct mschmd_file *)))) { - for (i=0, file=chm->files; file; file = file->next) f[i++] = file; - qsort(f, numf, sizeof(struct mschmd_file *), &sortfunc); - - for (i = 0; i < numf; i++) { - char *outname = create_output_name(f[i]->filename); - printf("Extracting %s\n", outname); - ensure_filepath(outname); - if (chmd->extract(chmd, f[i], outname)) { - printf("%s: extract error on \"%s\": %s\n", - *argv, f[i]->filename, ERROR(chmd)); - } - free(outname); - } - free(f); - } - chmd->close(chmd, chm); - } - else { - printf("%s: can't open -- %s\n", *argv, ERROR(chmd)); - } - } - mspack_destroy_chm_decompressor(chmd); - } - return 0; -} diff -Nru -w libmspack-0.8/src/error.h libmspack-0.10.1/src/error.h --- libmspack-0.8/src/error.h 2017-08-13 20:35:30.000000000 +0200 +++ libmspack-0.10.1/src/error.h 1970-01-01 01:00:00.000000000 +0100 @@ -1,22 +0,0 @@ -#define ERROR(base) error_msg(base->last_error(base)) - -const char *error_msg(int error) { - static char buf[32]; - switch (error) { - case MSPACK_ERR_OK: return "no error"; - case MSPACK_ERR_ARGS: return "bad arguments to library function"; - case MSPACK_ERR_OPEN: return "error opening file"; - case MSPACK_ERR_READ: return "read error"; - case MSPACK_ERR_WRITE: return "write error"; - case MSPACK_ERR_SEEK: return "seek error"; - case MSPACK_ERR_NOMEMORY: return "out of memory"; - case MSPACK_ERR_SIGNATURE: return "bad signature"; - case MSPACK_ERR_DATAFORMAT: return "error in data format"; - case MSPACK_ERR_CHECKSUM: return "checksum error"; - case MSPACK_ERR_CRUNCH: return "compression error"; - case MSPACK_ERR_DECRUNCH: return "decompression error"; - } - - snprintf(buf, sizeof(buf), "unknown error %d", error); - return buf; -} diff -Nru -w libmspack-0.8/src/msexpand.c libmspack-0.10.1/src/msexpand.c --- libmspack-0.8/src/msexpand.c 2017-08-14 11:10:40.000000000 +0200 +++ libmspack-0.10.1/src/msexpand.c 1970-01-01 01:00:00.000000000 +0100 @@ -1,48 +0,0 @@ -/* acts like Microsoft's EXPAND.EXE */ -#ifdef HAVE_CONFIG_H -#include <config.h> -#endif - -#include <stdio.h> -#include <stdlib.h> -#include <string.h> - -#include <mspack.h> -#include <error.h> - -int main(int argc, char *argv[]) { - struct msszdd_decompressor *szddd; - struct mskwaj_decompressor *kwajd; - int err; - - if (argc != 3) { - fprintf(stderr, "Usage: %s <input file> <output file>\n", argv[0]); - return 1; - } - - /* exit if self-test reveals an error */ - MSPACK_SYS_SELFTEST(err); - if (err) return 1; - - szddd = mspack_create_szdd_decompressor(NULL); - kwajd = mspack_create_kwaj_decompressor(NULL); - - if (szddd && kwajd) { - err = szddd->decompress(szddd, argv[1], argv[2]); - /* if not SZDD file, try decompressing as KWAJ */ - if (err == MSPACK_ERR_SIGNATURE) { - err = kwajd->decompress(kwajd, argv[1], argv[2]); - } - if (err != MSPACK_ERR_OK) { - fprintf(stderr, "%s -> %s: %s\n", argv[1], argv[2], error_msg(err)); - } - } - else { - fprintf(stderr, "can't create SZDD/KWAJ decompressor\n"); - err = 1; - } - - mspack_destroy_szdd_decompressor(szddd); - mspack_destroy_kwaj_decompressor(kwajd); - return err ? 1 : 0; -} diff -Nru -w libmspack-0.8/src/oabextract.c libmspack-0.10.1/src/oabextract.c --- libmspack-0.8/src/oabextract.c 2015-06-05 10:10:54.000000000 +0200 +++ libmspack-0.10.1/src/oabextract.c 1970-01-01 01:00:00.000000000 +0100 @@ -1,41 +0,0 @@ -#ifdef HAVE_CONFIG_H -#include <config.h> -#endif - -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <mspack.h> - -#include <error.h> - -int main(int argc, char *argv[]) { - struct msoab_decompressor *oabd; - int err; - - setbuf(stdout, NULL); - setbuf(stderr, NULL); - - MSPACK_SYS_SELFTEST(err); - if (err) return 0; - - if ((oabd = mspack_create_oab_decompressor(NULL))) { - if (argc == 3) { - err = oabd->decompress(oabd, argv[1], argv[2]); - if (err) fprintf(stderr, "%s -> %s: %s\n", argv[1], argv[2], error_msg(err)); - } - else if (argc == 4) { - err = oabd->decompress_incremental(oabd, argv[2], argv[1], argv[3]); - if (err) fprintf(stderr, "%s + %s -> %s: %s\n", argv[1], argv[2], argv[3], error_msg(err)); - } - else { - fprintf(stderr, "Usage: %s <input> <output>\n", *argv); - fprintf(stderr, " or %s <base> <patch> <output>\n", *argv); - } - mspack_destroy_oab_decompressor(oabd); - } - else { - fprintf(stderr, "%s: can't make OAB decompressor\n", *argv); - } - return 0; -}
diffstat for libmspack-0.9.1 libmspack-0.10.1 ChangeLog | 50 ++++ README | 2 config.h.in | 15 + -configure | 245 +++++++++++++++++++++++- configure.ac | 3 debian/changelog | 8 debian/copyright | 2 debian/libmspack-doc.docs | 4 mspack/cab.h | 2 mspack/cabd.c | 54 ++--- mspack/chmd.c | 27 +- mspack/mspack.h | 31 ++- mspack/oab.h | 1 mspack/oabd.c | 85 ++++---- mspack/system.c | 5 mspack/system.h | 19 - -test/chminfo.c | 2 -test/test_files/chmd/short-system-filenames.chm |binary 18 files changed, 447 insertions(+), 108 deletions(-) diff -Nru -w libmspack-0.9.1/ChangeLog libmspack-0.10.1/ChangeLog --- libmspack-0.9.1/ChangeLog 2018-11-04 01:04:14.000000000 +0100 +++ libmspack-0.10.1/ChangeLog 2019-02-18 21:01:59.000000000 +0100 @@ -1,3 +1,53 @@ +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * chmd_read_headers(): a CHM file name beginning "::" but shorter + than 33 bytes will lead to reading past the freshly-allocated name + buffer - checks for specific control filenames didn't take length + into account. Thanks to ADLab of Venustech for the report and + proof of concept. + +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * chmd_read_headers(): CHM files can declare their chunks are any + size up to 4GB, and libmspack will attempt to allocate that to + read the file. + + This is not a security issue; libmspack doesn't promise how much + memory it'll use to unpack files. You can set your own limits by + returning NULL in a custom mspack_system.alloc() implementation. + + However, it would be good to validate chunk size further. With no + offical specification, only empirical data is available. All files + created by hhc.exe have a chunk size of 4096 bytes, and this is + matched by all the files I've found in the wild, except for one + which has a chunk size of 8192 bytes, which was created by someone + developing a CHM file creator 15 years ago, and they appear to + have abandoned it, so it seems 4096 is a de-facto standard. + + I've changed the "chunk size is not a power of two" warning to + "chunk size is not 4096", and now only allow chunk sizes between + 22 and 8192 bytes. If you have CHM files with a larger chunk size, + please send them to me and I'll increase this upper limit. + + Thanks to ADLab of Venustech for the report. + +2019-02-18 Stuart Caie <ky...@cabextract.org.uk> + + * oabd.c: replaced one-shot copying of uncompressed blocks (which + requires allocating a buffer of the size declared in the header, + which can be 4GB) with a fixed-size buffer. The buffer size is + user-controllable with the new msoab_decompressor::set_param() + method (check you have version 2 of the OAB decompressor), and + also controls the input buffer used for OAB's LZX decompression. + + Reminder: compression formats can dictate how much memory is + needed to decompress them. If memory usage is a security concern + to you, write a custom mspack_system.alloc() that returns NULL + if "too much" memory is requested. Do not rely on libmspack adding + special heuristics to know not to request "too much". + + Thanks to ADLab of Venustech for the report. + 2018-11-03 Stuart Caie <ky...@cabextract.org.uk> * configure.ac, doc/Makefile.in, doc/Doxyfile.in: remove these diff -Nru -w libmspack-0.9.1/config.h.in libmspack-0.10.1/config.h.in --- libmspack-0.9.1/config.h.in 2018-11-06 12:17:02.000000000 +0100 +++ libmspack-0.10.1/config.h.in 2019-03-04 10:39:16.000000000 +0100 @@ -1,5 +1,8 @@ /* config.h.in. Generated from configure.ac by autoheader. */ +/* Define if building universal (internal helper macro) */ +#undef AC_APPLE_UNIVERSAL_BUILD + /* Turn debugging mode on? */ #undef DEBUG @@ -81,6 +84,18 @@ /* Version number of package */ #undef VERSION +/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most + significant byte first (like Motorola and SPARC, unlike Intel). */ +#if defined AC_APPLE_UNIVERSAL_BUILD +# if defined __BIG_ENDIAN__ +# define WORDS_BIGENDIAN 1 +# endif +#else +# ifndef WORDS_BIGENDIAN +# undef WORDS_BIGENDIAN +# endif +#endif + /* Enable large inode numbers on Mac OS X 10.5. */ #ifndef _DARWIN_USE_64_BIT_INODE # define _DARWIN_USE_64_BIT_INODE 1 diff -Nru -w libmspack-0.9.1/configure libmspack-0.10.1/configure diff -Nru -w libmspack-0.9.1/configure.ac libmspack-0.10.1/configure.ac --- libmspack-0.9.1/configure.ac 2018-11-06 12:16:17.000000000 +0100 +++ libmspack-0.10.1/configure.ac 2019-03-04 10:36:51.000000000 +0100 @@ -1,7 +1,7 @@ # -*- Autoconf -*- # Process this file with autoconf to produce a configure script. AC_PREREQ(2.59) -AC_INIT([libmspack],[0.9.1alpha],[ky...@cabextract.org.uk]) +AC_INIT([libmspack],[0.10.1alpha],[ky...@cabextract.org.uk]) AC_CONFIG_MACRO_DIR([m4]) AM_INIT_AUTOMAKE([1.11]) AM_SILENT_RULES([yes]) @@ -30,6 +30,7 @@ # Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_C_INLINE +AC_C_BIGENDIAN AC_TYPE_MODE_T AC_TYPE_OFF_T AC_TYPE_SIZE_T diff -Nru -w libmspack-0.9.1/debian/changelog libmspack-0.10.1/debian/changelog --- libmspack-0.9.1/debian/changelog 2018-11-06 14:38:49.000000000 +0100 +++ libmspack-0.10.1/debian/changelog 2019-03-05 11:03:29.000000000 +0100 @@ -1,3 +1,11 @@ +libmspack (0.10.1-1) unstable; urgency=medium + + * New upstream release: + + fix build on big-endian systems (Closes: #914794) + * Add missing JS files for documentation menu and search functions. + + -- Marc Dequènes (Duck) <d...@duckcorp.org> Tue, 05 Mar 2019 19:03:29 +0900 + libmspack (0.9.1-1) unstable; urgency=medium * New upstream release: diff -Nru -w libmspack-0.9.1/debian/copyright libmspack-0.10.1/debian/copyright --- libmspack-0.9.1/debian/copyright 2018-07-29 06:03:32.000000000 +0200 +++ libmspack-0.10.1/debian/copyright 2019-03-05 07:24:35.000000000 +0100 @@ -2,6 +2,8 @@ Upstream-Name: libmspack Upstream-Contact: Stuart Caie <ky...@4u.net> Source: https://www.cabextract.org.uk/libmspack/ +# doxygen-generated doc which is embedded in the upstream tarball; regenerated in the Debian build +Files-excluded: doc/html Files: * diff -Nru -w libmspack-0.9.1/debian/libmspack-doc.docs libmspack-0.10.1/debian/libmspack-doc.docs --- libmspack-0.9.1/debian/libmspack-doc.docs 2018-11-06 14:32:42.000000000 +0100 +++ libmspack-0.10.1/debian/libmspack-doc.docs 2019-03-05 10:52:56.000000000 +0100 @@ -1,4 +1,8 @@ doc/html/*.html doc/html/*.css doc/html/*.png +# jquery.js is generated by doxygen +# there is no easy way to replace it with a symlink to a system common version +# rationale: /usr/share/doc/doxygen/README.jquery +doc/html/*.js doc/html/search diff -Nru -w libmspack-0.9.1/mspack/cabd.c libmspack-0.10.1/mspack/cabd.c --- libmspack-0.9.1/mspack/cabd.c 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/cabd.c 2019-03-04 02:09:26.000000000 +0100 @@ -156,10 +156,10 @@ self->d = NULL; self->error = MSPACK_ERR_OK; - self->param[MSCABD_PARAM_SEARCHBUF] = 32768; - self->param[MSCABD_PARAM_FIXMSZIP] = 0; - self->param[MSCABD_PARAM_DECOMPBUF] = 4096; - self->param[MSCABD_PARAM_SALVAGE] = 0; + self->searchbuf_size = 32768; + self->fix_mszip = 0; + self->buf_size = 4096; + self->salvage = 0; } return (struct mscab_decompressor *) self; } @@ -203,7 +203,7 @@ if ((fh = sys->open(sys, filename, MSPACK_SYS_OPEN_READ))) { if ((cab = (struct mscabd_cabinet_p *) sys->alloc(sys, sizeof(struct mscabd_cabinet_p)))) { cab->base.filename = filename; - error = cabd_read_headers(sys, fh, cab, (off_t) 0, self->param[MSCABD_PARAM_SALVAGE], 0); + error = cabd_read_headers(sys, fh, cab, (off_t) 0, self->salvage, 0); if (error) { cabd_close(base, (struct mscabd_cabinet *) cab); cab = NULL; @@ -600,7 +600,7 @@ sys = self->system; /* allocate a search buffer */ - search_buf = (unsigned char *) sys->alloc(sys, (size_t) self->param[MSCABD_PARAM_SEARCHBUF]); + search_buf = (unsigned char *) sys->alloc(sys, (size_t) self->searchbuf_size); if (!search_buf) { self->error = MSPACK_ERR_NOMEMORY; return NULL; @@ -649,7 +649,7 @@ struct mspack_system *sys = self->system; unsigned char *p, *pend, state = 0; unsigned int cablen_u32 = 0, foffset_u32 = 0; - int false_cabs = 0, salvage = self->param[MSCABD_PARAM_SALVAGE]; + int false_cabs = 0; #if !LARGEFILE_SUPPORT /* detect 32-bit off_t overflow */ @@ -664,8 +664,8 @@ /* search length is either the full length of the search buffer, or the * amount of data remaining to the end of the file, whichever is less. */ length = flen - offset; - if (length > self->param[MSCABD_PARAM_SEARCHBUF]) { - length = self->param[MSCABD_PARAM_SEARCHBUF]; + if (length > self->searchbuf_size) { + length = self->searchbuf_size; } /* fill the search buffer with data from disk */ @@ -729,14 +729,14 @@ * mode, don't check the alleged length, allow it to be garbage */ if ((foffset_u32 < cablen_u32) && ((caboff + (off_t) foffset_u32) < (flen + 32)) && - (((caboff + (off_t) cablen_u32) < (flen + 32)) || salvage)) + (((caboff + (off_t) cablen_u32) < (flen + 32)) || self->salvage)) { /* likely cabinet found -- try reading it */ if (!(cab = (struct mscabd_cabinet_p *) sys->alloc(sys, sizeof(struct mscabd_cabinet_p)))) { return MSPACK_ERR_NOMEMORY; } cab->base.filename = filename; - if (cabd_read_headers(sys, fh, cab, caboff, salvage, 1)) { + if (cabd_read_headers(sys, fh, cab, caboff, self->salvage, 1)) { /* destroy the failed cabinet */ cabd_close((struct mscab_decompressor *) self, (struct mscabd_cabinet *) cab); @@ -1030,7 +1030,7 @@ */ filelen = file->length; if (filelen > CAB_LENGTHMAX || (file->offset + filelen) > CAB_LENGTHMAX) { - if (self->param[MSCABD_PARAM_SALVAGE]) { + if (self->salvage) { filelen = CAB_LENGTHMAX - file->offset; } else { @@ -1048,7 +1048,7 @@ /* if file goes beyond what can be decoded, given an error. * In salvage mode, don't assume block sizes, just try decoding */ - if (!self->param[MSCABD_PARAM_SALVAGE]) { + if (!self->salvage) { off_t maxlen = fol->base.num_blocks * CAB_BLOCKMAX; if ((file->offset + filelen) > maxlen) { sys->message(NULL, "ERROR; file \"%s\" cannot be extracted, " @@ -1165,24 +1165,22 @@ switch (ct & cffoldCOMPTYPE_MASK) { case cffoldCOMPTYPE_NONE: self->d->decompress = (int (*)(void *, off_t)) &noned_decompress; - self->d->state = noned_init(&self->d->sys, fh, fh, - self->param[MSCABD_PARAM_DECOMPBUF]); + self->d->state = noned_init(&self->d->sys, fh, fh, self->buf_size); break; case cffoldCOMPTYPE_MSZIP: self->d->decompress = (int (*)(void *, off_t)) &mszipd_decompress; - self->d->state = mszipd_init(&self->d->sys, fh, fh, - self->param[MSCABD_PARAM_DECOMPBUF], - self->param[MSCABD_PARAM_FIXMSZIP]); + self->d->state = mszipd_init(&self->d->sys, fh, fh, self->buf_size, + self->fix_mszip); break; case cffoldCOMPTYPE_QUANTUM: self->d->decompress = (int (*)(void *, off_t)) &qtmd_decompress; self->d->state = qtmd_init(&self->d->sys, fh, fh, (int) (ct >> 8) & 0x1f, - self->param[MSCABD_PARAM_DECOMPBUF]); + self->buf_size); break; case cffoldCOMPTYPE_LZX: self->d->decompress = (int (*)(void *, off_t)) &lzxd_decompress; self->d->state = lzxd_init(&self->d->sys, fh, fh, (int) (ct >> 8) & 0x1f, 0, - self->param[MSCABD_PARAM_DECOMPBUF], (off_t)0,0); + self->buf_size, (off_t)0,0); break; default: return self->error = MSPACK_ERR_DATAFORMAT; @@ -1221,10 +1219,10 @@ struct mspack_system *sys = self->system; int avail, todo, outlen, ignore_cksum, ignore_blocksize; - ignore_cksum = self->param[MSCABD_PARAM_SALVAGE] || - (self->param[MSCABD_PARAM_FIXMSZIP] && + ignore_cksum = self->salvage || + (self->fix_mszip && ((self->d->comp_type & cffoldCOMPTYPE_MASK) == cffoldCOMPTYPE_MSZIP)); - ignore_blocksize = self->param[MSCABD_PARAM_SALVAGE]; + ignore_blocksize = self->salvage; todo = bytes; while (todo > 0) { @@ -1244,7 +1242,7 @@ /* check if we're out of input blocks, advance block counter */ if (self->d->block++ >= self->d->folder->base.num_blocks) { - if (!self->param[MSCABD_PARAM_SALVAGE]) { + if (!self->salvage) { self->read_error = MSPACK_ERR_DATAFORMAT; } else { @@ -1481,17 +1479,17 @@ switch (param) { case MSCABD_PARAM_SEARCHBUF: if (value < 4) return MSPACK_ERR_ARGS; - self->param[MSCABD_PARAM_SEARCHBUF] = value; + self->searchbuf_size = value; break; case MSCABD_PARAM_FIXMSZIP: - self->param[MSCABD_PARAM_FIXMSZIP] = value; + self->fix_mszip = value; break; case MSCABD_PARAM_DECOMPBUF: if (value < 4) return MSPACK_ERR_ARGS; - self->param[MSCABD_PARAM_DECOMPBUF] = value; + self->buf_size = value; break; case MSCABD_PARAM_SALVAGE: - self->param[MSCABD_PARAM_SALVAGE] = value; + self->salvage = value; break; default: return MSPACK_ERR_ARGS; diff -Nru -w libmspack-0.9.1/mspack/cab.h libmspack-0.10.1/mspack/cab.h --- libmspack-0.9.1/mspack/cab.h 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/cab.h 2019-03-04 10:09:35.000000000 +0100 @@ -113,7 +113,7 @@ struct mscab_decompressor base; struct mscabd_decompress_state *d; struct mspack_system *system; - int param[4]; /* !!! MATCH THIS TO NUM OF PARAMS IN MSPACK.H !!! */ + int buf_size, searchbuf_size, fix_mszip, salvage; /* params */ int error, read_error; }; diff -Nru -w libmspack-0.9.1/mspack/chmd.c libmspack-0.10.1/mspack/chmd.c --- libmspack-0.9.1/mspack/chmd.c 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/chmd.c 2019-03-04 02:10:28.000000000 +0100 @@ -44,7 +44,7 @@ struct mschm_decompressor_p *self, struct mschmd_file *file); static int read_reset_table( struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, - int entry, off_t *length_ptr, off_t *offset_ptr); + unsigned int entry, off_t *length_ptr, off_t *offset_ptr); static int read_spaninfo( struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, off_t *length_ptr); @@ -381,14 +381,18 @@ D(("more than 100,000 chunks")) return MSPACK_ERR_DATAFORMAT; } + if (chm->chunk_size > 8192) { + D(("chunk size over 8192 (get in touch if this is valid)")) + return MSPACK_ERR_DATAFORMAT; + } if ((off_t)chm->chunk_size * (off_t)chm->num_chunks > chm->length) { D(("chunks larger than entire file")) return MSPACK_ERR_DATAFORMAT; } /* common sense checks on header section 1 fields */ - if ((chm->chunk_size & (chm->chunk_size - 1)) != 0) { - sys->message(fh, "WARNING; chunk size is not a power of two"); + if (chm->chunk_size != 4096) { + sys->message(fh, "WARNING; chunk size is not 4096"); } if (chm->first_pmgl != 0) { sys->message(fh, "WARNING; first PMGL chunk is not zero"); @@ -435,7 +439,7 @@ sys->message(fh, "WARNING; PMGL quickref area is too small"); } if (EndGetI32(&chunk[pmgl_QuickRefSize]) > - ((int)chm->chunk_size - pmgl_Entries)) + (chm->chunk_size - pmgl_Entries)) { sys->message(fh, "WARNING; PMGL quickref area is too large"); } @@ -483,20 +487,18 @@ if (name[0] == ':' && name[1] == ':') { /* system file */ - if (memcmp(&name[2], &content_name[2], 31L) == 0) { - if (memcmp(&name[33], &content_name[33], 8L) == 0) { + if (name_len == 40 && memcmp(name, content_name, 40) == 0) { chm->sec1.content = fi; } - else if (memcmp(&name[33], &control_name[33], 11L) == 0) { + else if (name_len == 44 && memcmp(name, control_name, 44) == 0) { chm->sec1.control = fi; } - else if (memcmp(&name[33], &spaninfo_name[33], 8L) == 0) { + else if (name_len == 41 && memcmp(name, spaninfo_name, 41) == 0) { chm->sec1.spaninfo = fi; } - else if (memcmp(&name[33], &rtable_name[33], 72L) == 0) { + else if (name_len == 105 && memcmp(name, rtable_name, 105) == 0) { chm->sec1.rtable = fi; } - } fi->next = chm->sysfiles; chm->sysfiles = fi; } @@ -589,7 +591,7 @@ } /* stop simple infinite loops: can't visit the same chunk twice */ - if ((int)n == EndGetI32(&chunk[pmgl_NextChunk])) { + if (n == EndGetI32(&chunk[pmgl_NextChunk])) { break; } } @@ -1154,7 +1156,8 @@ */ static int read_reset_table(struct mschm_decompressor_p *self, struct mschmd_sec_mscompressed *sec, - int entry, off_t *length_ptr, off_t *offset_ptr) + unsigned int entry, + off_t *length_ptr, off_t *offset_ptr) { struct mspack_system *sys = self->system; unsigned char *data; diff -Nru -w libmspack-0.9.1/mspack/mspack.h libmspack-0.10.1/mspack/mspack.h --- libmspack-0.9.1/mspack/mspack.h 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/mspack.h 2019-02-18 21:25:16.000000000 +0100 @@ -1,5 +1,5 @@ /* libmspack -- a library for working with Microsoft compression formats. - * (C) 2003-2016 Stuart Caie <ky...@cabextract.org.uk> + * (C) 2003-2019 Stuart Caie <ky...@cabextract.org.uk> * * libmspack is free software; you can redistribute it and/or modify it under * the terms of the GNU Lesser General Public License (LGPL) version 2.1 @@ -1554,7 +1554,7 @@ */ int (*set_param)(struct mschm_compressor *self, int param, - unsigned int value); + int value); /** * Returns the error code set by the most recently called method. @@ -1853,7 +1853,7 @@ */ int (*set_param)(struct msszdd_compressor *self, int param, - unsigned int value); + int value); /** * Returns the error code set by the most recently called method. @@ -2091,7 +2091,7 @@ */ int (*set_param)(struct mskwaj_compressor *self, int param, - unsigned int value); + int value); /** @@ -2353,8 +2353,31 @@ const char *input, const char *base, const char *output); + + /** + * Sets an OAB decompression engine parameter. Available only in OAB + * decompressor version 2 and above. + * + * - #MSOABD_PARAM_DECOMPBUF: How many bytes should be used as an input + * buffer by decompressors? The minimum value is 16. The default value + * is 4096. + * + * @param self a self-referential pointer to the msoab_decompressor + * instance being called + * @param param the parameter to set + * @param value the value to set the parameter to + * @return MSPACK_ERR_OK if all is OK, or MSPACK_ERR_ARGS if there + * is a problem with either parameter or value. + */ + int (*set_param)(struct msoab_decompressor *self, + int param, + int value); + }; +/** msoab_decompressor::set_param() parameter: size of decompression buffer */ +#define MSOABD_PARAM_DECOMPBUF (0) + #ifdef __cplusplus } #endif diff -Nru -w libmspack-0.9.1/mspack/oabd.c libmspack-0.10.1/mspack/oabd.c --- libmspack-0.9.1/mspack/oabd.c 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/oabd.c 2019-03-04 10:11:49.000000000 +0100 @@ -33,6 +33,11 @@ static int oabd_decompress_incremental(struct msoab_decompressor *self, const char *input, const char *base, const char *output); +static int oabd_param(struct msoab_decompressor *base, int param, int value); +static int copy_fh(struct mspack_system *sys, struct mspack_file *infh, + struct mspack_file *outfh, size_t bytes_to_copy, + unsigned char *buf, int buf_size); + struct msoab_decompressor * mspack_create_oab_decompressor(struct mspack_system *sys) @@ -45,7 +50,9 @@ if ((self = (struct msoab_decompressor_p *) sys->alloc(sys, sizeof(struct msoab_decompressor_p)))) { self->base.decompress = &oabd_decompress; self->base.decompress_incremental = &oabd_decompress_incremental; + self->base.set_param = &oabd_param; self->system = sys; + self->buf_size = 4096; } return (struct msoab_decompressor *) self; } @@ -132,17 +139,13 @@ block_max = EndGetI32(&hdrbuf[oabhead_BlockMax]); target_size = EndGetI32(&hdrbuf[oabhead_TargetSize]); - /* We use it for reading block headers too */ - if (block_max < oabblk_SIZEOF) - block_max = oabblk_SIZEOF; - outfh = sys->open(sys, output, MSPACK_SYS_OPEN_WRITE); if (!outfh) { ret = MSPACK_ERR_OPEN; goto out; } - buf = sys->alloc(sys, block_max); + buf = sys->alloc(sys, self->buf_size); if (!buf) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -181,14 +184,8 @@ ret = MSPACK_ERR_DATAFORMAT; goto out; } - if (sys->read(infh, buf, blk_dsize) != (int)blk_dsize) { - ret = MSPACK_ERR_READ; - goto out; - } - if (sys->write(outfh, buf, blk_dsize) != (int)blk_dsize) { - ret = MSPACK_ERR_WRITE; - goto out; - } + ret = copy_fh(sys, infh, outfh, blk_dsize, buf, self->buf_size); + if (ret) goto out; } else { /* LZX compressed block */ window_bits = 17; @@ -200,7 +197,7 @@ out_ofh.crc = 0xffffffff; lzx = lzxd_init(&oabd_sys, (void *)&in_ofh, (void *)&out_ofh, window_bits, - 0, 4096, blk_dsize, 1); + 0, self->buf_size, blk_dsize, 1); if (!lzx) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -214,18 +211,8 @@ lzx = NULL; /* Consume any trailing padding bytes before the next block */ - while (in_ofh.available) { - int count = block_max; - if ((size_t)count > in_ofh.available) - count = in_ofh.available; - - count = sys->read(infh, buf, count); - if (count < 0) { - ret = MSPACK_ERR_READ; - goto out; - } - in_ofh.available -= count; - } + ret = copy_fh(sys, infh, NULL, in_ofh.available, buf, self->buf_size); + if (ret) goto out; if (out_ofh.crc != blk_crc) { ret = MSPACK_ERR_CHECKSUM; @@ -301,7 +288,7 @@ goto out; } - buf = sys->alloc(sys, block_max); + buf = sys->alloc(sys, self->buf_size); if (!buf) { ret = MSPACK_ERR_NOMEMORY; goto out; @@ -364,18 +351,8 @@ lzx = NULL; /* Consume any trailing padding bytes before the next block */ - while (in_ofh.available) { - int count = block_max; - if ((size_t)count > in_ofh.available) - count = in_ofh.available; - - count = sys->read(infh, buf, count); - if (count < 0) { - ret = MSPACK_ERR_READ; - goto out; - } - in_ofh.available -= count; - } + ret = copy_fh(sys, infh, NULL, in_ofh.available, buf, self->buf_size); + if (ret) goto out; if (out_ofh.crc != blk_crc) { ret = MSPACK_ERR_CHECKSUM; @@ -394,3 +371,33 @@ return ret; } + +static int copy_fh(struct mspack_system *sys, struct mspack_file *infh, + struct mspack_file *outfh, size_t bytes_to_copy, + unsigned char *buf, int buf_size) +{ + while (bytes_to_copy) { + int run = buf_size; + if ((size_t) run > bytes_to_copy) { + run = (int) bytes_to_copy; + } + if (sys->read(infh, buf, run) != run) { + return MSPACK_ERR_READ; + } + if (outfh && sys->write(outfh, buf, run) != run) { + return MSPACK_ERR_WRITE; + } + bytes_to_copy -= run; + } + return MSPACK_ERR_OK; +} + +static int oabd_param(struct msoab_decompressor *base, int param, int value) { + struct msoab_decompressor_p *self = (struct msoab_decompressor_p *) base; + if (self && param == MSOABD_PARAM_DECOMPBUF && value >= 16) { + /* must be at least 16 bytes (patchblk_SIZEOF, oabblk_SIZEOF) */ + self->buf_size = value; + return MSPACK_ERR_OK; + } + return MSPACK_ERR_ARGS; +} diff -Nru -w libmspack-0.9.1/mspack/oab.h libmspack-0.10.1/mspack/oab.h --- libmspack-0.9.1/mspack/oab.h 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/oab.h 2019-02-18 17:39:40.000000000 +0100 @@ -27,6 +27,7 @@ struct msoab_decompressor_p { struct msoab_decompressor base; struct mspack_system *system; + int buf_size; /* todo */ }; diff -Nru -w libmspack-0.9.1/mspack/system.c libmspack-0.10.1/mspack/system.c --- libmspack-0.9.1/mspack/system.c 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/system.c 2019-02-18 17:26:17.000000000 +0100 @@ -31,12 +31,15 @@ * - added MSCABD_PARAM_SALVAGE */ case MSPACK_VER_MSCABD: + /* OAB decoder version 1 -> 2 changes: + * - added msoab_decompressor::set_param and MSOABD_PARAM_DECOMPBUF + */ + case MSPACK_VER_MSOABD: return 2; case MSPACK_VER_LIBRARY: case MSPACK_VER_SYSTEM: case MSPACK_VER_MSSZDDD: case MSPACK_VER_MSKWAJD: - case MSPACK_VER_MSOABD: return 1; case MSPACK_VER_MSCABC: case MSPACK_VER_MSCHMC: diff -Nru -w libmspack-0.9.1/mspack/system.h libmspack-0.10.1/mspack/system.h --- libmspack-0.9.1/mspack/system.h 2018-11-05 12:53:09.000000000 +0100 +++ libmspack-0.10.1/mspack/system.h 2019-03-04 02:10:28.000000000 +0100 @@ -81,20 +81,19 @@ #endif /* endian-neutral reading of little-endian data */ -#define __egi32(a,n) ( ((((unsigned char *) a)[n+3]) << 24) | \ - ((((unsigned char *) a)[n+2]) << 16) | \ - ((((unsigned char *) a)[n+1]) << 8) | \ - ((((unsigned char *) a)[n+0]))) -#define EndGetI64(a) ((((unsigned long long int) __egi32(a,4)) << 32) | \ - ((unsigned int) __egi32(a,0))) +#define __egi32(a,n) (((unsigned int) ((unsigned char *)(a))[n+3] << 24) | \ + ((unsigned int) ((unsigned char *)(a))[n+2] << 16) | \ + ((unsigned int) ((unsigned char *)(a))[n+1] << 8) | \ + ((unsigned int) ((unsigned char *)(a))[n])) +#define EndGetI64(a) (((unsigned long long int) __egi32(a,4) << 32) | __egi32(a,0)) #define EndGetI32(a) __egi32(a,0) #define EndGetI16(a) ((((a)[1])<<8)|((a)[0])) /* endian-neutral reading of big-endian data */ -#define EndGetM32(a) (((((unsigned char *) a)[0]) << 24) | \ - ((((unsigned char *) a)[1]) << 16) | \ - ((((unsigned char *) a)[2]) << 8) | \ - ((((unsigned char *) a)[3]))) +#define EndGetM32(a) (((unsigned int) ((unsigned char *)(a))[0] << 24) | \ + ((unsigned int) ((unsigned char *)(a))[1] << 16) | \ + ((unsigned int) ((unsigned char *)(a))[2] << 8) | \ + ((unsigned int) ((unsigned char *)(a))[3])) #define EndGetM16(a) ((((a)[0])<<8)|((a)[1])) extern struct mspack_system *mspack_default_system; diff -Nru -w libmspack-0.9.1/README libmspack-0.10.1/README --- libmspack-0.9.1/README 2018-11-06 12:16:32.000000000 +0100 +++ libmspack-0.10.1/README 2019-03-04 10:36:57.000000000 +0100 @@ -1,4 +1,4 @@ -libmspack 0.9.1alpha +libmspack 0.10.1alpha The purpose of libmspack is to provide compressors and decompressors, archivers and dearchivers for Microsoft compression formats: CAB, CHM, WIM,
