Recently I was working on my own project where I had to create a port that packs large binary files (20MB). I found that for some reason "make package" (or more likely pkg_create) would always fail, where it would not be able to create /usr/ports/packages/i386/all/port-1.0.0.tgz.
With some experimentation, I found that if the binary file was large
enough at a certain size, "make package" would not produce the
package. I tried searching the man pages, mailing list archives,
and source code of pkg_create and the various *.pm files but could not
find anything that says that there is a maximum file size limit that
"make package" or pkg_create would accept. So I am sending this email
to ports@ to see if anyone knows what the cause might be and whether
this behavior can be reproduced on other systems.
This is on the OpenBSD/i386 December 9 16:09:40 MST snapshot running
on VMware Fusion 3.
To illustrate, I have attached a test port that creates a dummy test
file from /dev/arandom whose size is determined by the ${BS}
(block size) variable, currently set to 10196840 bytes. For
easier discussion I have included the Makefile inline below.
[BEGIN]
# $OpenBSD$
COMMENT = test port
DISTNAME = test-1.0.0
CATEGORIES = net
MASTER_SITES = ${HOMEPAGE}
HOMEPAGE = http://www.openbsd.org/
MAINTAINER = Lawrence Teo <[email protected]>
# BSD
PERMIT_PACKAGE_CDROM= Yes
PERMIT_PACKAGE_FTP= Yes
PERMIT_DISTFILES_CDROM= Yes
PERMIT_DISTFILES_FTP= Yes
DISTFILES=
NO_CHECKSUM= Yes
EXTRACT_ONLY=
# Generate a file with this number of bytes
BS?= 10196840
do-build:
dd if=/dev/arandom of=${WRKSRC}/testfile bs=${BS} count=1
do-install:
${INSTALL_DATA_DIR} ${PREFIX}/share/test
${INSTALL_DATA} ${WRKSRC}/testfile ${PREFIX}/share/test
NO_REGRESS= Yes
.include <bsd.port.mk>
[END]
The port creates that 10196840-byte file and its pkg/PLIST just
packs that single file.
What is strange is that, sometimes "make package" works as you can
see below:
# make clean ; make clean=package; make package
===> Cleaning for test-1.0.0
===> Cleaning for test-1.0.0
rm -f /usr/ports/packages/i386/all/test-1.0.0.tgz
/usr/ports/packages/i386/ftp/test-1.0.0.tgz
/usr/ports/packages/i386/cdrom/test-1.0.0.tgz
===> Checking files for test-1.0.0
===> Extracting for test-1.0.0
===> Patching for test-1.0.0
===> Configuring for test-1.0.0
===> Building for test-1.0.0
dd if=/dev/arandom of=/usr/ports/pobj/test-1.0.0/test-1.0.0/testfile
bs=10196840 count=1
1+0 records in
1+0 records out
10196840 bytes transferred in 0.178 secs (57161661 bytes/sec)
===> Faking installation for test-1.0.0
install -d -o root -g bin -m 755
/usr/ports/pobj/test-1.0.0/fake-i386/usr/local/share/test
install -c -o root -g bin -m 444 /usr/ports/pobj/test-1.0.0/test-1.0.0/testfile
/usr/ports/pobj/test-1.0.0/fake-i386/usr/local/share/test
===> Building package for test-1.0.0
Create /usr/ports/packages/i386/all/test-1.0.0.tgz
Link to /usr/ports/packages/i386/ftp/test-1.0.0.tgz
Link to /usr/ports/packages/i386/cdrom/test-1.0.0.tgz
But sometimes it does not:
# make clean ; make clean=package ; make package
===> Cleaning for test-1.0.0
===> Cleaning for test-1.0.0
rm -f /usr/ports/packages/i386/all/test-1.0.0.tgz
/usr/ports/packages/i386/ftp/test-1.0.0.tgz
/usr/ports/packages/i386/cdrom/test-1.0.0.tgz
===> Checking files for test-1.0.0
===> Extracting for test-1.0.0
===> Patching for test-1.0.0
===> Configuring for test-1.0.0
===> Building for test-1.0.0
dd if=/dev/arandom of=/usr/ports/pobj/test-1.0.0/test-1.0.0/testfile
bs=10196840 count=1
1+0 records in
1+0 records out
10196840 bytes transferred in 0.168 secs (60350616 bytes/sec)
===> Faking installation for test-1.0.0
install -d -o root -g bin -m 755
/usr/ports/pobj/test-1.0.0/fake-i386/usr/local/share/test
install -c -o root -g bin -m 444 /usr/ports/pobj/test-1.0.0/test-1.0.0/testfile
/usr/ports/pobj/test-1.0.0/fake-i386/usr/local/share/test
===> Building package for test-1.0.0
Create /usr/ports/packages/i386/all/test-1.0.0.tgz
chown: /usr/ports/packages/i386/all/test-1.0.0.tgz: No such file or directory
Link to /usr/ports/packages/i386/ftp/test-1.0.0.tgz
cp: /usr/ports/packages/i386/all/test-1.0.0.tgz: No such file or directory
*** Error code 1
Stop in /usr/ports/mystuff/net/test (line 2410 of
/usr/ports/infrastructure/mk/bsd.port.mk).
*** Error code 1
Stop in /usr/ports/mystuff/net/test (line 1988 of
/usr/ports/infrastructure/mk/bsd.port.mk).
To test how often it fails, I wrote a test script that ran
"make package" 10 times on this port and collect the results:
[BEGIN]
#! /bin/sh
BS=10196840
for i in $(jot 10 1 10)
do
make clean
make clean=package
make package BS=${BS}
ret=$?
echo Try $i - exit code $ret >>results.txt
done
[END]
Here are the results:
# cat results.txt
Try 1 - exit code 0
Try 2 - exit code 0
Try 3 - exit code 0
Try 4 - exit code 1
Try 5 - exit code 0
Try 6 - exit code 0
Try 7 - exit code 0
Try 8 - exit code 1
Try 9 - exit code 0
Try 10 - exit code 0
So it fails approximately 20% of the time with that ${BS} value.
The bigger the ${BS} value, the more likely it will fail. For
example, if you set it to 20MB it will fail consistently.
Does anyone know what could be causing this and whether this is
expected behavior?
Thank you,
Lawrence
test.tar.gz
Description: Binary data
