I have decided to demonstrate where I am with the stress test
application. A small project to add command-line support using Picocli
ballooned into a lot of new code. Here's a PR with the new code:
https://github.com/apache/commons-rng/pull/34
Everything is still in the examples-stress module. All new files.
However the output is a command-line program that can be extended to do
other things. Perhaps this justifies renaming the module to
examples-rng-utils or creating a new module in the same vein that can be
used for utilities and this one dedicated to the stress test. One such
utility previously discussed was to produce output from a named
generator for a chosen method, e.g. nextLong, nextInt, nextDouble, etc.
Back to the application as it stands. It achieves the goal of:
- Having command-line help
- Listing all known generators
- Running a stress test for a list of generators, passing the output to
an external application (e.g. BigCrush)
- Allowing arguments to be passed to generators that need them, e.g.
TWO_CMRES_SELECT
- Allowing a variable number of stress test trials to be run
- Options for overwriting existing results files
- Support for big/little-endian platforms (this is natively detected)
- Other stuff too
Starting in the root directory:
cd commons-rng-examples/examples-stress
gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 \
-ltestu01probdist -ltestu01mylib -lm
mvn package -P examples-stress
java -jar target/rng-utils.jar
rng-utils [-hV] [--logging=<logLevel>] [COMMAND]
Apache Commons Random Number Generators Utilities.
Options:
--logging=<logLevel> Specify the logging level (default: INFO).
Valid values: ERROR, INFO, DEBUG
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
Commands:
stress Run repeat trials of random data generators using a provided test
application.
list List random generators.
java -jar target/rng-utils.jar list -h
rng-utils list [-hV] [--logging=<logLevel>] [-f=<format>] [-p=<idPrefix>]
[-t=<trials>]
List random generators.
Options:
--logging=<logLevel> Specify the logging level (default: INFO).
Valid values: ERROR, INFO, DEBUG
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
-f, --format=<format> The list format (default: STRESS_TEST).
Valid values: STRESS_TEST, PLAIN
-p, --prefix=<idPrefix> The ID prefix.
Used for the stress test format.
-t, --trials=<trials> The number of trials for each random generator.
Used for the stress test format.
java -jar target/rng-utils.jar list
# Random generators list.
# Any generator with no trials is ignored during testing.
#
# ID RandomSource trials [constructor arguments ...]
1 JDK 1
2 WELL_512_A 1
3 WELL_1024_A 1
4 WELL_19937_A 1
5 WELL_19937_C 1
6 WELL_44497_A 1
7 WELL_44497_B 1
8 MT 1
9 ISAAC 1
10 SPLIT_MIX_64 1
11 XOR_SHIFT_1024_S 1
12 TWO_CMRES 1
13 TWO_CMRES_SELECT 0 [1, 2]
14 MT_64 1
15 MWC_256 1
16 KISS 1
17 XOR_SHIFT_1024_S_PHI 1
java -jar target/rng-utils.jar stress -h
rng-utils stress [-hrV] [--dry-run] [--logging=<logLevel>]
[--prefix=<fileOutputPrefix>] [-b=<byteOrder>] [-l=<genList>]
[-n=<taskCount>] [-o=<outputMode>] [-t=<trials>] <executable>
[<argument>...]
Run repeat trials of random data generators using a provided test application.
Data is transferred to the application sub-process via standard input.
Parameters:
<executable> The stress test executable.
[<argument>...] The arguments to pass to the executable.
Options:
--logging=<logLevel> Specify the logging level (default: INFO).
Valid values: ERROR, INFO, DEBUG
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
--prefix=<fileOutputPrefix>
Results file prefix (default: stress_).
-o, --output-mode=<outputMode>
Output mode for existing files (default: ERROR).
Valid values: ERROR, SKIP, APPEND, OVERWRITE
-l, --list=<genList> List of random generators.
The default list is all known generators.
-t, --trials=<trials> The number of trials for each random generator.
Used only for the default list (default: 1).
-n, --tasks=<taskCount> Number of concurrent tasks (default: 4).
Two threads are required per task.
-b, --byte-order=<byteOrder>
Byte-order of the transferred data (default:
LITTLE_ENDIAN).
Valid values: BIG_ENDIAN, LITTLE_ENDIAN.
-r, --reverse-bits Reverse the bits in the data (default: false).
Note: Generators may fail tests for a reverse
sequence
when passing using the standard sequence.
--dry-run Perform a dry run where the generators and output
files
are created but the stress test is not executed.
java -jar target/rng-utils.jar stress ./stdin2testu01 SmallCrush
[INFO] Running stress test ...
[INFO] Progress 0 / 16 (0.00%)
[INFO] Progress 1 / 16 (6.25%)
[INFO] Progress 2 / 16 (12.50%)
[INFO] Progress 3 / 16 (18.75%)
[INFO] Progress 4 / 16 (25.00%)
[INFO] Progress 5 / 16 (31.25%)
[INFO] Progress 7 / 16 (43.75%)
[INFO] Progress 9 / 16 (56.25%)
[INFO] Progress 10 / 16 (62.50%)
[INFO] Progress 12 / 16 (75.00%)
[INFO] Progress 13 / 16 (81.25%)
[INFO] Progress 14 / 16 (87.50%)
[INFO] Progress 15 / 16 (93.75%)
[INFO] Progress 16 / 16 (100%)
Output files will be in 'stress_X_N' where X is the identifier for the
RNG and N is the trial number. The default identifier X is the order in
the RandomSource enum.
I have updated the results files to have some more information in the
header and footer including: start and end times; the byte/bit reversed
state is prepended to the RNG name; the exit value of the sub-process;
the count of numbers used for the test:
grep '^#' stress_4_1
#
# RNG: Byte-reversed org.apache.commons.rng.core.source32.Well19937a
#
# Java: 1.8.0_191
# Runtime: 1.8.0_191-8u191-b12-2ubuntu0.16.04.1-b12
# JVM: OpenJDK 64-Bit Server VM 25.191-b12
# OS: Linux 4.4.0-143-generic amd64
#
# Analyzer:
/home/ah403/git/commons-rng/commons-rng-examples/examples-stress/stdin2testu01
SmallCrush
#
# Start: 2019-04-03 16:40:30
#
#
# End: 2019-04-03 16:40:55
#
# Exit value: 0
# Numbers used: 226897920 >= 2^27 (907.6 MB)
#
# Test duration: 0.42309478473333334 minutes
#
So the current stress test results for all generators can be achieved using:
java -jar target/rng-utils.jar stress --trials 3 --prefix tu_ ./stdin2testu01
BigCrush
java -jar target/rng-utils.jar stress --trials 3 --prefix dh_
/usr/bin/dieharder -a -g 200 -Y 1 -k 2
Please try it out using the above PR and provide some feedback.
Alex
