'Overall fuzzer requirements' chapter contains the current product vision and features done and to be done. This chapter is still in progress.
Signed-off-by: Maria Kustova <mari...@catit.be> --- tests/image-fuzzer/docs/image-fuzzer.txt | 176 +++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 tests/image-fuzzer/docs/image-fuzzer.txt diff --git a/tests/image-fuzzer/docs/image-fuzzer.txt b/tests/image-fuzzer/docs/image-fuzzer.txt new file mode 100644 index 0000000..a9ed4b6 --- /dev/null +++ b/tests/image-fuzzer/docs/image-fuzzer.txt @@ -0,0 +1,176 @@ +Image fuzzer +============ + +Description +----------- + +The goal of the image fuzzer is to catch crashes of qemu-io/qemu-img providing +to them randomly corrupted images. +Test images are generated from scratch and have valid inner structure with some +elements, e.g. L1/L2 tables, having random invalid values. + + +Test runner +----------- + +The test runner generates test images, executes tests utilizing generated +images, indicates their results and collect all test related artifacts (logs, +core dumps, test images). +The test means one start of a system under test (SUT), e.g. qemu-io, with +specified arguments and one test image. +By default, the test runner generates new tests and executes them till +keyboard interruption. But if a test seed is specified via '-s' runner +parameter, then only one test with this seed will be executed, after its finish +the runner will exit. + +The runner uses an external image fuzzer to generate test images. An image +generator should be specified as a mandatory parameter of the test runner. +Details about interactions between the runner and fuzzers see "Module +interfaces". + +The runner activates generation of core dumps during test executions, but it +assumes that core dumps will be generated in the current working directory. +For comprehensive test results, please, set up your test environment +properly. + +Path to a SUT can be specified via environment variables (for now only +qemu-img) or via '--binary' parameter of the test runner. For details about +environment variables see qemu-iotests/check. + + +Qcow2 image generator +--------------------- + +The 'qcow2' generator is a Python package providing 'create_image' method as +a single public API. See details in 'Test runner/image fuzzer' in 'Module +interfaces'. + +Qcow2 contains two submodules: fuzz.py and layout.py. + +'fuzz.py' contains all fuzzing functions one per image field. It's assumed that +after code analysis every field will have own constraints for its value. For +now only universal potentially dangerous values are used, e.g. type limits for +integers or unsafe symbols as '%s' for strings. For bitmasks random amount of +bits are set to ones. All fuzzed values are checked on non-equality to the +current valid value of the field. In case of equality the value will be +regenerated. + +'layout.py' creates a random valid image, fuzzes a random subset of the image +fields by 'fuzz.py' module and writes a fuzzed image to the file specified. + +For now only header fields and header extensions are generated, the remaining +file is filled with zeros. + + +Module interfaces +----------------- + +* Test runner/image fuzzer + +The runner calls an image generator specifying path to a test image file. +An image generator is expected to provide 'create_image(test_img_path)' method +that creates a test image and writes it to the specified file. The file should +be created if it doesn't exist or overwritten otherwise. +Random seed is set by the runner at every test execution for the regression +purpose, so an image generator is not recommended to modify it internally. + +* Test runner/SUT + +A full test command is composed from the SUT, its arguments specified +via '-c' runner parameter and the name of generated image. To indicate where +a test image name should be placed TEST_IMG placeholder can be used. +For example, for the next test command + + qemu-img convert -O vmdk fuzzed_image test.vmdk + +a call of the image fuzzer will be following: + + ./runner.py -b qemu-img -c 'convert -O vmdk TEST_IMG test.vmdk' work_dir qcow2 + + +Overall fuzzer requirements +=========================== + +Input data: +---------- + + - image template (generator) + - work directory + - test run duration (optional) + - action vector (optional) + - seed (optional) + - SUT and its arguments (optional) + + +Fuzzer requirements: +------------------- + +1. Should be able to inject random data +2. Should be able to select a random value from the manually pregenerated + vector (boundary values, e.g. max/min cluster size) +3. Image template should describe a general structure invariant for all + test images (image format description) +4. Image template should be autonomous and other fuzzer parts should not + relate on it +5. Image template should contain reference rules (not only block+size + description) +6. Should generate the test image with the correct structure based on an image + template +7. Should accept a seed as an argument (for regression purpose) +8. Should generate a seed if it is not specified as an input parameter. +9. The same seed should generate the same image, if no or the same action + vector is specified +10. Should accept a vector of actions as an argument (for test reproducing and + for test case specification, e.g. group of tests for header structure, + group of test for snapshots, etc) +11. Action vector should be randomly generated from the pool of available + actions, if it is not specified as an input parameter +12. Pool of actions should be defined automatically based on an image template +13. Should accept a SUT and its call parameters as an argument or select them + randomly otherwise. As far as it's expected to be rarely changed, the list + of all possible test commands can be available in the test runner + internally. +14. Should accept a test run duration as an argument. Tests should be executed + during a minimum period from a test run duration and time while fuzzer can + generate different test images +15. Should support an external cancellation of a test run +16. Seed and action vector should be logged (for regression purpose) +17. All files related to test result should be collected: a test image, + SUT logs, fuzzer logs and crash dumps +18. Should be compatible with python version 2.4-2.7 +19. Usage of external libraries should be limited as much as possible. + + +Image formats: +------------- + +Main target image format is qcow2, but support of image templates should +provide an ability to add any other image format. + + +Effectiveness: +------------- + +Fuzzer can be controlled via template, seed and action vector; +it makes the fuzzer itself invariant to an image format and test logic. +It should be able to perform rather complex and precise tests, that can be +specified via action vector. Otherwise, knowledge about an image structure +allows the fuzzer to generate the pool of all available areas can be fuzzed +and randomly select some of them and so compose its own action vector. +Also complexity of a template defines complexity of the fuzzer, so its +functionality can be varied from simple model-independent fuzzing to smart +model-based one. + + +Glossary: +-------- + +Action vector is a sequence of structure elements retrieved from an image +format, each of them will be fuzzed for the test image. It's a subset of +elements of the action pool. Example: header, refcount table, etc. +Action pool is all available elements of an image structure that generated +automatically from an image template. +Image template is a formal description of an image structure and relations +between image blocks. +Test image is an output image of the fuzzer defined by the current seed and +action vector. -- 1.9.3
