Machine learning device APIs test application =============================================
This series of patches introduces a test application for machine learning device APIs. A test framework is implemented with multiple test enabled, to validate the device, model and fast-path functions. New tests can be added using the test framework. List of tests supported ----------------------- 1) device_ops: Test case to validate device re-configuration 2) model_ops: Collection of 4 sub-tests to validate model slow APIs. Each sub-test would invoke the slow path model APIs (load / start / stop / unload) in different order. 3) inference_ordered: Test case to validate execution of end-to-end inferences on ML device with one active model at a time. This test can execute inference requests for multiple models, with inferences for a model executed after completion of inferences for a previously loaded model. 4) inference_interleave: Test case to validate end-to-end inferences with multiple active models concurrently. This case would work as a stress test to validate ML device. Options supported the by tests include burst size for enqueuing and dequeuing inference requests, enabling multiple queue pairs with a user specified value for queue size. Support is also enabled for batch inferencing, output validation and statistics. v5: * Cleanup of header includes * Addressed issues with unclean patches * Changed input/output size variables to uint32_t * Rebase over main branch v4: * Updated model_id as uint16_t * Updated license info in SVG files * Updated release notes v3: * Code rebase Srikanth Yalavarthi (12): app/mldev: implement test framework for mldev app/mldev: add common test functions app/mldev: add test case to validate device ops app/mldev: add test case to validate model ops app/mldev: add ordered inference test case app/mldev: add test case to interleave inferences app/mldev: enable support for burst inferences app/mldev: enable support for queue pairs and size app/mldev: enable support for inference batches app/mldev: enable support for inference validation app/mldev: enable reporting stats in mldev app app/mldev: add documentation for mldev test cases MAINTAINERS | 2 + app/meson.build | 1 + app/test-mldev/meson.build | 24 + app/test-mldev/ml_common.h | 29 + app/test-mldev/ml_main.c | 113 ++ app/test-mldev/ml_options.c | 325 +++++ app/test-mldev/ml_options.h | 57 + app/test-mldev/ml_test.c | 41 + app/test-mldev/ml_test.h | 76 ++ app/test-mldev/parser.c | 380 ++++++ app/test-mldev/parser.h | 55 + app/test-mldev/test_common.c | 136 ++ app/test-mldev/test_common.h | 27 + app/test-mldev/test_device_ops.c | 228 ++++ app/test-mldev/test_device_ops.h | 17 + app/test-mldev/test_inference_common.c | 1122 +++++++++++++++++ app/test-mldev/test_inference_common.h | 74 ++ app/test-mldev/test_inference_interleave.c | 118 ++ app/test-mldev/test_inference_ordered.c | 116 ++ app/test-mldev/test_model_common.c | 164 +++ app/test-mldev/test_model_common.h | 46 + app/test-mldev/test_model_ops.c | 428 +++++++ app/test-mldev/test_model_ops.h | 20 + doc/guides/rel_notes/release_23_03.rst | 8 + .../tools/img/mldev_inference_interleave.svg | 669 ++++++++++ .../tools/img/mldev_inference_ordered.svg | 528 ++++++++ .../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++ doc/guides/tools/index.rst | 1 + doc/guides/tools/testmldev.rst | 441 +++++++ 32 files changed, 6879 insertions(+) create mode 100644 app/test-mldev/meson.build create mode 100644 app/test-mldev/ml_common.h create mode 100644 app/test-mldev/ml_main.c create mode 100644 app/test-mldev/ml_options.c create mode 100644 app/test-mldev/ml_options.h create mode 100644 app/test-mldev/ml_test.c create mode 100644 app/test-mldev/ml_test.h create mode 100644 app/test-mldev/parser.c create mode 100644 app/test-mldev/parser.h create mode 100644 app/test-mldev/test_common.c create mode 100644 app/test-mldev/test_common.h create mode 100644 app/test-mldev/test_device_ops.c create mode 100644 app/test-mldev/test_device_ops.h create mode 100644 app/test-mldev/test_inference_common.c create mode 100644 app/test-mldev/test_inference_common.h create mode 100644 app/test-mldev/test_inference_interleave.c create mode 100644 app/test-mldev/test_inference_ordered.c create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg create mode 100644 doc/guides/tools/testmldev.rst -- 2.17.1
