Hi, A few months ago I announced that I was planning to extend the I/O accounting in QEMU based on the previous plans by Benoît and his discussions in the mailing list in 2014.
Here are the links for reference: https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04954.html https://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00080.html https://lists.nongnu.org/archive/html/qemu-block/2015-06/msg00071.html In June I sent a patch series with the infrastructure that I was going to use to compute averages. After that the 2.4 release happened and I had some time to continue working on this, so this new series contains the complete implementation of the new statistics, plus tests and a couple of bug fixes. The series is long but most patches are quite simple so they should be easy to understand. It applies on top of Max's "BlockBackend and media v5" that moves BlockAcctStats to BlockBackend. Here's the summary of what's new: - New block_acct_failed() and block_acct_invalid() calls. We keep track now of the number of successful, failed and invalid operations (each one separated into read, write and flush). So from the API point of view, BlockDeviceStats contains 6 new fields for those. - idle_time_ns: time since the last I/O operation. - New BlockDeviceTimedStats struct: it has statistics for the I/O during a given interval of time. It keeps minimum, maximum and average latencies for read, write and flush operations. It also keeps the average read and write queue depths. - New 'stats-intervals' option that allows the user to define the intervals used to keep the aforementioned statistics. An arbitrary number of intervals can be specified, the length of each one is in seconds. For the API I opted for a colon-separated list of numbers, stats-intervals=60:3600:86400 I also considered something a different syntax, stats-intervals.0.length=60, stats-intervals.1.length=3600, stats-intervals.2.length=86400 This one could be useful if we want to specify any other attribute for each interval, but I couldn't come up with any, so I chose the simpler solution. - Two new options, stats-account-invalid and stats-account-failed, which allow the user to decide whether to count invalid and failed operations when computing the idle time and total latency. - 'supports_stats': a new field for the BlockStats structure that tells you whether the BDS supports statistics or not. This one can probably be improved by asking the device model. I think that's all. I'm sure there will be questions and rough edges to discuss, so I'm all yours. Regards, Berto v2: - First complete implementation of the new statistics v1: https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg03321.html - Initial series containing only the timed average infrastructure. Alberto Garcia (22): xen_disk: Account for flush operations ide: Account for write operations correctly block: define 'clock_type' for the accounting code util: Infrastructure for computing recent averages block: Add idle_time_ns to BlockDeviceStats block: Add "supports_stats" field to BlockStats block: Add statistics for failed and invalid I/O operations block: Allow configuring whether to account failed and invalid ops block: Compute minimum, maximum and average I/O latencies block: Add average I/O queue depth to BlockDeviceTimedStats block: New option to define the intervals for collecting I/O statistics qemu-io: Account for failed, invalid and flush operations block: Use QEMU_CLOCK_VIRTUAL for the accounting code in qtest mode iotests: Add test for the block device statistics nvme: Account for failed and invalid operations virtio-blk: Account for failed and invalid operations xen_disk: Account for failed and invalid operations atapi: Account for failed and invalid operations ide: Account for failed and invalid operations macio: Account for failed operations scsi-disk: Account for failed operations block: Update copyright of the accounting code block/accounting.c | 118 ++++++++++++++- block/block-backend.c | 1 + block/qapi.c | 53 +++++++ blockdev.c | 53 +++++++ hmp.c | 4 +- hw/block/nvme.c | 11 +- hw/block/virtio-blk.c | 4 +- hw/block/xen_disk.c | 27 +++- hw/ide/atapi.c | 31 ++-- hw/ide/core.c | 12 +- hw/ide/macio.c | 12 +- hw/scsi/scsi-disk.c | 46 ++++-- include/block/accounting.h | 28 ++++ include/qemu/timed-average.h | 64 ++++++++ qapi/block-core.json | 106 ++++++++++++- qemu-io-cmds.c | 9 ++ qmp-commands.hx | 86 ++++++++++- tests/Makefile | 4 + tests/qemu-iotests/136 | 349 +++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/136.out | 5 + tests/qemu-iotests/group | 1 + tests/test-timed-average.c | 89 +++++++++++ util/Makefile.objs | 1 + util/timed-average.c | 227 ++++++++++++++++++++++++++++ 24 files changed, 1293 insertions(+), 48 deletions(-) create mode 100644 include/qemu/timed-average.h create mode 100644 tests/qemu-iotests/136 create mode 100644 tests/qemu-iotests/136.out create mode 100644 tests/test-timed-average.c create mode 100644 util/timed-average.c -- 2.5.3
