On Mon, Jan 29, 2024 at 12:53 AM Peter Xu <pet...@redhat.com> wrote:
> On Sun, Dec 31, 2023 at 08:58:04PM +0000, Bryan Zhang wrote:
> > Adds an integration test for 'qatzip'.
>
> Please use "tests" as prefix of this patch. It can be "tests/migration:",
> "tests/migration-test:", etc.
>
> Will do.
> >
> > Signed-off-by: Bryan Zhang <bryan.zh...@bytedance.com>
> > Signed-off-by: Hao Xiang <hao.xi...@bytedance.com>
>
> [...]
>
> > test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from,
> > @@ -3480,6 +3504,19 @@ int main(int argc, char **argv)
> > qtest_add_func("/migration/multifd/tcp/plain/zstd",
> > test_multifd_tcp_zstd);
> > #endif
> > +#ifdef CONFIG_QATZIP
> > + /*
> > + * Use QATzip's qzInit() function as a runtime hardware check.
> > + * Ideally there might be a cleaner way to probe for the presence
> of QAT.
> > + */
> > + QzSession_T sess;
> > + memset(&sess, 0, sizeof(QzSession_T));
> > + if (qzInit(&sess, 0) == QZ_OK) {
>
> Does "0" means it'll fail if no hardware is available, no matter whether
> due to processor too old, or limited resources?
>
> Would it make sense to test it even if only software fallbacks are
> available? IIUC the migration path will still be covered in software
> fallbacks, so it may still makes sense to me. It can be very likely that
> most CIs will not cover the qatzip paths otherwise, because of either no
> control over hardwares, or limited privileges of the CI user (does qat HWs
> need special privilege normally? I have no idea how QAT resource
> management is done if there're only limited HW resources).
>
> Besides, I believe all the data points you provided in the cover letter are
> collected with 2 QAT HWs enabled? I'm curious how's the performance of the
> software fallback of qatzip comparing to existing algorithm: iiuc zstd is
> mostly always preferred if sololy comparing to zlib? where does qatzip
> soft-mode stand in the picture?
> Yes, as I understand it, 0 means the code will error if the hardware is
unavailable for whatever reason.
We can enable software fallback in the live migration path, which will also
enable using software to run the QATzip tests (and will conveniently allow
us to remove the awkward `qzInit` check in the test code). It also might be
a good idea since it would also avoid failure in case of, e.g., a transient
hardware problem?
Performance: The software fallback seems prohibitively slow. QATzip
fallback uses the built-in zlib implementation, but I ran a migration test
that normally takes zlib about 150 seconds and the QATzip fallback took
about 30 minutes before my SSH session disconnected :P
Also, a note: When enabling software fallback, QATzip and the QAT driver
will both repeatedly print a complaint to the QEMU monitor when trying to
compress without hardware. Is it bad form to introduce
seemingly-unsuppressable prints from dependencies, or is this acceptable?
>
> Thanks,
>
> --
> Peter Xu
>
> Thanks for your feedback.
--
Bryan Zhang
