Hi,
We've previously talked about doing something to enable running tests in
parallel on the same host. Since quite a long time actually.
The current problem is that we fire up a number of test servers that listen to
TCP ports and we run curl, and other test-tools, against those servers. We
have somewhere around 20 different server variations, each using its own TCP
port.
Right now, since the dawn of time, we run each test one by one in a serial
fashion. On a moderately fast machine, running all existing tests (with
valgrind) now take well over twenty minutes. Maybe thirty. And as we keep
adding more tests over time, this isn't likely to improve unless we
drastically change something.
It would be much cooler if we could run more than one test at a time. Most of
us have multi-core machines and a lot of the tests are done waiting so doing
10-20 in parallel can probably reduce the total time needed to a tenth of the
current or something (I'm guessing).
Running multiple test suites like we do now but changing the base port number
per instance is possible but only so far as each new would need 20 consequtive
available ports etc - and it gets fairly complicated.
Running test instances in separate multiple containers is possible but adds
necessary infra, a little overhead and debugging test failures might become
slightly more complicated.
Let me propose a third way: The LD_PRELOAD way.
We use a preload library both for the servers and for curl that hijacks the
network calls and instead of using TCP, communicates over named pipes. Each
test case uses its own unique directory where these named pipes are created so
they would not clash with each other. Both the server and the clients *can*
still use TCP the old way - for the systems that can't preload or if we for
other reasons want to run the tests in "actual" TCP mode. I presume some tests
won't be suitable for preloading as well and then they can stay "on the TCP
track".
This way, we can basically run all tests at once if we just have the ram and
cpu for it. Not that I think that's necessarily a clever idea.
What's even better: the preload library for this was already developed so it
already exists and works and is being used for exactly this sort of testing in
the Samba project. The library is aptly called socket_wrapper:
https://cwrap.org/socket_wrapper.html
I tried to brainstorm this in a gist document on github that also includes
some command lines showing how (basically) our test 1 can be run using the
preload wrapper:
https://gist.github.com/bagder/5914029ba6ffca1fd4b74b010eab33ed
Thoughts?
--
/ daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
