On Wednesday, March 6, 2019 9:32:57 AM CET Daniel Stenberg via curl-library
wrote:
> Hi,
>
> We've previously talked about doing something to enable running tests in
> parallel on the same host. Since quite a long time actually.
>
> The current problem is that we fire up a number of test servers that listen
> to TCP ports and we run curl, and other test-tools, against those servers.
> We have somewhere around 20 different server variations, each using its own
> TCP port.
>
> Right now, since the dawn of time, we run each test one by one in a serial
> fashion. On a moderately fast machine, running all existing tests (with
> valgrind) now take well over twenty minutes. Maybe thirty. And as we keep
> adding more tests over time, this isn't likely to improve unless we
> drastically change something.
>
> It would be much cooler if we could run more than one test at a time. Most
> of us have multi-core machines and a lot of the tests are done waiting so
> doing 10-20 in parallel can probably reduce the total time needed to a
> tenth of the current or something (I'm guessing).
>
> Running multiple test suites like we do now but changing the base port
> number per instance is possible but only so far as each new would need 20
> consequtive available ports etc - and it gets fairly complicated.
>
> Running test instances in separate multiple containers is possible but adds
> necessary infra, a little overhead and debugging test failures might become
> slightly more complicated.
>
> Let me propose a third way: The LD_PRELOAD way.
>
> We use a preload library both for the servers and for curl that hijacks the
> network calls and instead of using TCP, communicates over named pipes. Each
> test case uses its own unique directory where these named pipes are created
> so they would not clash with each other. Both the server and the clients
> *can* still use TCP the old way - for the systems that can't preload or if
> we for other reasons want to run the tests in "actual" TCP mode. I presume
> some tests won't be suitable for preloading as well and then they can stay
> "on the TCP track".
>
> This way, we can basically run all tests at once if we just have the ram and
> cpu for it. Not that I think that's necessarily a clever idea.
>
> What's even better: the preload library for this was already developed so it
> already exists and works and is being used for exactly this sort of testing
> in the Samba project. The library is aptly called socket_wrapper:
> https://cwrap.org/socket_wrapper.html
>
> I tried to brainstorm this in a gist document on github that also includes
> some command lines showing how (basically) our test 1 can be run using the
> preload wrapper:
>
> https://gist.github.com/bagder/5914029ba6ffca1fd4b74b010eab33ed
>
> Thoughts?
This kind of instrumentation is fragile and suffers from portability issues.
I remember a Fedora bug where a user of socket_wrapper was blaming bash for
crashing his test-suite on the ppc64le architecture. It later appeared to
be a tricky bug in socket_wrapper:
https://bugzilla.redhat.com/1465147
Note that, when you run LD_PRELOAD=./libsocket_wrapper.so ../src/curl as in
your example, you might be instrumenting the shell interpreter, too, in case
libtool wrapped ../src/curl as a shell script.
Kamil
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
