On Dec 16, 2024, at 9:18 AM, Jakub Zelenka <bu...@php.net> wrote: > > Hi, > > I have been looking into how to test some cases where integration tests are > very difficult or even impossible to create for. Those are often found in > networking related and system specific code code (network.c, streams, FPM and > more). I was recently fixing one such bug and decided to give a try which > resulted in this PR: https://github.com/php/php-src/pull/16987 . > > There was a suggestion of RFC but that might be a bit too much as it's just > an internal change / addition. But certainly some overview on internals > should be done so writing this instead. > > I decided to use cmocka in that PR because I had some experience with that. > It's quite small and still very powerful and allow vast mocking options. It's > a bit manual but it gives a bigger control over the mock. It relies on --wrap > linking option that replaces original functions with wraps. This is however > available only on Linux (or maybe some other Unix variants) but doesn't work > on MacOS or Windows. The developers that want to use it on those platforms > would need to use some Linux Virtualisation option (e.g. Docker). It also > requires static library which is supported by embed SAPI that can be compiled > statically. That limits number of extensions to use but the main use cases > don't really have deps so it should be fine. > > I did also some research into the other mocking libraries in C. There is a > Unity with CMock, FFF and some C++ libs like GUnit, Criterion and Trompeloeil > that I looked into. I quickly discarded GUnit and Trompeloeil as they relay > on C++ virtual methods and require wrapping C code to C++ which is very > inconvenient. FFF seems too simple and maybe quite inflexible for our needs > as well. Criterion also optionally uses wrap so I didn't see much advantages > compare to cmocka. So it left Unity with CMock that allows generating custom > mocks using a Ruby script. That seemed initially quite nice but after > spending around two hours with trying to make it works for PHP codebase, I > just gave up. It gets quite messy for complex scenarios and I just didn't > figure out how to nicely mock libc functions without any modification to > php-src. > > In terms of CI. It has got its own build which is very simple and it tests > just specific parts so we could just limit it to run only for changed files > which might be quite convenient. > > So the proposed PR is probably the only reasonable unit testing that I can > come up with. I think it should be completely optional initially for people > to use - more like an experiment. If it becomes used, then good of course. > And if it becomes pain, we can just get rid of it. Has anyone got any > objections to get this merged? If not I plan to merge it early in January. > > Cheers > > Jakub >
I'm assuming that uses ELF symbol interposition or something like that, which is why it seems Linux/BSD specific. That seems fragile to me. I think currently for wanting to test C functions, we're adding custom functions into ext/zend_test and writing PHPT. Would this work? We already have that, after all. If not, it'd be helpful to list the challenges that approach faces.
