Hi Dominik, those tests look great. Something like that is exactly what I had on mind for dnsmasq itself. Would you mind if I borrow few things and try to make some dnsmasq-only parts, not dependent on pihole?
Just curious. Why do you support linking only to static libraries? Is pihole project opposed to be eventually packages as a normal distribution package? I had to modify CMakeLists.txt, when I had all devel libraries needed. It would not even compile. Though it seems amazing to find my own commits in a project I never contributed directly. Do you use some kind of container or dedicated VM to run these tests? Do you rely just on github services to run those tests? I admit those checks run on every PR looks quite neat, I would love to run something similar also for dnsmasq. Once we had some tests to run, it might be possible to run them on all new commits just similar way. Cheers, Petr On 10/5/21 20:35, Dominik Derigs wrote: > Hey Petr and others, > > On Tue, 2021-10-05 at 17:13 +0200, Petr Menšík wrote: >> It would be beneficial to have also behavior tests. > it may be the time to mention that we do exactly this for Pi-hole > FTL which embeds the full dnsmasq for the DNS part. On every > commit, a virtual machine is started which firstly compiles our > project (including dnsmasq) and secondly runs it with standard > parameters and starts a full test bench with more than a hundred > individual tests. While the majority of tests are for extensions > we made to dnsmasq (regular expressions, database integration, > CNAME inspection, etc), we also have some standard DNS tests > sending some A, AAAA, PTR, CNAME, SRV, SOA, ANY, TXT, NAPTR, MX, > DS, RRSIG, etc. queries to specific domains and checking the > answers. We then also check the logs and if anything is still > working. > > For instance, our tests complained when merging my patch adding > all the known DNS RR types because "query[type=5]" changes to > "query[CNAME]". This was not a bug but you see the tests have > noticed it. > > Even when this isn't directly applicable to the dnsmasq core > project, it is still something that tests dnsmasq, even when > embedded in another project. There is no unittest library > whatsoever involved. The tests simply run on a compiled binary. > > You can find everything here if you're curious: > https://github.com/pi-hole/FTL/tree/master/test > > Best, > Dominik > -- Petr Menšík Software Engineer Red Hat, http://www.redhat.com/ email: pemen...@redhat.com PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB _______________________________________________ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
