Dave W. and I were chatting about putting together a collection of agreed upon improvements for the test infrastructure.
If people want to respond to this thread or email me, I will start a .rst that we can maintain in the repo. I want to start the thread off with my disagreement with previous comments that the module owner owns the tests. The module owner owns the test objectives. I never commented until now because I wanted to see where things went. Back in May, Neale fixed this bug: https://gerrit.fd.io/r/c/vpp/+/27187 It was a typo that was slipping by because in the test framework, we send the *same* packet over and over again. So, by design, b[0] == b[1] and the test didn't actually test the code. We have never addressed the failing in the test framework. In this case, I think we actually have to make changes to VPP to make it more testable. Say if we add a .stride_width to a node registration, the test could query the .stride_width and make sure there were always (stride_width + x) distinct packets. (I guess that's my long winded way of saying that to fix the tests, we may need to fix VPP) I'm also curious as to what you think what the goal of the test framework is. Even though it relies on the python.unittest library, it rarely runs unit tests. We have cases where it runs and reports on c unit tests, but mainly my experience is that it is a sort of regression test. A while back when the api was being changed, the tests were just adjusted to match the new values. I'm curious to hear what others have to add. Paul
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18068): https://lists.fd.io/g/vpp-dev/message/18068 Mute This Topic: https://lists.fd.io/mt/78323792/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
