On 20 May 2017, at 20:12, gan...@apache.org wrote:
Hi trafficserver-dev,
(1) I was wondering if we have any preference for a C/C++ unit test
framework in the ATS community.
Background and thoughts:
I needed a unit test framework quickly to do TDD in order to deliver
the AWS auth v4 (PR #1946).
Google Test seemed easy to bootstrap, popular, feature rich and easy
to use, also its license is "BSD 3-clauses”.
The unit tests I wrote (PR# 1953) got the work done, they don't do
anything fancy or Google Test specific and can be converted to use any
other framework of our choice.
(2) I was wondering what to do with the unit tests until we decide on
a unit test framework that we all going to use.
Background and thoughts:
People have concerns that if every developer starts checking into
master unit tests using frameworks of their choice (even if not run)
we will end up with a mess/chaos.
I understand the benefits of having one or two agreed upon frameworks
and run all our tests with every build.
One framework :)
I have a similar case with the cachekey plugin tests where I used the
“old” TSQA framework (at the time) which I still run to test for
regressions and enhance when I make changes.
Do these run as part of regressions now?
If we plan to rip out the “old” TSQA framework should I then
remove those tests from the source tree until I convert them to
“autest"?
We should have one true integration test framework. AFAIK autest is it,
but I still think it needs a bit better documentation and getting
started affordances.
My personal take is that in both examples (s3_auth_v3 and cachekey)
the tests have 100% code and very high branch coverage (the wonders of
TDD) so it would be a pity to rip them out and keep them some else
until converted.
See ci/coverage for test coverage instrumentation example. It would be
pretty great to have persistent coverage data from the automated
testing.
If they are checked into the source tree we could still use them
“offline” to test for regressions in meanwhile, also they will be
available to anyone to work on them and would be a pretty good
starting point for bringing the tests up-to-date to the agreed upon
framework when we have a chance.
We have to converge on 1 unit test framework and 1 integration test
framework. I really think there are only 2 realistic unit test choices:
1. We have an internal unit test harness with existing tests. It’s
primitive, but basically works for unit testing of traffic_server and
standalone test drivers. If we don’t run with this, we should figure
out how to replace it.
2. gtest is good and widely used. The google benchmark package is nice
and it would be helpful to have micro benchmarks for hot path code. If
we go down this path, we should bundle gtest and not use the system
version at all.
James
