Brad King wrote:
troy d. straszheim wrote:
I don't quite get "That doesn't mean we can't test some tools without
log-scraping".
I see two different cases here. There's the developer working under
visual studio or emacs who wants to run some tests. This guy knows (or
should know) how to find what compile/link flags were used, chase down
warnings and whatnot. In this case the message "test X failed, go look
at the logs" is fine. Let the user use his tool.
How do we know that a VS IDE build works unless we test it? Testing it
requires log scraping unless we write some kind of plugin, which may not
be possible for all tools. Even if we do log scraping for VS IDE
builds, we can still use "CWrap" for generators where it can be implemented.
I think we're agreeing here. There is testing the VS IDE build, and then there
is testing the VS IDE build and trying to report every single thing that could
go wrong in the course of the build to a cdashish server somewhere. I'd assume
that IDE builds would need to be tested by somebody sitting in front of the IDE.
If something goes wrong the IDE tells you and you go figure out what it is.
make and nmake builds, running on slaves in a datacenter someplace, would need
to report everything that goes wrong. I don't think the slave-testing-process
should get complicated by IDEs that constantly get tested manually anyhow.
CTest has several parts. One is to monitor builds, another is to run
the tests. Currently it uses log scraping to monitor builds since the
native tools don't provide per-rule reports and we haven't created
"CWrap" to work around the limitation. For running the tests it has
always separated output on a per-test basis.
OK, we're on the same page.
It runs all the tests through python command wrappers to capture
individual output, and therefore has to generate its own compiler
command line invocations instead of using CMake's knowledge of the
native tools. Currently -c and -o options are hard-coded AFAICS.
Right... do you see a different way to get this done?
Use --build-and-test and with "CWrap" enabled (or currently the python
approximation of it).
Ok.
We could make this a CMake feature by teaching the generators to wrap
the compiler up with a tool we distribute with CMake. Then you won't
have to hack the compilation rule variables for Boost or depend on
python.
Presumably the name of this tool (let's call it CWrap?) would have some
interface that you could implement however you like... and if that is
implementable in python, you've given your users lots of ways to tweak
their build system. We're OK with being dependent on python.
Whatever interface we create to tell the generators to do this can also
specify the wrapper (hook) command. We can provide a simple tool to be
used by default and just specify a command-line interface for a custom tool.
Sounds good.
Testing with ctest's --build-and-test feature. The entire build and
execution of every test would be captured independently of other tests.
Or a python script that does what ctest's --build-and-test does...
IMV a lot more flexibility, a lot less code.
How is duplicating functionality less code? If --build-and-test is
missing something, we can extend it.
Well, the ability to easily talk to anything that has python bindings and e.g.
report results via XML-RPC to a trac server doesn't duplicate anything in ctest,
and that'd be a real PITA to code up in C++. To support this kind of thing, I
could envision some python bindings to ctest: ctest does its thing, calls a
python function (that it has been passed) as it collects results. I'd have to
think about this.
The code in question tells CMake to generate a python script that looks
like this (on Windows):
sys.path.append("c:\path\with\backslashes\to\some\file.txt")
# ^^ escape sequence?
I'd have to look back. This stuff was indeed working on windows;
anyhow, looks like we can detangle a lot of this with some of your help
making tweaks to cmake itself.
Python seems to leave the backslashes if the sequence doesn't happen to
be a supported escape. You may just be getting lucky.
Yeah it'd be nice to not have to do all this, obviously bugs breed in this kind
of stuff.
So the first order of business would be to remove this kind of thing:
set(CMAKE_CXX_COMPILE_OBJECT "\"${PYTHON_EXECUTABLE}\" \"${BOOST_TEST_DRIVER}\"
<CMAKE_CURRENT_BINARY_DIR> cxx_compile_object <OBJECT>
${CMAKE_CXX_COMPILE_OBJECT}" )
We're 100% agreed on the need for this, as far as I can see. You may have ideas
about the interface. It gets executed as:
wrapper build_dir opcode target arg0 arg1 arg2 ... argN
e.g.:
mywrap.py /path/to/build create_shared_library libsomething.so gcc -shared
-o libsomething.so somebody.o somebodyelse.o -lstdc++
'opcode' is one of cxx_compile_object, create_shared_library,
create_static_library, link_executable.
The driver currently pickles all this information as well as the exit code,
stderr and stdout, and elapsed time to a file, where it is later collected and
sent to the reporting system. IIRC a nonzero exit code indicates error, and
output to stderr indicates warning, on most platforms (but there was some
hackery involving windows).
-t
_______________________________________________
Boost-cmake mailing list
Boost-cmake@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-cmake
