> On Nov 29, 2017, at 2:51 PM, Zachary Turner via lldb-commits > <lldb-commits@lists.llvm.org> wrote: > > > > On Wed, Nov 29, 2017 at 1:59 PM Jim Ingham <jing...@apple.com> wrote: > I'm mostly basing this concern on the bad effect this had on gdb all of whose > testing was expect based command scraping. gdb is a tool that's much closer > to lldb than any of the compiler tools so that experience seems relevant. > It's been a decade or so since I worked on gdb, but back when I was working > on it, you really had to tread very carefully if you wanted to change the > output of, say, the break command, or to thread listings, etc, and a bunch of > times I just had to bag some cleanup of output I wanted to do because fixing > up all the tests was too time consuming. Because Jason and I had both had > this experience when we started working on lldb, we promised ourselves we > wouldn't go down this path again... > > > Couple of things: > > 1) I wouldn't dare to use this approach for anything that required > interactivity. If you need to run one command, extract a value from the > output, and use that value as input to another command, I think that would be > a big mistake. I have no intention of ever proposing something like that. > > 2) FileCheck is very flexible in the way it matches output and tests can be > written so that they are resilient to minor format tweaks. I have no doubt > that with pure regex matching, or with pretty much any other tool, you would > have a really bad time. Of course, that doesn't mean it would be hard to > construct an example of a format change that would break a FileCheck test. > But I think it would happen *far* less frequently than it did on GDB. That > said, I still understand your concerns that it's fragile, so... > > 3) I would not be opposed to a tool called lldb-test, which was basically > just LLDB with a different, and much more limited set of commands, and was > completely non-interactive and would produce output in a format designed for > being scraped, and which never had to be changed since it was never presented > to the user.
100% agree with #3. We could go back and forth about using lldb-mi, but I think a specialized driver using SB API, designed for testing, would be a great approach. _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
