Thanks for the pointers, Jonas. I cloned a new repo, then ran cmake, setting up arbitrary test compiler and lldb locations. Then ran llvm-lit tools/lldb/lit/Suite , and the test suite ran as expected. There were a few failures, but I've got a plan now.
Ted > -----Original Message----- > From: Jonas Devlieghere <jo...@devlieghere.com> > Sent: Friday, August 9, 2019 4:57 PM > To: Ted Woodward <tedw...@quicinc.com> > Cc: LLDB <lldb-dev@lists.llvm.org> > Subject: [EXT] Re: How do I use lit to only run the lldb test suite, now that > dotest multiprocessing capabilities have been removed? > > So far the only thing that changed by removing multiprocess is that `--no- > multiprocess` is always enabled. Everything else you describe is still > possible, > and will continue to be possible. > > Even when we remove the driver functionality from dotest.py this will all > continue to work. The only difference is that dotest.py will operate on a > single > file after being invoked by some other driver. The problem you describe with > the dual arguments seems relatively easy to fix by loading a different > configuration in lit. It's not something I've looked at yet, because > everything > we care about is configurable from CMake or overridable by passing --params > to lit. > > > On Fri, Aug 9, 2019 at 1:30 PM Ted Woodward <tedw...@quicinc.com> > wrote: > > > > Hi Jonas, > > > > What I need is a way to run the test suite with arbitrary command line > arguments. Sometimes I want to run one or more tests with -f, sometimes I > want to run one or more test files with -p, and sometimes I want to run the > entire suite, either in parallel or 1 at a time (--no-multiprocess). I might > be > running from a directory where I've built lldb (but not clang, using clang > from > an arbitrary location), a directory where I've built everything (but in this > case > we set everything up with cmake), or a directory where I've checkout out just > the sources and have copied the binaries from a distribution, so no cmake. > > > > dotest used to handle these cases; how do I handle them now? > > > > In our environment, lldb will launch and connect to the hexagon simulator, > much like how it launches and connects to debugserver or lldb-server. But it > has to use a specific version of clang, because we've found that if we have > mismatches in clang/simulator/RTOS, bad things can happen. User code is in a > shared library that gets loaded by a wrapper run under the RTOS. The RTOS, > wrapper and user code all need to be built with the same complier and c/c++ > libraries. > > > > Ted > > > > > -----Original Message----- > > > From: Jonas Devlieghere <jo...@devlieghere.com> > > > Sent: Friday, August 9, 2019 3:18 PM > > > To: Ted Woodward <tedw...@quicinc.com> > > > Cc: LLDB <lldb-dev@lists.llvm.org> > > > Subject: [EXT] Re: How do I use lit to only run the lldb test suite, > > > now that dotest multiprocessing capabilities have been removed? > > > > > > Hey Ted, > > > > > > On Thu, Aug 8, 2019 at 2:08 PM Ted Woodward <tedw...@quicinc.com> > > > wrote: > > > > > > > > Thanks Jonas. > > > > > > > > Is full support for --param fairly recent? I tried it with a > > > > version of our > > > master, based on top-of-tree from about a month ago, and it didn't > > > work quite right. It's passing the dotest args, but it's also > > > generating some args, so I'm seeing odd effects. > > > > > > It's not something I touched recently, but it's always possible they > > > made some changes in LLVM. > > > > > > > Here is my run line: > > > > bin/python bin/llvm-lit /local/mnt/ted/8.4/llvm/lldb/lit/Suite > > > > --param > > > 'dotest-args=-A v66 -C > > > /prj/dsp/qdsp6/release/internal/HEXAGON/branch- > > > 8.4/linux64/latest/Tools/bin/hexagon-clang --executable > > > /local/scratch/ted/8.4/build/bin/lldb -t -v -f > > > RecursiveTypesTestCase.test_recursive_type_1_dwarf' > > > > > > > > I only want to run > > > > RecursiveTyepsTestCase.test_recursive_type_1_dwarf, > > > but it's running the whole test suite. > > > > > > Do you know about the lldb-dotest binary? You can still use it to > > > invoke a single test, similar to how lit does it. You should be able > > > to just pass your arguments to that. > > > > > > Here's a dotest line from the run: > > > > > > > > /local/mnt/ted/8.4/build/bin/python > > > > /local/mnt/ted/8.4/llvm/lldb/test/dotest.py -q --arch=v66 -s > > > > /local/mnt/ted/8.4/build/lldb-test-traces --build-dir > > > > /local/mnt/ted/8.4/build/lldb-test-build.noindex -S nm -u CXXFLAGS > > > > -u CFLAGS --executable /local/mnt/ted/8.4/build/./bin/lldb > > > > --dsymutil /local/mnt/ted/8.4/build/./bin/dsymutil --filecheck > > > > /local/mnt/ted/8.4/build/./bin/FileCheck -C > > > > /local/mnt/ted/8.4/build/./bin/clang --env ARCHIVER=/usr/bin/ar > > > > --env OBJCOPY=/usr/bin/objcopy -A v66 -C > > > > /prj/dsp/qdsp6/release/internal/HEXAGON/branch-8.4/linux64/latest/ > > > > Tool s/bin/hexagon-clang --executable > > > > /local/scratch/ted/8.4/build/bin/lldb > > > > -t -v -f RecursiveTypesTestCase.test_recursive_type_1_dwarf --env > > > > LLVM_LIBS_DIR=/local/mnt/ted/8.4/build/./lib > > > > /local/mnt/ted/8.4/llvm/lldb/packages/Python/lldbsuite/test/functi > > > > onal ities/breakpoint/debugbreak -p TestDebugBreak.py > > > > > > > > > > > > It's got both --arch= and -A, -C is set to my build directory > > > > clang as well as the > > > clang I told it to use, --executable is set twice, and it's got -f > > > RecursiveTypesTestCase.test_recursive_type_1_dwarf and -p > > > TestDebugBreak.py . > > > > > > Both lit and lldb-dotest are configured using the dotest arguments > > > that we can configure at CMake configuration time. That would > > > explain where the extra options come from. If those are not the ones > > > you want, you can still invoke dotest.py directly. > > > > > > > > > > > These tests that do a "process launch" (which is most of them) > > > > invoke the > > > hexagon simulator, but it was never launched. There was also only 1 > > > testcase built in /local/mnt/ted/8.4/build/lldb-test-build.noindex - > > > types/TestRecursiveTypes.test_recursive_type_1_dwarf . > > > > > > Other than the extra arguments I can't think of any reason why this > > > would behave differently. Lit is just a simple wrapper that invokes > > > dotest.py with the right arguments. > > > > > > > This does not have the patch that removes multiprocess support > > > > from > > > dotest. > > > > > > > > Ted > > > > > > > > > -----Original Message----- > > > > > From: Jonas Devlieghere <jo...@devlieghere.com> > > > > > Sent: Thursday, August 8, 2019 2:50 PM > > > > > To: Ted Woodward <tedw...@quicinc.com> > > > > > Cc: LLDB <lldb-dev@lists.llvm.org> > > > > > Subject: [EXT] Re: How do I use lit to only run the lldb test > > > > > suite, now that dotest multiprocessing capabilities have been > removed? > > > > > > > > > > Hey Ted, > > > > > > > > > > 1. You can run just the dotest-tests by pointing lit at the > > > > > `lit/Suite` > > > directory. > > > > > 2. You can pass arguments to dotest by passing `dotest-args` in -- > param. > > > > > > > > > > The invocation would look something like this: > > > > > > > > > > /path/to/llvm/bin/llvm-lit /path/to/lldb/lit/Suite --param > > > > > 'dotest-args=--foo -- bar' > > > > > > > > > > Hope that helps, > > > > > Jonas > > > > > > > > > > On Thu, Aug 8, 2019 at 9:31 AM Ted Woodward > > > > > <tedw...@quicinc.com> > > > > > wrote: > > > > > > > > > > > > RE: https://reviews.llvm.org/D65311 > > > > > > > > > > > > > > > > > > > > > > > > Internally we use dotest to run the lldb test suite with > > > > > > various RTOS > > > > > configurations for the test binaries. In these runs we don’t > > > > > care about the lit tests or the unit tests, because they are OS > > > > > agnostic. > > > > > We do this by specifying the compiler, lldb, and test flavor > > > > > (static testcase + os, dynamic library testcase loaded by an OS > > > > > image, dynamic library testcase loaded by an OS image running > > > > > the OS’ debug > > > stub). > > > > > > > > > > > > > > > > > > > > > > > > With the multiprocess testrunner removed, how do I have lit: > > > > > > > > > > > > Only run the lldb test suite > > > > > > Run dotest with specific arguments > > > > > > > > > > > > > > > > > > > > > > > > We’re not running cmake; we’re taking an existing tools build > > > > > > and running > > > > > the tests from the source directory using the toolset. _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
