Hi Jonas, What I need is a way to run the test suite with arbitrary command line arguments. Sometimes I want to run one or more tests with -f, sometimes I want to run one or more test files with -p, and sometimes I want to run the entire suite, either in parallel or 1 at a time (--no-multiprocess). I might be running from a directory where I've built lldb (but not clang, using clang from an arbitrary location), a directory where I've built everything (but in this case we set everything up with cmake), or a directory where I've checkout out just the sources and have copied the binaries from a distribution, so no cmake.
dotest used to handle these cases; how do I handle them now? In our environment, lldb will launch and connect to the hexagon simulator, much like how it launches and connects to debugserver or lldb-server. But it has to use a specific version of clang, because we've found that if we have mismatches in clang/simulator/RTOS, bad things can happen. User code is in a shared library that gets loaded by a wrapper run under the RTOS. The RTOS, wrapper and user code all need to be built with the same complier and c/c++ libraries. Ted > -----Original Message----- > From: Jonas Devlieghere <jo...@devlieghere.com> > Sent: Friday, August 9, 2019 3:18 PM > To: Ted Woodward <tedw...@quicinc.com> > Cc: LLDB <lldb-dev@lists.llvm.org> > Subject: [EXT] Re: How do I use lit to only run the lldb test suite, now that > dotest multiprocessing capabilities have been removed? > > Hey Ted, > > On Thu, Aug 8, 2019 at 2:08 PM Ted Woodward <tedw...@quicinc.com> > wrote: > > > > Thanks Jonas. > > > > Is full support for --param fairly recent? I tried it with a version of our > master, based on top-of-tree from about a month ago, and it didn't work quite > right. It's passing the dotest args, but it's also generating some args, so > I'm > seeing odd effects. > > It's not something I touched recently, but it's always possible they made some > changes in LLVM. > > > Here is my run line: > > bin/python bin/llvm-lit /local/mnt/ted/8.4/llvm/lldb/lit/Suite --param > 'dotest-args=-A v66 -C /prj/dsp/qdsp6/release/internal/HEXAGON/branch- > 8.4/linux64/latest/Tools/bin/hexagon-clang --executable > /local/scratch/ted/8.4/build/bin/lldb -t -v -f > RecursiveTypesTestCase.test_recursive_type_1_dwarf' > > > > I only want to run RecursiveTyepsTestCase.test_recursive_type_1_dwarf, > but it's running the whole test suite. > > Do you know about the lldb-dotest binary? You can still use it to invoke a > single test, similar to how lit does it. You should be able to just pass your > arguments to that. > > Here's a dotest line from the run: > > > > /local/mnt/ted/8.4/build/bin/python > > /local/mnt/ted/8.4/llvm/lldb/test/dotest.py -q --arch=v66 -s > > /local/mnt/ted/8.4/build/lldb-test-traces --build-dir > > /local/mnt/ted/8.4/build/lldb-test-build.noindex -S nm -u CXXFLAGS -u > > CFLAGS --executable /local/mnt/ted/8.4/build/./bin/lldb --dsymutil > > /local/mnt/ted/8.4/build/./bin/dsymutil --filecheck > > /local/mnt/ted/8.4/build/./bin/FileCheck -C > > /local/mnt/ted/8.4/build/./bin/clang --env ARCHIVER=/usr/bin/ar --env > > OBJCOPY=/usr/bin/objcopy -A v66 -C > > /prj/dsp/qdsp6/release/internal/HEXAGON/branch-8.4/linux64/latest/Tool > > s/bin/hexagon-clang --executable /local/scratch/ted/8.4/build/bin/lldb > > -t -v -f RecursiveTypesTestCase.test_recursive_type_1_dwarf --env > > LLVM_LIBS_DIR=/local/mnt/ted/8.4/build/./lib > > /local/mnt/ted/8.4/llvm/lldb/packages/Python/lldbsuite/test/functional > > ities/breakpoint/debugbreak -p TestDebugBreak.py > > > > > > It's got both --arch= and -A, -C is set to my build directory clang as well > > as the > clang I told it to use, --executable is set twice, and it's got -f > RecursiveTypesTestCase.test_recursive_type_1_dwarf and -p > TestDebugBreak.py . > > Both lit and lldb-dotest are configured using the dotest arguments that we can > configure at CMake configuration time. That would explain where the extra > options come from. If those are not the ones you want, you can still invoke > dotest.py directly. > > > > > These tests that do a "process launch" (which is most of them) invoke the > hexagon simulator, but it was never launched. There was also only 1 testcase > built in /local/mnt/ted/8.4/build/lldb-test-build.noindex - > types/TestRecursiveTypes.test_recursive_type_1_dwarf . > > Other than the extra arguments I can't think of any reason why this would > behave differently. Lit is just a simple wrapper that invokes dotest.py with > the > right arguments. > > > This does not have the patch that removes multiprocess support from > dotest. > > > > Ted > > > > > -----Original Message----- > > > From: Jonas Devlieghere <jo...@devlieghere.com> > > > Sent: Thursday, August 8, 2019 2:50 PM > > > To: Ted Woodward <tedw...@quicinc.com> > > > Cc: LLDB <lldb-dev@lists.llvm.org> > > > Subject: [EXT] Re: How do I use lit to only run the lldb test suite, > > > now that dotest multiprocessing capabilities have been removed? > > > > > > Hey Ted, > > > > > > 1. You can run just the dotest-tests by pointing lit at the `lit/Suite` > directory. > > > 2. You can pass arguments to dotest by passing `dotest-args` in --param. > > > > > > The invocation would look something like this: > > > > > > /path/to/llvm/bin/llvm-lit /path/to/lldb/lit/Suite --param > > > 'dotest-args=--foo -- bar' > > > > > > Hope that helps, > > > Jonas > > > > > > On Thu, Aug 8, 2019 at 9:31 AM Ted Woodward <tedw...@quicinc.com> > > > wrote: > > > > > > > > RE: https://reviews.llvm.org/D65311 > > > > > > > > > > > > > > > > Internally we use dotest to run the lldb test suite with various > > > > RTOS > > > configurations for the test binaries. In these runs we don’t care > > > about the lit tests or the unit tests, because they are OS agnostic. > > > We do this by specifying the compiler, lldb, and test flavor (static > > > testcase + os, dynamic library testcase loaded by an OS image, > > > dynamic library testcase loaded by an OS image running the OS’ debug > stub). > > > > > > > > > > > > > > > > With the multiprocess testrunner removed, how do I have lit: > > > > > > > > Only run the lldb test suite > > > > Run dotest with specific arguments > > > > > > > > > > > > > > > > We’re not running cmake; we’re taking an existing tools build and > > > > running > > > the tests from the source directory using the toolset. _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
