> On Aug 15, 2018, at 12:27 PM, Zachary Turner <[email protected]> wrote:
>
> Back to the original proposal, my biggest concern is that a single inline
> test could generate many FileCheck invocations. This could cause measurable
> performance impact on the test suite. Have you considered this?
That's a good point. I hadn't considered that. My thoughts on that are;
- It's relatively cheap to create a FileCheck process. If the build is
(A|T)sanified, we can copy in a non-sanitized FileCheck to speed things up.
- Based on the time it takes to run check-{llvm,clang} locally, which have
~56,000 FileCheck invocations, my intuition is that the overhead ought to be
manageable.
- The status quo is doing Python's re.search over a chunk of command output. My
(unverified) intuition is that FileCheck won't be slower than that. Actually,
FileCheck has an algorithmic advantage because it doesn't re-scan the input
text from the beginning of the text each time it tries to match a substring.
`self.expect` does.
>
> Another possible solution is what i mentioned earlier, basically to expose a
> debugger object model. This would allow you to accomplish what you want
> without FileCheck, while simultaneously being making many other types of
> tests easier to write at the same time. On the other hand, it’s a larger
> effort to create this system, but I think long term it would pay back
> enormously (it’s even useful as a general purpose debugger feature, not
> limited to testing)
I'd volunteer to work on that. At the moment I really need to get some form of
testing put together for my patches soon.
vedant
>
> On Tue, Aug 14, 2018 at 5:31 PM Vedant Kumar via lldb-dev
> <[email protected] <mailto:[email protected]>> wrote:
> Hello,
>
> I'd like to make FileCheck available within lldb inline tests, in addition to
> existing helpers like 'runCmd' and 'expect'.
>
> My motivation is that several tests I'm working on can't be made as rigorous
> as they need to be without FileCheck-style checks. In particular, the
> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow
> me to verify the ordering of checked input, to be stringent about line
> numbers, or to capture & reuse snippets of text from the input stream.
>
> I'd curious to know if anyone else is interested or would be willing to
> review this (https://reviews.llvm.org/D50751
> <https://reviews.llvm.org/D50751>).
>
> Here's an example of an inline test which benefits from FileCheck-style
> checking. This test is trying to check that certain frames appear in a
> backtrace when stopped inside of the "sink" function. Notice that without
> FileCheck, it's not possible to verify the order in which frames are printed,
> and that dealing with line numbers would be cumbersome.
>
> ```
> ---
> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> +++
> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> @@ -9,16 +9,21 @@
>
> volatile int x;
>
> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
> void __attribute__((noinline)) sink() {
> - x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3',
> 'sink'])
> + x++; //% self.filecheck("bt", "main.cpp")
> }
>
> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
> void __attribute__((noinline)) func3() { sink(); /* tail */ }
>
> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
> void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /*
> regular */ }
>
> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
> void __attribute__((noinline)) func1() { func2(); /* tail */ }
>
> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
> int __attribute__((disable_tail_calls)) main() {
> func1(); /* regular */
> return 0;
> ```
>
> For reference, here's the output of the "bt" command:
>
> ```
> runCmd: bt
> output: * thread #1, queue = 'com.apple.main-thread', stop reason =
> breakpoint 1.1
> * frame #0: 0x000000010c6a6f64 a.out`sink() at main.cpp:14 [opt]
> frame #1: 0x000000010c6a6f70 a.out`func3() at main.cpp:15 [opt]
> [artificial]
> frame #2: 0x000000010c6a6f89 a.out`func2() at main.cpp:21 [opt]
> frame #3: 0x000000010c6a6f90 a.out`func1() at main.cpp:21 [opt]
> [artificial]
> frame #4: 0x000000010c6a6fa9 a.out`main at main.cpp:28 [opt]
> ```
>
> thanks,
> vedant
> _______________________________________________
> lldb-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> <http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>
_______________________________________________
lldb-dev mailing list
[email protected]
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
