labath wrote: > > Can't say I'm excited to see this feature (although I admit it is looking > > less daunting than I originally feared). Can you explain why you need this > > feature? Is there some particular aspect of lldb's remote operation that > > you think isn't well covered by the existing tests? Is there a particular > > category of tests that you'd like to have more than others? How many of the > > existing tests would exercise the remote platform capability in a > > meaningful way (a lot of these tests don't even build runnable binaries)? > > I'm sorry, it took a bit longer time to answer than I expected, my apologies. > > In general, our goal is to maximize the functionality of the testing toolkit > in case of cross-compilation+remote launch setup. > > I agree that not every Shell test launches binaries. According to my > grep-based statistics, it's roughly one-fifth of the tests currently (117/533 > test files that contain "r", "run", or "process launch" commands). Though > it's still a number.
Thank you for finding that number. > > Also, launching the "platform select" and "platform connect ..." commands > before executing the rest of a Shell test script may be useful. Here is an > example case of a potentially unexpected behavior #94165. So it's not always > necessary to have a Shell test build and run a binary to reveal some nuances. That's true, but this still seems like a pretty big hammer for that. > And it seems to me, that it won't cause excessive maintenance overhead since > we essentially add just two extra commands to %lldb invocation. Maybe. Like I said, it's cleaner than I though, but I'm still somewhat worried about where this will lead. If you can support running these tests on a different machine, then I think the next question becomes "why not run them with a different compiler as well". And then you need to XFAIL the tests for a particular compiler, so you end up replicating all of the `@skipIfClang71ExceptOnTuesdays` logic. I don't know if this is a slippery slope fallacy or an actual slippery slope, but this extra bit of test coverage does not seem worth it to me, and I sort of liked the separation how API tests can run in a multitude of configurations while a Shell test runs in one. I know that isn't really a good separation since "the way I write the tests" is in principle independent of "where I want to run it", but since the two tests use completely separate infrastructures, I think it would make sense to keep this functionality in just one of them. That said I don't want to be only person blocking this, so I'd like to hear what others think as well. @JDevlieghere ? > I haven't opened an RFC since I thought it would be clearer to discuss that > idea with the showcase of implementation. I'm open to moving a discussion > there if it's considered as a better approach in this case (please let me > know😅). Maybe the best would be an RFC with a link to a POC, but now that we're here, I think we can stick with it and see how it goes. https://github.com/llvm/llvm-project/pull/95986 _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
