On 6 February 2018 at 04:11, Davide Italiano via lldb-dev <lldb-dev@lists.llvm.org> wrote: > Hi, > in the last couple of months a lot of people put a lot of attentions > and energy into lldb and we're starting to see the first results. I > decided to sit down and write this e-mail to state where we are and > what are some possible future directions for the projects, in terms of > better quality/higher testability. > > Current state: > > 1) We got the testsuite on MacOS to build with zero unexpected > successes and zero failures (modulo one change I'm going to push > tomorrow). This is a collaborative effort and it's very important > because it allows us to push for unexpected successes as failures on > the bots, allowing us to discover issues quicker. Other platform I > think are improving their state as well, mainly thanks to the work of > Pavel and Jan.
I don't mean to belittle this statement, as I think the situation has definitely improved a lot lately, but I feel I have to point out that's I've never been able to get a clean test suite run on a mac (not even the "0 failures" kind of clean). I'm not sure what are these caused by, but I guess that's because the tests are still very much dependent on the environment. So, I have to ask: what kind of environment are you running those tests in? My machine is not a completely off-the-shelf mac, as it has some google-specific customizations. I don't really know what this encompasses, but I would be surprised if these impact the result of the test suite. If I had to bet, my money would be on your machines having some apple-specific stuff which is not available in regular macs. I tried an experiment today. I configured with: cmake ../../llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -GNinja. First problem I ran into was that I couldn't run check-lldb, as the clang I have just built was unable to compile any of the test binaries due to missing headers (this could be a manifestation of the SDKROOT issue we ran into a couple of weeks ago). So, I tried running with the system compiler and I got this output: FAIL: test_c_global_variables_dwarf (lang/c/global_variables/TestGlobalVariables.py) FAIL: test_c_global_variables_gmodules (lang/c/global_variables/TestGlobalVariables.py) FAIL: test_dsym (functionalities/ubsan/basic/TestUbsanBasic.py) FAIL: test_dwarf (functionalities/ubsan/basic/TestUbsanBasic.py) FAIL: test_gmodules (functionalities/ubsan/basic/TestUbsanBasic.py) FAIL: test_with_python_api_dsym (lang/cpp/class_static/TestStaticVariables.py) FAIL: test_with_python_api_dwarf (lang/cpp/class_static/TestStaticVariables.py) FAIL: test_with_python_api_gmodules (lang/cpp/class_static/TestStaticVariables.py) ERROR: test_debug_info_for_apple_types_dsym (macosx/debug-info/apple_types/TestAppleTypesIsProduced.py) ERROR: test_debug_info_for_apple_types_dwarf (macosx/debug-info/apple_types/TestAppleTypesIsProduced.py) ERROR: test_debug_info_for_apple_types_gmodules (macosx/debug-info/apple_types/TestAppleTypesIsProduced.py) UNEXPECTED SUCCESS: test_lldbmi_output_grammar (tools/lldb-mi/syntax/TestMiSyntax.py) UNEXPECTED SUCCESS: test_process_interrupt_dsym (functionalities/thread/state/TestThreadStates.py) UNEXPECTED SUCCESS: test_process_interrupt_gmodules (functionalities/thread/state/TestThreadStates.py) So, I guess my question is: are you guys looking into making sure that others are also able to reproduce the 0-fail+0-xpass state? I would love to run the mac test suite locally, as I tend to touch a lot of stuff that impacts all targets, but as it stands now, I have very little confidence that the test I am running reflect in any way the results you will get when you run the test on your end. I am ready to supply any test logs or information you need if you want to try to tackle this. > 3) In the short term I plan to remove support for unmaintained > languages (Java/Go/Swift). This allows us to bring them back again (or I hope you meant OCaml instead of Swift. :P _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
