Ned Deily <n...@python.org> added the comment:
Terry, using an out-of-date version of Tcl/Tk is the least of the problems here. My point was that, the way things are now, IDLE GUI tests are not being run at all on macOS Azure CI runs regardless of the Tk version in use and, AFAIK, there is no easy way to tell that from the default output from the tests. The particular test failures reported here *do* fail when properly configured to actually run using the Apple-supplied Tk 8.5.9 (in fact, with the latest fixes to the tests, all of test_idle passes with Tk 8.5.9). And I'm guessing that most times when you or Tal or other developers are running the tests from your own local builds, you aren't aware that the GUI tests are being skipped. There is a long-standing issue with Aqua Tk on macOS that, if a process tries to call Tk to draw objects on a screen but the process is not running under a username that is logged in through the normal macOS GUI (i.e. does not have a macOS Finder window, does not have a macOS "console"), Tk will segfault crashing Python along with it, which is very bad when running tests. There have been various issues about this going back a long time (see, for example, Issue22770 but there are older ones). To avoid those segfaults, we take a conservative approach in test.support._is_gui_available() (Lib/test/support/__init__.py) which is effectively (I believe) only allows GUI tests on macOS to run if they are being run under an application bundle which effectively means the Python under test must be a framework build (./configure ... --enable-framework=... ...). I believe that test is overly stringent but we haven't up to now found a reliable way to test and prevent the segfaults without depen ding on the framework approach. So (1) if you don't build Python with an --enable-framework= option, Tk and IDLE GUI tests are not going to be run. (2) If the GUI tests are skipped, there is no indication of this in the default log output. You only see this if you run, for example, with the regrtest -v (verbose) option: $ ./python.exe -m test -uall -j2 test_idle Run tests in parallel using 2 child processes 0:00:01 load avg: 2.56 [1/1] test_idle passed == Tests result: SUCCESS == 1 test OK. Total duration: 1 sec 753 ms Tests result: SUCCESS $ ./python.exe -m test -v -uall -j2 test_idle [...] Run tests in parallel using 2 child processes 0:00:01 load avg: 2.22 [1/1] test_idle passed skipped 'cannot run without OS X gui process' skipped 'cannot run without OS X gui process' skipped 'cannot run without OS X gui process' test_geticonname (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok test_getsublist (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok test_gettext (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok test_init (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok test_isexpandable (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok test_ondoubleclick (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest) ... ok skipped 'cannot run without OS X gui process' [...] skipped 'cannot run without OS X gui process' idlelib.idle_test.test_textview (unittest.loader.ModuleSkipped) ... skipped 'cannot run without OS X gui process' idlelib.idle_test.test_tooltip (unittest.loader.ModuleSkipped) ... skipped 'cannot run without OS X gui process' idlelib.idle_test.test_tree (unittest.loader.ModuleSkipped) ... skipped 'cannot run without OS X gui process' idlelib.idle_test.test_undo (unittest.loader.ModuleSkipped) ... skipped 'cannot run without OS X gui process' test_run_show (idlelib.idle_test.test_warning.RunWarnTest) ... ok test_showwarnings (idlelib.idle_test.test_warning.RunWarnTest) ... ok test_idle_formatter (idlelib.idle_test.test_warning.ShellWarnTest) ... ok test_shell_show (idlelib.idle_test.test_warning.ShellWarnTest) ... ok test_showwarnings (idlelib.idle_test.test_warning.ShellWarnTest) ... ok skipped 'cannot run without OS X gui process' test_init (idlelib.idle_test.test_window.WindowListTest) ... ok skipped 'cannot run without OS X gui process' ---------------------------------------------------------------------- Ran 257 tests in 1.001s OK (skipped=70) == Tests result: SUCCESS == 1 test OK. Total duration: 1 sec 836 ms Tests result: SUCCESS So there is a false sense of security that tests are being run when they aren't and those skipped GUI test might be failing if they were actually run as was the case in this issue. Possible followup actions: 1. Have test_idle (and test_ttk_guionly?) issue a warning message by default if GUI tests are being skipped. 2. Investigate if it is practical to run GUI tests under Azure CI (I'm guessing it is not) or one of the other CI runners we use (Appveyor maybe?). (Note I personally will not have time to look into this until at least the Sep dev sprint so it would be good for someone else to look into it). That would also require changing the CI to make and test framework builds. 3. It *should* be possible to run GUI tests on a macOS buildbot where the console is logged into the buildbot user. That will also take some investigation and reconfiguration as a framework build. 4. Find an easier way to run macOS tests with an up-to-date Tk. I have plans to provide something in the near future. 5. It may be possible to loosen the test for GUI environment so that a framework build is not necessary but having a logged-in "console" user would need to happen anyway so that's much less important than the other items. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue37706> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
