On 20. 5. 25 20:19, Timofei Zhakov wrote:
+def ensure_dependencies():
+ """Install the dependencies we need for running the tests.
+
+ NOTE: this function des not handle the case where the Python
+ version has changed. In theory, we could automagically
+ upgrade the venv in that case. In practice, we won't.
+ """
+
+ global dependencies_ensured
+ if dependencies_ensured:
+ return
+
+ package_path = os.path.join(venv_dir, "lib",
+ "python%d.%d" % sys.version_info[:2],
+ "site-packages")
+ package_path = os.path.abspath(package_path)
+ if package_path in sys.path:
+ dependencies_ensured = True
+ return
+
+ try:
+ # Create the virtual environment
+ if not os.path.isdir(venv_dir):
+ if os.path.exists(venv_dir):
+ safe_rmtree(venv_dir)
+ venv.create(venv_dir, with_pip=True)
+
+ # Install any (new) dependencies
+ pip = os.path.join(venv_dir, venv_bin, "pip"+_exe)
+ pip_options = ("--disable-pip-version-check",
"--require-virtualenv")
*+ subprocess.run([pip, *pip_options, "install",
*SVN_TESTS_REQUIRE],
+ check=True)*
+
+ sys.path.append(package_path)
+ dependencies_ensured = True
+ return package_path
+ except Exception as ex:
+ print("WARNING: Could not install test dependencies,"
+ " some tests will be skipped", file=sys.stderr)
+ print(ex, file=sys.stderr)
It is generally a great idea to verify the xml schema of the --xml output.
However, I'd not agree with the part where a pip library is installed
during the execution of the test script. Let me explain my point...
Imagine I am running the test suite in a clean environment and I don't
want to download anything from the internet. I download the library
source code distributions directly from tarballs, and verify the hash
to ensure it's the correct one. In the other words, doing anything to
avoid unwanted files appearing on my computer. But when I run the
tests, I'm implicitly getting this pip package of the latest version
on my machine.
You should read up a bit about how the Python venv module works. :) Pip
doesn't get installed from the 'net, some recent version of it is part
of the venv module which is part of every Python 3 installation.
Also, if I had the internet disabled, the tests would have been
skipped.. I don't think this would be the best choice for such users
who want their scripts to run without external things affecting their
configuration.
Also, we are avoiding to build our dependencies manually in the build
scripts. All the build systems I've dug down into (autoconf, cmake,
gen-win) are searching for the dependencies on either the system or in
a specific location. (except for the tools\dev\unix-build\Makefile.svn
script which is actually made to build all the dependencies from blank.
The external things installed by pip end up only inside the virtual
environment just created, inside the svn-test-work directory. None of
this pollutes the rest of the system. No root access required.
Instead of downloading the required library directly from the tests,
I'd suggest adding an option whether we want to verify schemas or not.
Then we can say that if so, you should install the library using your
own way. Also we can not only warn if the dependency could not be
located, but fail it straight away to avoid implicit changes of behaviour.
Add an option, yes, that's on my list. Make the user run a few lines of
Python manually – I don't agree. What can and should be improved is what
happens if the virtual environment already exists (/that/ is for the
internetly challenged, so that they can set up their Python dependencies
ahead of time).
Alternatively, if the license allows so and the library is like just a
few files and it's not that hard to maintain the versions, you just
could put its source code directly into the repo and assume that we
always have it in the source tree. As far as I understand, this is
currently used for ezt python generator (which is actually made by
gstein), lz4, utf8proc, but I think these libraries can also be
installed on the system.
Greg built ezt, yes, and at the time it was used almost exclusively by
Subversion. The package is now available on PyPI, so maybe we should use
that? lz4 and utf8proc, and support for amalgamated sqlite3, are there
because those packages were a royal pain in the (censored) to install on
Windows, in any useful form. Yes, I'm well aware why those are in our
source code, I added two of the three...
This is a case where we have a script used mainly by developers and a
way to easily get the dependencies onto any platform that has a Python 3
installation (without which the tests won't run).
And no, we certainly will not include lxml in our source tree.
Thoughts?
My first thought is that I already found a couple of nasty bugs in our
--xml output code, and I only just started. So indeed, I'll keep
improving on this. How we get lxml and rnc2rng installed without network
access is the least of my priorities right now.
PS: maybe it makes sense to add a possibility to run the whole test
suite in venv? I'm actually aware of how it works, sorry if I'm wrong...
Sure you can run the whole test suite like that, that's basically what
my changes are doing.
-- Brane
