On Tue, Apr 15, 2025 at 04:51:02PM +0800, Mauro Carvalho Chehab wrote: > Em Tue, 15 Apr 2025 16:40:34 +0800 > Mauro Carvalho Chehab <mchehab+hua...@kernel.org> escreveu: > > Em Tue, 15 Apr 2025 11:19:26 +0300 > > Andy Shevchenko <andriy.shevche...@intel.com> escreveu: > > > On Tue, Apr 15, 2025 at 11:17:12AM +0300, Andy Shevchenko wrote: > > > > On Tue, Apr 15, 2025 at 10:49:29AM +0300, Jani Nikula wrote: > > > > > On Tue, 15 Apr 2025, Andy Shevchenko <andriy.shevche...@intel.com> > > > > > wrote: > > > > > > On Tue, Apr 15, 2025 at 10:01:04AM +0300, Andy Shevchenko wrote: > > > > > >> On Mon, Apr 14, 2025 at 09:17:51AM -0600, Jonathan Corbet wrote: > > > > > >> > > > > > >> > Andy Shevchenko <andriy.shevche...@intel.com> writes: > > > > > >> > > On Wed, Apr 09, 2025 at 12:30:00PM -0600, Jonathan Corbet > > > > > >> > > wrote: > > > > > >> > >> Mauro Carvalho Chehab <mchehab+hua...@kernel.org> writes: > > > > > >> > >> > > > > > >> > >> > This changeset contains the kernel-doc.py script to replace > > > > > >> > >> > the verable > > > > > >> > >> > kernel-doc originally written in Perl. It replaces the > > > > > >> > >> > first version and the > > > > > >> > >> > second series I sent on the top of it. > > > > > >> > >> > > > > > >> > >> OK, I've applied it, looked at the (minimal) changes in > > > > > >> > >> output, and > > > > > >> > >> concluded that it's good - all this stuff is now in > > > > > >> > >> docs-next. Many > > > > > >> > >> thanks for doing this! > > > > > >> > >> > > > > > >> > >> I'm going to hold off on other documentation patches for a > > > > > >> > >> day or two > > > > > >> > >> just in case anything turns up. But it looks awfully good. > > > > > >> > >> > > > > > >> > > > > > > > >> > > This started well, until it becomes a scripts/lib/kdoc. > > > > > >> > > So, it makes the `make O=...` builds dirty *). Please make > > > > > >> > > sure this doesn't leave > > > > > >> > > "disgusting turd" )as said by Linus) in the clean tree. > > > > > >> > > > > > > > >> > > *) it creates that __pycache__ disaster. And no, .gitignore IS > > > > > >> > > NOT a solution. > > > > > >> > > > > > > >> > If nothing else, "make cleandocs" should clean it up, certainly. > > > > > >> > > > > > > >> > We can also tell CPython to not create that directory at all. > > > > > >> > I'll run > > > > > >> > some tests to see what the effect is on the documentation build > > > > > >> > times; > > > > > >> > I'm guessing it will not be huge... > > > > > >> > > > > > >> I do not build documentation at all, it's just a regular code > > > > > >> build that leaves > > > > > >> tree dirty. > > > > > >> > > > > > >> $ python3 --version > > > > > >> Python 3.13.2 > > > > > >> > > > > > >> It's standard Debian testing distribution, no customisation in the > > > > > >> code. > > > > > >> > > > > > >> To reproduce. > > > > > >> 1) I have just done a new build to reduce the churn, so, running > > > > > >> make again does nothing; > > > > > >> 2) The following snippet in shell shows the issue > > > > > >> > > > > > >> $ git clean -xdf > > > > > >> $ git status --ignored > > > > > >> On branch ... > > > > > >> nothing to commit, working tree clean > > > > > >> > > > > > >> $ make LLVM=-19 O=.../out W=1 C=1 CF=-D__CHECK_ENDIAN__ -j64 > > > > > >> make[1]: Entering directory '...' > > > > > >> GEN Makefile > > > > > >> DESCEND objtool > > > > > >> CALL .../scripts/checksyscalls.sh > > > > > >> INSTALL libsubcmd_headers > > > > > >> .pylintrc: warning: ignored by one of the .gitignore files > > > > > >> Kernel: arch/x86/boot/bzImage is ready (#23) > > > > > >> make[1]: Leaving directory '...' > > > > > >> > > > > > >> $ touch drivers/gpio/gpiolib-acpi.c > > > > > >> > > > > > >> $ make LLVM=-19 O=.../out W=1 C=1 CF=-D__CHECK_ENDIAN__ -j64 > > > > > >> make[1]: Entering directory '...' > > > > > >> GEN Makefile > > > > > >> DESCEND objtool > > > > > >> CALL .../scripts/checksyscalls.sh > > > > > >> INSTALL libsubcmd_headers > > > > > >> ... > > > > > >> OBJCOPY arch/x86/boot/setup.bin > > > > > >> BUILD arch/x86/boot/bzImage > > > > > >> Kernel: arch/x86/boot/bzImage is ready (#24) > > > > > >> make[1]: Leaving directory '...' > > > > > >> > > > > > >> $ git status --ignored > > > > > >> On branch ... > > > > > >> Untracked files: > > > > > >> (use "git add <file>..." to include in what will be committed) > > > > > >> scripts/lib/kdoc/__pycache__/ > > > > > >> > > > > > >> nothing added to commit but untracked files present (use "git add" > > > > > >> to track) > > > > > > > > > > > > FWIW, I repeated this with removing the O=.../out folder > > > > > > completely, so it's > > > > > > fully clean build. Still the same issue. > > > > > > > > > > > > And it appears at the very beginning of the build. You don't need > > > > > > to wait to > > > > > > have the kernel to be built actually. > > > > > > > > > > kernel-doc gets run on source files for W=1 builds. See > > > > > Makefile.build. > > > > > > > > Thanks for the clarification, so we know that it runs and we know that > > > > it has > > > > an issue. > > > > > > Ideal solution what would I expect is that the cache folder should respect > > > the given O=... argument, or disabled at all (but I don't think the latter > > > is what we want as it may slow down the build). > > > > From: > > > > https://github.com/python/cpython/commit/b193fa996a746111252156f11fb14c12fd6267e6 > > and: > > https://peps.python.org/pep-3147/ > > > > It sounds that Python 3.8 and above have a way to specify the cache > > location, via PYTHONPYCACHEPREFIX env var, and via "-X pycache_prefix=path". > > > > As the current minimal Python version is 3.9, we can safely use it. > > > > So, maybe this would work: > > > > make O="../out" PYTHONPYCACHEPREFIX="../out" > > > > or a variant of it: > > > > PYTHONPYCACHEPREFIX="../out" make O="../out" > > > > If this works, we can adjust the building system to fill PYTHONPYCACHEPREFIX > > env var when O= is used. > > That's interesting... Sphinx is already called with PYTHONDONTWRITEBYTECODE. > From Documentation/Makefile: > > quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) > cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) > $(build)=Documentation/userspace-api/media $2 && \ > PYTHONDONTWRITEBYTECODE=1 \ > ... > > It seems that the issue happens only when W=1 is used and kernel-doc > is called outside Sphinx. > > Anyway, IMHO, the best would be to change the above to: > > PYTHONPYCACHEPREFIX=$(abspath $(BUILDDIR)) > > And do the same for the other places where kernel-doc is called: > > include/drm/Makefile: $(srctree)/scripts/kernel-doc -none > $(if $(CONFIG_WERROR)$(CONFIG_DRM_WERROR),-Werror) $<; \ > scripts/Makefile.build: cmd_checkdoc = $(srctree)/scripts/kernel-doc > -none $(KDOCFLAGS) \ > scripts/find-unused-docs.sh: str=$(scripts/kernel-doc -export > "$file" 2>/dev/null) > > Comments?
I would like that, but it should be properly formed, because somewhere we drop the path in the source tree and if $O is used as is, it becomes _the_ pycache folder! -- With Best Regards, Andy Shevchenko
