Re: RFC: Python minimization in Fedora

2020-03-28 Thread Miro Hrončok
On 15. 01. 20 23:59, Zbigniew Jędrzejewski-Szmek wrote: ### Solution 5: Stop shipping mandatory bytecode cache This solution sounds simple: We do no longer ship the bytecode cache mandatorily. Technically, we move the `.pyc` files to a subpackage of `python3-libs` (or three different subpackages

Re: RFC: Python minimization in Fedora

2020-01-19 Thread Zbigniew Jędrzejewski-Szmek
On Sat, Jan 18, 2020 at 03:35:29PM -0500, James Cassell wrote: > > On Thu, Jan 16, 2020, at 5:16 AM, Zbigniew Jędrzejewski-Szmek wrote: > > > > A quick benchmark: > > $ time python3 -c 'import importlib as i, pydoc_data.topics as t; > > [i.reload(t) for _ in range(1)]' > > python3 -c 4.16s

Re: RFC: Python minimization in Fedora

2020-01-18 Thread James Cassell
On Thu, Jan 16, 2020, at 5:16 AM, Zbigniew Jędrzejewski-Szmek wrote: > > A quick benchmark: > $ time python3 -c 'import importlib as i, pydoc_data.topics as t; > [i.reload(t) for _ in range(1)]' > python3 -c 4.16s user 0.45s system 99% cpu 4.646 total [...] > sudo rm /usr/lib64/python3.7/p

Re: RFC: Python minimization in Fedora

2020-01-18 Thread Barry Scott
> On 15 Jan 2020, at 17:05, Miro Hrončok wrote: > > In Python Maint, we sat down and we came up with several ideas how to > minimize the filesystem footprint of Python. Unfortunately, the result is > horribly long, sorry about that. Did you calculate file sizes including rounding up by the "

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miro Hrončok
On 16. 01. 20 21:55, David Malcolm wrote: If a traceback for an exception includes files from the .zip, can the traceback-printing machinery still print the pertinent lines of source? Apparently no: $ echo 0/0 > t.py $ zip t.zip t.py adding: t.py (stored 0%) $ python -c 'import t' Traceback

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Nicolas Mailhot via devel
Le jeudi 16 janvier 2020 à 22:00 +0100, Felix Schwarz a écrit : > > If I understood Nicolas correctly this was about installing multiple > versions > of the same *library* in the global Python site-packages directory? Whatever you wish to call it :) The non stdlib parts projects do not seem to ag

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Felix Schwarz
Am 16.01.20 um 21:15 schrieb Zbigniew Jędrzejewski-Szmek: >> Accommodating component versioning would mean deploying >> >> /usr/lib/pythonxx/site-packages/something-semver.zip > > This path includes xx, which contains the major and minor numbers. So > adding "semver" would only allow accommodatin

Re: RFC: Python minimization in Fedora

2020-01-16 Thread David Malcolm
On Thu, 2020-01-16 at 10:27 +0100, Miro Hrončok wrote: > On 15. 01. 20 23:11, Victor Stinner wrote: > > > Solution 4: ZIP the entire standard library > > > (...) > > > Nevertheless, this might (in theory) **save 17.8 MiB / 47 %**. > > > > It's my favorite option. Almost 50% smaller is quite good!

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Zbigniew Jędrzejewski-Szmek
On Thu, Jan 16, 2020 at 03:36:11PM +0100, Nicolas Mailhot via devel wrote: > Le 2020-01-16 15:10, Felix Schwarz a écrit : > >Am 16.01.20 um 13:37 schrieb Nicolas Mailhot via devel: > >>If we start messing with the Python tree it would be nice to put > >>each shared > >>python component in a separat

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Neal Gompa
On Thu, Jan 16, 2020 at 2:13 PM Christian Glombek wrote: > > On a side note (and without reading all of the above in detail), I'd like to > note that Fedora CoreOS (aka FCOS) is completely Python free by now - > probably not achievable for Desktop, but it may be for IoT. > Unfortunately neither

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Christian Glombek
On a side note (and without reading all of the above in detail), I'd like to note that Fedora CoreOS (aka FCOS) is completely Python free by now - probably not achievable for Desktop, but it may be for IoT. Miroslav Suchý schrieb am Do., 16. Jan. 2020, 16:35: > Dne 15. 01. 20 v 18:05 Miro Hronč

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miroslav Suchý
Dne 15. 01. 20 v 18:05 Miro Hrončok napsal(a): > ### Solution 2: Move developer oriented modules to python3-devel (or split > the stdlib into pieces) +1 > ### Solution 5: Stop shipping mandatory bytecode cache +1 > Problem 5.1: Slower starts without bytecode cache Especially in container

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Nicolas Mailhot via devel
Le 2020-01-16 15:10, Felix Schwarz a écrit : Am 16.01.20 um 13:37 schrieb Nicolas Mailhot via devel: If we start messing with the Python tree it would be nice to put each shared python component in a separate zip/xz/whatever, and allow versioning those archives (ie use the highest semver zip

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Felix Schwarz
Am 16.01.20 um 13:37 schrieb Nicolas Mailhot via devel: > If we start messing with the Python tree it would be nice to put each shared > python component in a separate zip/xz/whatever, and allow versioning those > archives > > (ie use the highest semver zip present unless the code explicitely req

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Nicolas Mailhot via devel
Le 2020-01-16 11:18, Felix Schwarz a écrit : Am 15.01.20 um 23:11 schrieb Victor Stinner: This solution is well supported by unmodified Python: it's part of the default sys.path search path: $ python3 Python 3.7.6 (default, Dec 19 2019, 22:52:49) import sys; sys.path ['', '/usr/lib64/python37

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miro Hrončok
On 16. 01. 20 11:16, Zbigniew Jędrzejewski-Szmek wrote: So we clearly need to keep the possibility of installing .pyc files, at least optionally. To clarify: We would keep the .pyc files by default. We would only provide an opt-out. No, it will not (TTBMK). The file (or the lack of it) will

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Felix Schwarz
Am 15.01.20 um 23:11 schrieb Victor Stinner: > This solution is well supported by unmodified Python: it's part of the > default sys.path search path: > > $ python3 > Python 3.7.6 (default, Dec 19 2019, 22:52:49) import sys; sys.path > ['', '/usr/lib64/python37.zip', ...] > > It's the second

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Zbigniew Jędrzejewski-Szmek
On Thu, Jan 16, 2020 at 10:33:32AM +0100, Miro Hrončok wrote: > On 15. 01. 20 23:59, Zbigniew Jędrzejewski-Szmek wrote: > >On Wed, Jan 15, 2020 at 06:05:42PM +0100, Miro Hrončok wrote: > >>### File types (and bytecode caches) > >> > >>The orthogonal dimension is the file type. Python standard libra

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miro Hrončok
On 15. 01. 20 23:59, Zbigniew Jędrzejewski-Szmek wrote: On Wed, Jan 15, 2020 at 06:05:42PM +0100, Miro Hrončok wrote: ### File types (and bytecode caches) The orthogonal dimension is the file type. Python standard library contains directories with both "extension modules" (written in C (usually

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miro Hrončok
On 15. 01. 20 23:11, Victor Stinner wrote: Solution 4: ZIP the entire standard library (...) Nevertheless, this might (in theory) **save 17.8 MiB / 47 %**. It's my favorite option. Almost 50% smaller is quite good! It would be very efficient to have such disk space gain! Using a ZIP file for t

Re: RFC: Python minimization in Fedora

2020-01-16 Thread Miro Hrončok
On 15. 01. 20 18:56, Chris wrote: That's an amazing amount of work! Thanks. My only criticism would be: - the quest for reducing disk space is getting a bit over the top.  I mean to make comparisons to 3.5" floppy disks which haven't been around for 20 years? That is obviously only used t

Re: RFC: Python minimization in Fedora

2020-01-15 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Jan 15, 2020 at 06:05:42PM +0100, Miro Hrončok wrote: > ### File types (and bytecode caches) > > The orthogonal dimension is the file type. Python standard library > contains directories with both "extension modules" (written in C > (usually) and compiled to `*.cpython-38-x86_64-linux-gnu.

Re: RFC: Python minimization in Fedora

2020-01-15 Thread Victor Stinner
> Solution 4: ZIP the entire standard library > (...) > Nevertheless, this might (in theory) **save 17.8 MiB / 47 %**. It's my favorite option. Almost 50% smaller is quite good! It would be very efficient to have such disk space gain! Using a ZIP file for the stdlib is commonly suggested solution

Re: RFC: Python minimization in Fedora

2020-01-15 Thread Łukasz Posadowski
Wed, 15 Jan 2020 18:05:42 +0100 Miro Hrončok : > Hello Fedora! > > In Python Maint, we sat down and we came up with several ideas how to > minimize the filesystem footprint of Python. Unfortunately, the > result is horribly long, sorry about that. It was delightfull to read. I have some better u

Re: RFC: Python minimization in Fedora

2020-01-15 Thread Przemek Klosowski via devel
On 1/15/20 12:56 PM, Chris wrote: That's an amazing amount of work! My only criticism would be: - the quest for reducing disk space is getting a bit over the top.  I mean to make comparisons to 3.5" floppy disks which haven't been around for 20 years? Why is ~100MB so much? If you scale up from

Re: RFC: Python minimization in Fedora

2020-01-15 Thread Chris
That's an amazing amount of work! My only criticism would be: - the quest for reducing disk space is getting a bit over the top. I mean to make comparisons to 3.5" floppy disks which haven't been around for 20 years? Why is ~100MB so much? If you scale up from floppy disks and even reference a 8GB