On Fri, Nov 8, 2019 at 6:01 PM Charalampos Stratakis <cstra...@redhat.com> wrote: > > > > ----- Original Message ----- > > From: "Vít Ondruch" <vondr...@redhat.com> > > Cc: devel@lists.fedoraproject.org > > Sent: Friday, November 8, 2019 10:01:47 AM > > Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to > > statically link with libpython3.8.a for better > > performance > > > > > > Dne 07. 11. 19 v 23:08 Florian Weimer napsal(a): > > > * Vít Ondruch: > > > > > >> Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a): > > >>> On 07/11/2019 14:59, Victor Stinner wrote: > > >>> > > >>>> I cannot explain why PLT is needed when a libpython function calls a > > >>>> libpython function. > > >>> Because an exported symbol in an ELF shared library is subject to > > >>> potential interposition using LD_PRELOAD so the calls need to go > > >>> through the PLT to be resolved. > > >> > > >> Not sure what PLT is (pre load table?), but is it something what could > > >> be disabled? > > > Procedure Linkage Table. > > > > > > It can be disabled by using hidden symbols. For internal symbols, that > > > is easy. For symbols that are used externally, I do not think we have > > > good toolchain support. Link-time optimization can detect truly > > > internal symbols and make them hidden. Some targets can also perform > > > relaxation of relocations, eliminating most of the PLT indirection > > > overhead if it turns out a function is not exported after all and > > > therefore cannot be interposed. But that needs a version script, and it > > > can't work for calls to functions that are in fact public. > > > > > > In glibc, we create hidden aliases for public functions which should not > > > be interposed. It's not too bad if you have preprocessor macros for > > > this task, but you do need to annotate each function declaration and > > > definition separately. > > > > > >> This sounds like the whole system could be 25% faster if we link > > >> statically. > > > Not reallly, quite a few system components already do this kind of > > > optimization. > > > > > > Toolchain support for this is quite poor however. Ideally, we would > > > have a compilation mode that reuses the annotations that Windows uses, > > > but given that our system headers currently lack __dllimport specifiers > > > (or whatever they are called), even with that approach, it's quite a lot > > > of work. I might be mistaken about this, but I think there was a huge > > > conflict about some intermediate visibility setting (protected?) that > > > might help with this, basically creating non-interposable function > > > symbols, but I don't think it's usable for that in its current state. > > > > > > Thx for explanation Florian. > > > > > > Generally, I am against this change proposal, because: > > > > 1) Apparently, there is some work which needs to be done on the > > toolchain. Applying workarounds just hides the issues and we won't move > > forward ever. > > I think it's more reasonable to do a small SPEC change in Python to achieve a > 27% performance boost, than wait for the toolchain to catch up on things that > are not well defined yet. I don't see that as a valid reason for not > accepting the change, although you might want to elaborate further here. > > > > > 2) I am asking this questions, because I believe that the same issue > > might suffer Ruby and others are concerned about Perl. Applying this > > just to Python is not systematic. > > Maybe. Systematic or not when compared to other dynamic languages is not > really relevant for this change to take effect. I don't know about Perl's or > Ruby's architecture design, but is there a reason to keep them in line in > that aspect? Or for any other aspect at all, apart from the general packaging > guidelines? > > > > > However, if part of this change proposal was actually collecting the > > information what have to be done to have similar performance for the > > dynamically linked libraries comparing to static linking, if there is > > push to improve the toolchain and if there is generally better > > understanding of the issue, then I would not mind if this is accepted as > > temporary measure. Unfortunately, nothing like this is mentioned in the > > change proposal. > > > > No this is not intended to be temporal, hence why it's not mentioned as such. > The information has been collected as a case for the change. If other > languages would like to conduct similar benchmarks and experiments they are > free to do so, but the scope of this change is just for Python. Also it's not > intended as a push for certain toolchain changes/optimizations, although thet > would be more than welcome. In addition this change is not a case for the > benefits of static linking in general, as we are not aware or experimented on > other things, we outlined a specific case, which is in line with the > packaging guidelines, since statically linking a binary against its own > library is permitted. > > This change intends to speed up Python by a significant margin in the context > of the current toolchain, with the outlined effects on compatibility and > package changes. We can debate on that, the rest seem irrelevant in the > context of accepting it or not.
I agree. The benefit of this change for python (and hence, a big amount of software that runs on both users' *and* fedora infra machines) is rather big, while the impact on other packages is either non-existent, or pretty small. Additionally, I think Miro mentioned that this change was already done in other distros, so we're also not the first ones doing this. (Side note: I wouldn't even have objected to this being a Self-Contained Change, since it basically only affects one package - albeit an important one (python3) - and the Change owners have offered to help with adapting other packages, in the very few cases where that would be necessary. I also trust the members of the Python SIG have done / will do their homework here, as they have done with both the python 3.8 transition and python2 retirement, which were / are both being executed almost flawlessly.) Fabio > > > > Vít > > > > > > > > > > Thanks, > > > Florian > > _______________________________________________ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > > > -- > Regards, > > Charalampos Stratakis > Software Engineer > Python Maintenance Team, Red Hat > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
