Alecsandru Patrascu added the comment:
Serhiy, I am curious why did you have chosen to compute the CRC32 table
everytime? It is standard (the generator polynomial does not change) and always
will output the same values. And it is also less computational intensive to
loading a precomputed
Alecsandru Patrascu added the comment:
Hello Brett, I saw that there was no other activity on this issue so far. Can
you please update this, as I do not have rights?
--
___
Python tracker
<http://bugs.python.org/issue26
Alecsandru Patrascu added the comment:
I've updated the patch for 2.7 branch to contain the LTO part merged by Gregory.
--
Added file:
http://bugs.python.org/file43142/cpython2_with_optimizations_v02.patch
___
Python tracker
<http://bugs.py
Alecsandru Patrascu added the comment:
The patches in discussions are the v02 ones.
--
___
Python tracker
<http://bugs.python.org/issue26359>
___
___
Python-bug
New submission from Alecsandru Patrascu:
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation.
I would like to submit a request to turn-on Profile Guided Optimization or PGO
as the default build option for Python (both 2.7 and 3.6), given its
Changes by Alecsandru Patrascu :
--
keywords: +patch
Added file: http://bugs.python.org/file40231/python2.7-pgo.patch
___
Python tracker
<http://bugs.python.org/issue24
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40232/python3.6-pgo.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Python-bug
Changes by Alecsandru Patrascu :
Removed file: http://bugs.python.org/file40226/python2.7-3.6-pgo.zip
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I added the patches as individual files and removed the zip file.
--
___
Python tracker
<http://bugs.python.org/issue24
Alecsandru Patrascu added the comment:
The patches are tested on Linux machines, with GNU GCC >4.8.3. From your output
I see that you are using the CLANG compiler. CLANG uses a different set of
flags for PGO that are not compatible with GCC's, therefore the compilation
will fail.
Alecsandru Patrascu added the comment:
I received the review and will post new patch versions as soon as I update
them.
Regarding PGO on clang, I will need a bit more time to edit the Makefile and
will post it just for clang, to be easier for us to see the differences
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40236/python2.7-pgo-v02.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40239/python3.6-pgo-v02-mac.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40238/python2.7-pgo-v02-mac.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Alecsandru Patrascu added the comment:
I modified the patches after the review made by Brett (python2.7-pgo-v02.patch
and python3.6-pgo-v02.patch):
- removed the call to pybench
- left the PGO steps as optional. To use it we run "make profile-opt"
- in the initial patches,
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40237/python3.6-pgo-v02.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40255/python3.5-pgo-v03.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I modified the patches to be compatible with both environments. The new
versions modify the configure.ac file also, therefore you will need to run
"autoconf" by hand. Also, in case of MaOS you will need to have llvm-profdata
installed and in
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40254/python2.7-pgo-v03.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
Sorry, it was a typo. I made a correction to it. I will also modify to -m flag,
instead of the explicit file execution.
Regarding the clang/gcc support, in v03 version of patches, GCC is supported.
On Linux is straightforward. On Mac I see that the
Alecsandru Patrascu added the comment:
Thank you for the clarifications! Your point make sense, we don't want to
exclude clang environments. I will analyze this and post some patches once I'm
done with it.
--
___
Python trac
Alecsandru Patrascu added the comment:
I modified the patches with clang support.
Also, I added an important check for the architecture on which PGO is running.
Our proposal targets x86 platforms, since our measurements are made only on x86
hardware
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40262/python2.7-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40263/python3.6-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Removed file: http://bugs.python.org/file40262/python2.7-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Changes by Alecsandru Patrascu :
Removed file: http://bugs.python.org/file40263/python3.6-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40268/python3.6-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40267/python2.7-pgo-v04.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40274/python3.6-pgo-v05.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40273/python2.7-pgo-v05.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I fixed the files after the review. Regarding the PROFILE_OPT_OLD line, I think
that it is better to keep also the old task used for PGO, until clear evidence
and measurements that regrtest is performing better on other architectures
exists
Alecsandru Patrascu added the comment:
The profile merging is necessary in case you want to use a pure clang compiler
or you use GCC in OSX. For example, a general profiling action using clang will
result in at least one binary profile. For our case, when using regrtest, we
will have multiple
Alecsandru Patrascu added the comment:
I've updated the patches after review and implemented the checkup for
llvm-profdata for both Linux and OSX.
--
___
Python tracker
<http://bugs.python.org/is
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40289/python3.6-pgo-v06.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40288/python2.7-pgo-v06.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
That's a good point Skip. I added another set of patches, just for the README
files, explaining the entire procedure, so now anyone reading it will see that
PGO is available, what are the steps involved and a brief comment about the
wa
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40299/README2.7-pgo-v01.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40300/README3.6-pgo-v01.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I've updated the patches with redirect to /dev/null, as is it is more clearer
to the user what is our intent, without having him to necessarily read the
regrtest documentation. I've also added a warning message regarding the output
and ported
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40389/README2.7-pgo-v02.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40390/README3.6-pgo-v02.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I named this task PROFILE_TASK_X86 because it is rigorously tested and we have
proven that regrtest performs better on this architecture. Until any other
clear evidence and solid measurements that regrtest is performing better on
other architectures
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40391/python2.7-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Removed file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40392/python3.6-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Changes by Alecsandru Patrascu :
Removed file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Alecsandru Patrascu added the comment:
If you are talking just about the 3.6 patch, it is called this way to emphasize
the fact that it is intended for the development branch. It is perfectly
compatible with 3.5, therefore it is not needed for packagers to maintain two
distinct versions. I&#x
New submission from Alecsandru Patrascu:
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation.
I would like to submit a patch that improves the performance of the hash
computation code on stringobject, bufferobject and unicodeobject. As can be
Alecsandru Patrascu added the comment:
Yes, sure it can be implemented in smaller space, as Serhiy and Chris well
pointed out. I submitted the 01 version like that just to be clear that I don't
want to re-implement a new hash computing value and just unroll the loop in the
existing o
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40460/hash8-v02.patch
___
Python tracker
<http://bugs.python.org/issue25106>
___
___
Python-bug
Alecsandru Patrascu added the comment:
I've submitted a more compact version of the previous code (hash8-v02.patch)
--
___
Python tracker
<http://bugs.python.org/is
Alecsandru Patrascu added the comment:
This patch is not applicable to Python 3, as it already has a better hash
function and has the same unrolling technique applied.
As Brett well observed, it is a backport of an optimization technique used in
Python 3, applied to the existing hash
Alecsandru Patrascu added the comment:
Yes, it doesn't break the compatibility with existing applications. To make
sure, we tested it in various scenarios and workloads.
--
___
Python tracker
<http://bugs.python.org/is
Alecsandru Patrascu added the comment:
Please hold this patch for a while, while I analyze the received feedback.
--
___
Python tracker
<http://bugs.python.org/issue25
Alecsandru Patrascu added the comment:
Thank you for upstreaming this in both branches of Python!
Do you think that a different version of regrtest.py, that will be used only
for PGO training, should be better in this case? I mean, by implementing a
custom version, I think we can control
Alecsandru Patrascu added the comment:
I can work on modifying the existing regrtest and adding a distinct flag, --pgo
for example, as Antoine suggested. Indeed, it will not be trivial as regrtest
has a dual approach (single process and multi process), but I will give it a
try and post a
New submission from Alecsandru Patrascu:
This issue adds improved support for Profile Guided Optimization builds in the
regression test suite.
Mainly, we keep the visual progress status in this process and suppress errors
or any other unnecessary output that can alarm users
Changes by Alecsandru Patrascu :
--
keywords: +patch
Added file: http://bugs.python.org/file40532/regrtest_27.patch
___
Python tracker
<http://bugs.python.org/issue25
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40533/regrtest_36.patch
___
Python tracker
<http://bugs.python.org/issue25188>
___
___
Python-bug
Alecsandru Patrascu added the comment:
I've added the patches for Python default and 2.7. To apply them please use the
command "hg import --no-commit". PGO flag is implemented for both variants
(single and multi process). Default in profile-opt will be
Alecsandru Patrascu added the comment:
The patch does not display dots, as they are not that intuitive and users
cannot make an estimate of the time left or the position in the training phase.
Instead, the regrtest.py, will display the progress as [X/Y], like the default
version, but without
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file40547/regrtest_36_v02.patch
___
Python tracker
<http://bugs.python.org/issue25188>
___
___
Pytho
Alecsandru Patrascu added the comment:
Thank you for the link. I modified regrtest.py for Python3 according to the
specifications.
--
___
Python tracker
<http://bugs.python.org/issue25
Alecsandru Patrascu added the comment:
Thank you for the tweaks. I tested the patch and it is all working fine.
Without failure count the PGO run is even more transparent to the user.
I tested your final version of the patch on several Linux server and regular
desktop distributions and
Alecsandru Patrascu added the comment:
Thank you Brett for your effort to merge all the changes made to regrtest
despite the changes appeared on the way!
--
___
Python tracker
<http://bugs.python.org/issue25
New submission from Alecsandru Patrascu:
Title: Link Time Optimizations support for GCC and CLANG
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation. I would like to submit a patch that adds support for Link Time
Optimization (LTO) when using
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41131/lto-cpython3-v01.patch
___
Python tracker
<http://bugs.python.org/issue25702>
___
___
Pytho
Changes by Alecsandru Patrascu :
--
keywords: +patch
Added file: http://bugs.python.org/file41130/lto-cpython2-v01.patch
___
Python tracker
<http://bugs.python.org/issue25
Alecsandru Patrascu added the comment:
LTO exists in GCC since version 4.5, but it is true that only recent versions
(>=4.8) perform it in good conditions. It is not enabled by default in this
patch, it is only available when building with PGO support. Running just "make"
will
Alecsandru Patrascu added the comment:
Meanwhile I've added the patches (v02) for LTO enabled only if the "./configure
--with-lto" command is issued.
--
___
Python tracker
<http://bugs.pyt
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41138/lto-cpython3-v02.patch
___
Python tracker
<http://bugs.python.org/issue25702>
___
___
Pytho
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41137/lto-cpython2-v02.patch
___
Python tracker
<http://bugs.python.org/issue25702>
___
___
Pytho
Alecsandru Patrascu added the comment:
Hi, I added a dedicated issue just for LTO only when using GCC and CLANG
(http://bugs.python.org/issue25702), that works well with PGO also.
--
nosy: +alecsandru.patrascu
___
Python tracker
<h
Alecsandru Patrascu added the comment:
Thank you David for including me in this issue.
On ICC, when executing the instrumented applications, it will generate dynamic
information file that has a unique name and .dyn suffix. From my previous
experience with GCC and CLANG PGO patch (issue24915
New submission from Alecsandru Patrascu:
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation. I would like to submit a patch that adds extended support for
coverage, in the same way that it is used in CPython3. Used in conjunction with
the
Alecsandru Patrascu added the comment:
Thank you for the reviews. I've added a new patch (v02) to correct the issues.
--
Added file: http://bugs.python.org/file41386/coverage-2.7-v02.patch
___
Python tracker
<http://bugs.python.org/is
Alecsandru Patrascu added the comment:
I'm adding Brett, Gregory, Stefan and Victor as nosy because this issue might
be interesting for them also.
--
nosy: +brett.cannon, gregory.p.smith, haypo, scoder, skrah
___
Python tracker
Alecsandru Patrascu added the comment:
Hello, I've added an updated set of patches (v03) for the current CPython2 and
CPython3 codebase. Also made some small changes to reduce the number of places
where the flags are set.
--
Added file: http://bugs.python.org/file41493/lto-cpy
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41494/lto-cpython3-v03.patch
___
Python tracker
<http://bugs.python.org/issue25702>
___
___
Pytho
Alecsandru Patrascu added the comment:
Thank you for your feedback, I've updated the patches and now LTO flags are
used only when building with PGO (v04). CFLAGS/LDFLAGS remain untouched, as
Antoine and Victor suggested is better.
--
Added file: http://bugs.python.org/file4166
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41666/lto-cpython3-v04.patch
___
Python tracker
<http://bugs.python.org/issue25702>
___
___
Pytho
Alecsandru Patrascu added the comment:
As Steve mentioned, the Microsoft compiler uses LTO (they call it Link-Time
Code Generation) and the flags are used when compiling CPython on Windows
systems. Thus our proposal to enable it on GCC and CLANG also
Changes by Alecsandru Patrascu :
--
nosy: +alecsandru.patrascu
___
Python tracker
<http://bugs.python.org/issue26110>
___
___
Python-bugs-list mailing list
Unsub
Alecsandru Patrascu added the comment:
Yury, thank you for the heads up! Here at Intel, in the Dynamic Scripting
Languages Optimization Team, we can help the community with reviewing and
measuring this patch in our quiet and stable environment, the same one that we
use to provide public
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41806/cpython3-deadcode-v01.patch
___
Python tracker
<http://bugs.python.org/issue26285>
___
___
New submission from Alecsandru Patrascu:
Hi all,
This is Alecsandru from the Dynamic Scripting Languages Optimization Team at
Intel Corporation. I would like to submit a patch that enables garbage
collection of unused input sections from the CPython2 and CPython3 binaries, by
using the "
Alecsandru Patrascu added the comment:
I realize now that I should have explained a bit more the background of this
patch. I'll do this now, for everyone to be clear of what is the effect of
those flags.
This issue was revealed after running the coverage target over various
workloads
Alecsandru Patrascu added the comment:
Sure, I attached them as files because they have a lot of lines for posting
here (~90 in total).
The linker offers the possibility to show what piece of data/functions was
removed, but I intentionally omitted it in order not to clutter the build
trace
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41809/gc-removed-cpython3.txt
___
Python tracker
<http://bugs.python.org/issue26285>
___
___
Pytho
Alecsandru Patrascu added the comment:
I've done again the experiments on larger workloads, such as our OpenStack
Swift cluster, and it works without any issues.
Also, I've attached an archive with a simple external module in CPython3 that
uses PyMem_RawMalloc. The output is ok
Alecsandru Patrascu added the comment:
I attached the list for CPython3 (gc-removed-zones-cpython3.txt), but now split
in two sections (core and parser), for more clarity as from what and where it
is removed. As you can see, the reason why the module works is because the API
that can be used
Alecsandru Patrascu added the comment:
Hello and thank you for your feedback. For CPython this does not apply because
due to the structure of the build system, inside the "build" directory there
are no PGO profiles saved. You can run find . -name '
Alecsandru Patrascu added the comment:
That's interesting. Even on CPython3, I still don't see any gcda's inside the
build directory, nor the tree structure you are seeing there. Can you please
give me a couple of details regarding your environment (os, distribution, gcc
ver
Alecsandru Patrascu added the comment:
For my responses, I modified locally the Makefile so that it will not remove
the build directory and any of the gcda files. I will make some more tests
tomorrow, but i think that this problem will solve simpler if the removal of
the build directory is
Changes by Alecsandru Patrascu :
--
nosy: +alecsandru.patrascu
___
Python tracker
<http://bugs.python.org/issue26307>
___
___
Python-bugs-list mailing list
Unsub
Alecsandru Patrascu added the comment:
Thank you for noticing and signaling this issue. Since I proposed the PGO
patches, I will fix it in another patch, as I don't want it to break any builds.
--
___
Python tracker
<http://bugs.py
Changes by Alecsandru Patrascu :
Added file: http://bugs.python.org/file41917/pgofix-cpython3.patch
___
Python tracker
<http://bugs.python.org/issue24915>
___
___
Pytho
Alecsandru Patrascu added the comment:
I've added a fix for the PGO builds after a issue pointed out in #26307. Thank
you Christos for your observation!
--
Added file: http://bugs.python.org/file41916/pgofix-cpython2.patch
___
Python tracker
1 - 100 of 129 matches
Mail list logo
