Hi, I'd like to edit this to update some stuff (there are broken links, and outdated packages), as well as to include a reference to Scalene, a Python profiler I developed that offers a number of useful advantages over other profilers.
Thanks, -- emery (Scalene developer, among many other things) >From https://github.com/emeryberger/scalene: Scalene is a high-performance CPU *and* memory profiler for Python that does a few things that other Python profilers do not and cannot do. It runs orders of magnitude faster than other profilers while delivering far more detailed information. 1. Scalene is *fast*. It uses sampling instead of instrumentation or relying on Python's tracing facilities. Its overhead is typically no more than 10-20% (and often less). 2. Scalene is *precise*. Unlike most other Python profilers, Scalene performs CPU profiling *at the line level*, pointing to the specific lines of code that are responsible for the execution time in your program. This level of detail can be much more useful than the function-level profiles returned by most profilers. 3. Scalene separates out time spent running in Python from time spent in native code (including libraries). Most Python programmers aren't going to optimize the performance of native code (which is usually either in the Python implementation or external libraries), so this helps developers focus their optimization efforts on the code they can actually improve. 4. Scalene *profiles memory usage*. In addition to tracking CPU usage, Scalene also points to the specific lines of code responsible for memory growth. It accomplishes this via an included specialized memory allocator. Below is a table comparing the performance of various profilers to Scalene: ProfilerTimeSlowdown *original program* 6.71s 1.0x cProfile 11.04s 1.65x Profile 202.26s 30.14x pyinstrument 9.83s 1.46x line_profiler 78.0s 11.62x pprofile *(deterministic)* 403.67s 60.16x pprofile *(statistical)* 7.47s 1.11x yappi *(CPU)* 127.53s 19.01x yappi *(wallclock)* 21.45s 3.2x py-spy 7.25s 1.08x memory_profiler *> 2 hours* >1000x scalene *(CPU only)* 6.98s 1.04x scalene *(CPU + memory)* 7.68s 1.14x the features of other profilers vs. Scalene: ProfilerLine-level?CPU?Wall clock vs. CPU time?Python vs. native?Memory?Unmodified code?Threads? cProfile ✔ wall clock ✔ Profile ✔ CPU time ✔ pyinstrument ✔ wall clock ✔ line_profiler ✔ ✔ wall clock pprofile *(deterministic)* ✔ ✔ wall clock ✔ ✔ pprofile *(statistical)* ✔ ✔ wall clock ✔ ✔ yappi *(CPU)* ✔ CPU time ✔ ✔ yappi *(wallclock)* ✔ wall clock ✔ ✔ py-spy ✔ ✔ both ✔ ✔ memory_profiler ✔ ✔ scalene *(CPU only)* ✔ ✔ both ✔ ✔ ✔ scalene *(CPU + memory)* ✔ ✔ both ✔ ✔ ✔ ✔ -- Professor Emery Berger College of Information and Computer Sciences University of Massachusetts Amherst www.emeryberger.org, @emeryberger
_______________________________________________ pydotorg-www mailing list [email protected] https://mail.python.org/mailman/listinfo/pydotorg-www
