On 2021-11-20 17:40, Marco Sulla wrote:
Indeed I have introduced a command line parameter in my bench.py
script that simply specifies the number of times the benchmarks are
performed. This way I have a sort of segfault checker.
But I don't bench any part of the library. I suppose I have to create
a separate script that does a simple loop for all the cases, and
remove the optional parameter from bench. How boring.
PS: is there a way to monitor the Python consumed memory inside Python
itself? In this way I could also trap memory leaks.
I'm on Windows 10, so I debug in Microsoft Visual Studio. I also have a
look at the memory usage in Task Manager. If the program uses more
memory when there are more iterations, then that's a sign of a memory
leak. For some objects I'd look at the reference count to see if it's
increasing or decreasing for each iteration when it should be constant
over time.
On Sat, 20 Nov 2021 at 01:46, MRAB <pyt...@mrabarnett.plus.com> wrote:
On 2021-11-19 23:44, Marco Sulla wrote:
> On Fri, 19 Nov 2021 at 20:38, MRAB <pyt...@mrabarnett.plus.com> wrote:
>>
>> On 2021-11-19 17:48, Marco Sulla wrote:
>> > I have a battery of tests done with pytest. My tests break with a
>> > segfault if I run them normally. If I run them using pytest -v, the
>> > segfault does not happen.
>> >
>> > What could cause this quantical phenomenon?
>> >
>> Are you testing an extension that you're compiling? That kind of problem
>> can occur if there's an uninitialised variable or incorrect reference
>> counting (Py_INCREF/Py_DECREF).
>
> Ok, I know. But why can't it be reproduced if I do pytest -v? This way
> I don't know which test fails.
> Furthermore I noticed that if I remove the __pycache__ dir of tests,
> pytest does not crash, until I re-ran it with the __pycache__ dir
> present.
> This way is very hard for me to understand what caused the segfault.
> I'm starting to think pytest is not good for testing C extensions.
>
If there are too few Py_INCREF or too many Py_DECREF, it'll free the
object too soon, and whether or when that will cause a segfault will
depend on whatever other code is running. That's the nature of the
beast: it's unpredictable!
You could try running each of the tests in a loop to see which one
causes a segfault. (Trying several in a loop will let you narrow it down
more quickly.)
pytest et al. are good for testing behaviour, but not for narrowing down
segfaults.
--
https://mail.python.org/mailman/listinfo/python-list
