Hi all, We are running Python embedded in our C++ product and are now experiencing crashes (access violation reading 0xffffffffff on Windows) in the Python garbage collector.
We got this on Python 3.6.4 originally, but I can reproduce it with both Python 3.6.8 and Python 3.7.4. The chances of producing a minimal example that reproduces it reliably are currently small I would say. All attempts to simplify the set up seem to cause the problem to go away. Indeed I can only reproduce it by sending a fairly large amount of data 2 or 3 times to our server - sending either half of the data does not reproduce it. I note that there have been a lot of crashes in this method reported over the years, and that efforts are underway in Python 3.8 to make it easier to find issues like this, i.e. https://bugs.python.org/issue36389. I won't be able to use Python 3.8 yet but I could potentially use it as a debugging tool I guess, if there was documentation available how to use these new features. I tried setting PYTHONMALLOC=debug but didn't get any more information that way. The python code is calling "json.loads" on a fairly large json unicode string with plenty of Swedish characters in it. It works at least once, and then crashes on the second or third send of the same data. I paste the stacktrace from Python 3.7.4 below. Please let me know how I can debug this further. I'm using Visual Studio 2017 on Windows 10 if that helps. Regards, Geoff Bache > [Inline Frame] python37.dll!visit_decref(_object *) Line 271 C python37.dll!dict_traverse(_object * op, int(*)(_object *, void *) visit, void * arg) Line 2982 C python37.dll!subtract_refs(_gc_head * containers) Line 296 C python37.dll!collect(int generation, __int64 * n_collected, __int64 * n_uncollectable, int nofail) Line 861 C python37.dll!collect_with_callback(int generation) Line 1029 C python37.dll!collect_generations() Line 1051 C [Inline Frame] python37.dll!_PyObject_GC_Alloc(int) Line 1708 C [Inline Frame] python37.dll!_PyObject_GC_Malloc(unsigned __int64 basicsize) Line 1718 C python37.dll!_PyObject_GC_New(_typeobject * tp) Line 1730 C python37.dll!new_dict(_dictkeysobject * keys, _object * * values) Line 584 C python37.dll!PyDict_New() Line 678 C python37.dll!_parse_object_unicode(_PyScannerObject * s, _object * pystr, __int64 idx, __int64 * next_idx_ptr) Line 730 C python37.dll!scan_once_unicode(_PyScannerObject * s, _object * pystr, __int64 idx, __int64 * next_idx_ptr) Line 1162 C python37.dll!_parse_array_unicode(_PyScannerObject * s, _object * pystr, __int64 idx, __int64 * next_idx_ptr) Line 869 C python37.dll!scan_once_unicode(_PyScannerObject * s, _object * pystr, __int64 idx, __int64 * next_idx_ptr) Line 1101 C python37.dll!scanner_call(_object * self, _object * args, _object * kwds) Line 1188 C python37.dll!_PyObject_FastCallKeywords(_object * callable, _object * const * stack, __int64 nargs, _object * kwnames) Line 199 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4619 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3095 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 3930 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 433 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4621 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3140 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 3930 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 433 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4621 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3111 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 3930 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 433 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4621 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3095 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C [Inline Frame] python37.dll!function_code_fastcall(PyCodeObject * args, _object * const *) Line 283 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 408 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4616 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3125 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C [Inline Frame] python37.dll!function_code_fastcall(PyCodeObject * args, _object * const *) Line 283 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 408 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4616 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3125 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C [Inline Frame] python37.dll!function_code_fastcall(PyCodeObject * args, _object * const *) Line 283 C [Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * const *) Line 408 C python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4616 C python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3125 C [Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 547 C [Inline Frame] python37.dll!function_code_fastcall(PyCodeObject * args, _object * const *) Line 283 C python37.dll!_PyFunction_FastCallDict(_object * func, _object * const * args, __int64 nargs, _object * kwargs) Line 322 C python37.dll!PyObject_Call(_object * callable, _object * args, _object * kwargs) Line 226 C estx.exe!PyObject_CallObject(_object * obj, _object * args) Line 191 C++ .....our server's C++ code in stack frames above this -- https://mail.python.org/mailman/listinfo/python-list
