[issue10156] Initialization of globals in unicodeobject.c

Changes by Stefan Krah : -- resolution: -> fixed stage: patch review -> committed/rejected ___ Python tracker ___ ___ Python-bugs-lis

[issue10156] Initialization of globals in unicodeobject.c

Changes by Stefan Krah : -- status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Buildbots etc. look all good. Thanks for fixing this. -- ___ Python tracker ___ ___ Python-bugs-list ma

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: Committed. Thank you for review, Stefan. Close this issue if the work is finished. -- ___ Python tracker ___

[issue10156] Initialization of globals in unicodeobject.c

Roundup Robot added the comment: New changeset 7c8ad0d02664 by Serhiy Storchaka in branch '2.7': Issue #10156: In the interpreter's initialization phase, unicode globals http://hg.python.org/cpython/rev/7c8ad0d02664 New changeset f7eda8165e6f by Serhiy Storchaka in branch '3.2': Issue #10156: In

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Nice. I think the latest patches are commit-ready. -- ___ Python tracker ___ ___ Python-bugs-list maili

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: > The 2.7 comments also apply to the 3.2 patch. Otherwise the 3.2 patch > (without the _sre changes :) looks good to me. Patches updated addressing new Stefan's comments. Unicode globals no longer reinitialized in _PyUnicode_Init(). Note that I have added a c

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: The 2.7 comments also apply to the 3.2 patch. Otherwise the 3.2 patch (without the _sre changes :) looks good to me. -- ___ Python tracker ___

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Since Rietveld didn't mail me this time: I left some comments on the 2.7 patch. -- versions: +Python 3.3, Python 3.4 ___ Python tracker ___ ___

[issue10156] Initialization of globals in unicodeobject.c

Nick Coghlan added the comment: Serhiy's general approach here looks good to me (although there seem to be some unrelated changes to the re module in the current 3.2 patch). For PEP 432, I want to try to rearrange things so that _PyUnicode_Init is one of the *first* calls made in Py_BeginIniti

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: There is a set of updated patches. -- Added file: http://bugs.python.org/file28815/unicode_globals-2.7_2.patch Added file: http://bugs.python.org/file28816/unicode_globals-3.2_2.patch Added file: http://bugs.python.org/file28817/unicode_globals-3.3_2.p

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Nick Coghlan wrote: > There should still be a check in tp_new (IIRC) that calls PyType_Ready on > unready types. Indeed there is one in type_new(), but that isn't used here AFAICS. If you apply this patch and start up python, there are many "str: not ready" instan

[issue10156] Initialization of globals in unicodeobject.c

Nick Coghlan added the comment: There should still be a check in tp_new (IIRC) that calls PyType_Ready on unready types. While doing something systematic about this kind of problem is part of the rationale of PEP 432, that won't help earlier versions. -- ___

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Nick, I'm adding you to the nosy list since this issue seems related to PEP 432. Quick summary: Globals are used in unicodeobject.c before they are initialized. Also, Unicode objects are created before PyType_Ready(&PyUnicode_Type) has been called. This happens d

[issue10156] Initialization of globals in unicodeobject.c

Changes by Stefan Krah : -- priority: high -> critical ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue10156] Initialization of globals in unicodeobject.c

Changes by Stefan Krah : -- nosy: +Gregory.Andersen, georg.brandl, kushou, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing

[issue10156] Initialization of globals in unicodeobject.c

Changes by Serhiy Storchaka : -- stage: commit review -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubsc

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: Here are patches for all four Python versions. They fixes possible usage of the followed non-initialized global variables: free_list, numfree, interned, unicode_empty, static_strings, unicode_latin1, bloom_linebreak, unicode_default_encoding. -- Add

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: > unicode-leak.patch doesn't fix #16143 though. unicode_empty and > unicode_latin1 need to be initialized, too. Indeed. I'll upload patches tomorrow. -- ___ Python tracker _

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: unicode-leak.patch doesn't fix #16143 though. unicode_empty and unicode_latin1 need to be initialized, too. Actually we could close this in favor of #16143. -- ___ Python tracker ___

[issue10156] Initialization of globals in unicodeobject.c

Serhiy Storchaka added the comment: Daniel's patch looks good for me. 2.7 looks affected too. -- nosy: +serhiy.storchaka stage: patch review -> commit review versions: +Python 2.7 -Python 3.3, Python 3.4 ___ Python tracker

[issue10156] Initialization of globals in unicodeobject.c

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: See also #16143. -- versions: +Python 3.3, Python 3.4 ___ Python tracker ___ ___ Python-bugs-list maili

[issue10156] Initialization of globals in unicodeobject.c

Changes by Mark Dickinson : -- nosy: -mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: The PEP-393 changes apparently fix this leak; at least I can't reproduce it in default any longer (but still in 3.2). -- ___ Python tracker ___ __

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: Stefan Krah wrote: > Is the module initialization procedure documented somewhere? I get > the impression that unicodeobject.c depends on dict.c and dict.c > depends on unicodeobject.c. s/dict.c/dictobject.c/ -- ___ Py

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: [Merging with issue 11402] Daniel's patch is much simpler, but I think that unicode_empty and unicode_latin1 would need to be protected before _PyUnicode_Init is called. Is the module initialization procedure documented somewhere? I get the impression that unicod

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: > why should _PyUnicode_Init() try to call _PyUnicode_InitGlobals() again? For the embedding scenario (when only Py_Initialize() is called) I wanted to preserve the old behavior of _PyUnicode_Init(). But this is not really enough. I wrote a new patch that also c

[issue10156] Initialization of globals in unicodeobject.c

Amaury Forgeot d'Arc added the comment: About the patch: why should _PyUnicode_Init() try to call _PyUnicode_InitGlobals() again? -- ___ Python tracker ___

[issue10156] Initialization of globals in unicodeobject.c

Stefan Krah added the comment: I've verified the leak manually. The cause is that global variables in unicodeobject.c, e.g. free_list, are used before _PyUnicode_Init() is called. Later on _PyUnicode_Init() sets these variables to NULL, losing the allocated memory. Here is an example of the ear