[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Changes by Serhiy Storchaka : -- resolution: -> fixed stage: commit review -> committed/rejected status: open -> closed ___ Python tracker ___ __

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Roundup Robot added the comment: New changeset 874813a3523d by Charles-François Natali in branch '2.7': Issue #19965: Make sure that Python-ast.h is properly taken into account in the http://hg.python.org/cpython/rev/874813a3523d New changeset cfe0a293551f by Charles-François Natali in branch '3

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Serhiy Storchaka added the comment: LGTM. -- assignee: -> neologix stage: -> commit review ___ Python tracker ___ ___ Python-bugs-li

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Charles-François Natali added the comment: Here's a patch. -- Added file: http://bugs.python.org/file33113/makefile_ast_h.diff ___ Python tracker ___

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Charles-François Natali added the comment: Actually no, I don't understand this patch: if the makefile was correct, C files depending on Include/Python-ast.h and Python/Python-ast.c shouldn't be built before those files have been generated by asdl_c.py. The problem doesn't lie in the non-atomicit

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Charles-François Natali added the comment: > > > > Since the patch doesn't use O_EXCL for the temporary file, it suffers > > from the exact same race condition as the current code. > > This does not make race condition. Only one thread writes .tmp files. > > The problem is that other threads can

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Serhiy Storchaka added the comment: > Since the patch doesn't use O_EXCL for the temporary file, it suffers > from the exact same race condition as the current code. This does not make race condition. Only one thread writes .tmp files. The problem is that other threads can read *-ast.[ch] files

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Charles-François Natali added the comment: Since the patch doesn't use O_EXCL for the temporary file, it suffers from the exact same race condition as the current code. It should be using the "x" open mode. -- nosy: +neologix ___ Python tracker

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Serhiy Storchaka added the comment: We can save them in a list of created files. -- ___ Python tracker ___ ___ Python-bugs-list mailin

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Arfrever Frehtes Taifersar Arahesis added the comment: 'p' variable in one part of code has different value than in other part. -- ___ Python tracker ___

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Serhiy Storchaka added the comment: I think os.replace() should be used instead os.rename(). Or pair os.unlink()/os.rename(). And perhaps for decreasing chance of race between creating .c and .h files, both renames should be done after writing both files. -- _

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

Changes by Serhiy Storchaka : -- nosy: +benjamin.peterson, brett.cannon, georg.brandl, ncoghlan, serhiy.storchaka ___ Python tracker ___ _

[issue19965] Non-atomic generation of Include/Python-ast.h and Python/Python-ast.c

New submission from Arfrever Frehtes Taifersar Arahesis: A user reported this error with `make -j${high_value}`: x86_64-pc-linux-gnu-gcc -pthread -c -Wno-unused-result -DNDEBUG -O2 -march=atom -pipe -fwrapv-I. -IInclude -I./Include -fPIC -DPy_BUILD_CORE -o Pytho