[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I don't have Python2.6 installed. I just have ever read a bench article comparing the performance between different version of Python, including 2.5, 2.6 and 3.0. That article shows, for the same script, Python2.6 has longer running time than Python2.

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I'm using Python2.5 in which close_fds is not available in Windows. And I cannot use Python2.6 because I need to redirect the stdout and stderr and there's performance concern. I have questions on the root cause: a. why doesn't fopen() has sha

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: The test code and script which can reproduce the problem has been uploaded. Added file: http://bugs.python.org/file12808/test_file_flush.py ___ Python tracker <http://bugs.python.org/issue4

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I did trace the order of file creation and process launch. It shows the file is indeed created before the process launch. I did another investigation. I added a new process, "cmd.exe /c copy Makefile NewMakefile", to copy the the Makefile created, j

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I'm quite certain that the makefile is generated before the "make" program is launched in separated process. Follow is the original code where the makefile is created (line 14) and a task is put in queue (line 19). It's executed in the main

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I narrowed down the root cause in the GIL of Python. I read the source code of implementing os.fsync and found it's using Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS to enclose the calling of _commit() in order to be thread safe. I tried to add them in

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: I agree multithread programming is kind of difficult and tricky. But I don't think there's a race condition in _MultiThreadBuildPlatform method, because I do have only one producer. And the consumer consumes the product only when it's done (i.e. the

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: Thank you very much for the trying. You might miss the step 4 in my previous message. The step is: C:\test\edk2> edksetup.bat C:\test\edk2> set PYTHONPATH=C:\test\tools\Source\Python C:\test\edk2> python.exe C:\test\tools\Source\Python\build\build.p

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: There would be more chances to see the problem by doing this: C:\test\edk2> python.exe C:\test\tools\Source\Python\build\build.py -n 2 -p IntelFrameworkModulePkg\IntelFrameworkModulePkg.dsc -a IA32 -s ___ Python tracker &l

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: The created files are on local drive. I saw the problem on the laptop (XP-SP2), desktop(XP-SP3) and server (Win2003). But there's no such problem on the Linux and Mac boxes. I tried to use fopen/fwrite in my extension according to your suggestion. The pr

[issue4944] os.fsync() doesn't work as expect in Windows

Javen Wang added the comment: gagenellina, My application is a little bit different from your test code. It won't wait for the exit of new process and there're file writing operations during makefile running. I changed the test code to be real multi-process and tried many file si

[issue4944] os.fsync() doesn't work as expect in Windows

New submission from Javen Wang : I encountered a very strange issue in file flush operation in Windows. Here's the scenario of my application: 1. The main thread of my application will create makefiles sequentially. 2. Once a makefile is generated, launch a separate process ca