Dragoljub <dragol...@gmail.com> added the comment:
@cgohlke compared the statement df2 = pd.read_csv(csv) on Python 3.7.0a3 and a4 in the Visual Studio profiler. The culprit is the isdigit function called in the parsers extension module. On 3.7.0a3 the function is fast at ~8% of samples. On 3.7.0a4 the function is slow at ~64% samples because it calls the _isdigit_l function, which seems to update and restore the locale in the current thread every time... 3.7.0a3: Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % Module Name + [parsers.cp37-win_amd64.pyd] 705 347 28.52% 14.04% parsers.cp37-win_amd64.pyd isdigit 207 207 8.37% 8.37% ucrtbase.dll - _errno 105 39 4.25% 1.58% ucrtbase.dll toupper 24 24 0.97% 0.97% ucrtbase.dll isspace 21 21 0.85% 0.85% ucrtbase.dll [python37.dll] 1 1 0.04% 0.04% python37.dll 3.7.0a4: Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % Module Name + [parsers.cp37-win_amd64.pyd] 8,613 478 83.04% 4.61% parsers.cp37-win_amd64.pyd + isdigit 6,642 208 64.04% 2.01% ucrtbase.dll + _isdigit_l 6,434 245 62.03% 2.36% ucrtbase.dll + _LocaleUpdate::_LocaleUpdate 5,806 947 55.98% 9.13% ucrtbase.dll + __acrt_getptd 2,121 1,031 20.45% 9.94% ucrtbase.dll FlsGetValue 647 647 6.24% 6.24% KernelBase.dll - RtlSetLastWin32Error 296 235 2.85% 2.27% ntdll.dll _guard_dispatch_icall_nop 101 101 0.97% 0.97% ucrtbase.dll GetLastError 46 46 0.44% 0.44% KernelBase.dll + __acrt_update_multibyte_info 1,475 246 14.22% 2.37% ucrtbase.dll - __crt_state_management::get_current_state_index 1,229 513 11.85% 4.95% ucrtbase.dll + __acrt_update_locale_info 1,263 235 12.18% 2.27% ucrtbase.dll - __crt_state_management::get_current_state_index 1,028 429 9.91% 4.14% ucrtbase.dll _ischartype_l 383 383 3.69% 3.69% ucrtbase.dll ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue35195> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
