[issue26032] Use scandir() to speed up pathlib globbing

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue26032] Use scandir() to speed up pathlib globbing

Change by Barney Gale : -- nosy: +barneygale nosy_count: 8.0 -> 9.0 pull_requests: +24730 pull_request: https://github.com/python/cpython/pull/25701 ___ Python tracker ___

[issue26032] Use scandir() to speed up pathlib globbing

Changes by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ _

[issue26032] Use scandir() to speed up pathlib globbing

Roundup Robot added the comment: New changeset 927665c4aaab by Serhiy Storchaka in branch 'default': Issue #26032: Optimized globbing in pathlib by using os.scandir(); it is now https://hg.python.org/cpython/rev/927665c4aaab -- nosy: +python-dev ___ P

[issue26032] Use scandir() to speed up pathlib globbing

Ben Hoyt added the comment: Guido, I've made some tweaks and improvements to the DirEntry docs here: http://bugs.python.org/issue26248 -- the idea is to fix the issues you mentioned to clarify when system calls are required with symlinks, mentioning that the results are cached separately for f

[issue26032] Use scandir() to speed up pathlib globbing

Serhiy Storchaka added the comment: Proposed minimal patch implements globbing in pathlib using os.scandir(). Here are results of microbenchmarks: $ ./python -m timeit -s "from pathlib import Path; p = Path()" -- "list(p.glob('**/*'))" Unpatched: 598 msec per loop Patched: 372 msec per loop

[issue26032] Use scandir() to speed up pathlib globbing

Changes by Serhiy Storchaka : -- components: +Library (Lib) dependencies: +File descriptor leaks in os.scandir(), regular files handled as directories in the glob module versions: -Python 3.5 ___ Python tracker _

[issue26032] Use scandir() to speed up pathlib globbing

Guido van Rossum added the comment: Ben, I think it's worth calling out what the rules are around symlinks. I'm guessing the info that is initially present is a subset of lstat(), so if that indicates it's a symlink, is_dir() and is_file() will need a stat() call, *unless* follow_symlinks is Fals

[issue26032] Use scandir() to speed up pathlib globbing

Ben Hoyt added the comment: Guido, it's true that in almost all cases you get the speedup (no system call), and it's very much worth using. But the idea with the docs being non-committal is because being specific would make the docs fairly complex. I believe it's as follows for is_file/is_dir/

[issue26032] Use scandir() to speed up pathlib globbing

Guido van Rossum added the comment: The DirEntry docs say for most methods "In most cases, no system call is required" which is pretty non-committal. :-( The only firm promise is for inode(), which is pretty useless. -- ___ Python tracker

[issue26032] Use scandir() to speed up pathlib globbing

STINNER Victor added the comment: > As I recall, if the platform's DirEntry doesn't provide the cacheable > attributes when first called, those attributes will be looked up (and cached) > on first access. scandir() is not magic. It simply provides info given by the OS: see readdir() on UNIX a

[issue26032] Use scandir() to speed up pathlib globbing

Ethan Furman added the comment: As I recall, if the platform's DirEntry doesn't provide the cacheable attributes when first called, those attributes will be looked up (and cached) on first access. -- nosy: +ethan.furman ___ Python tracker

[issue26032] Use scandir() to speed up pathlib globbing

STINNER Victor added the comment: Related issue: issue #25596 "regular files handled as directories in the glob module". -- nosy: +haypo ___ Python tracker ___ _

[issue26032] Use scandir() to speed up pathlib globbing

Changes by STINNER Victor : -- nosy: +benhoyt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue26032] Use scandir() to speed up pathlib globbing

Changes by STINNER Victor : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue26032] Use scandir() to speed up pathlib globbing

New submission from Guido van Rossum: The globbing functionality in pathlib (Path.glob() and Path.rglob()) might benefit from using the new optimized os.scandir() interface. It currently just uses os.listdir(). The Path.iterdir() method might also benefit (though less so). There's also a side

[issue26032] Use scandir() to speed up pathlib globbing

Changes by Guido van Rossum : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt