On 08/04/2025 19:05, Mark Liam Brown via Cygwin wrote:
Greetings!Are there tuning variables to improve ls, ls -l, find ., find . -ls performance for very large dirs? If we have a SMB dir with 60000+ entries a simple ls -l can take MANY minutes (22+mins), while cmd.exe dir just floods the terminal with results immediately. Mark
By default, to simulate POSIX using local & remote MS-DOS/Windows-type mounts, Cygwin's readdir(), stat() and other filesystem functions have to jump through a lot of hoops. For a start, Windows 'Unicode' characters have to be converted to the locale, usually UTF-8, being used by Cygwin. Then either the Windows (NTFS or SMB) ACLs have to be mapped to POSIX file permissions & ACLs and/or the files' extensions have to be checked and sometimes the file opened to read the first few bytes to determine executable permissions. Windows local & remote SIDs have to be converted to POSIX UIDs & GIDs. And Cygwin then has to convert to convert the UID/GID passed back from the clients like 'ls' to Windows user/group names for display.
As discussed by other replies, sixty thousand entries in a single directory is ridiculously poor practice and the best thing would be to redesign whatever is creating these directories & their contents.
But if this is not possible, there are mount options that can be used in /etc/fstab to reduce the amount of overhead. Try using 'noacl' & 'notexec' in /etc/fstab and then access files from Cygwin using the mount point rather than the UNC path. This disables the advanced ACL/SID mapping and reports all files as -rw-r--r--, directories drwxr-x-rx with UID of the calling user and GID None. (Of course Windows & SMB will still restrict access based on the actual permissions available to the caller's access tokens.) If sort order isn't important, use '--sort=none' with ls.
//h/s /mnt/h/s smbfs noacl,notexec,nouser 0 0I doubt this is going to get a 60k entry remote mount directory listing down to a reasonable time but it should definitely help.
See https://www.cygwin.com/cygwin-ug-net/using.html#mount-table. -- Sam Edge
OpenPGP_0x8AC2CEBF54528E30.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
-- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
