Re: Windows Remote Driver Identification Re: Cygwin wishlist for NFSv4.2 driver?
On Apr 3 14:28, Jeremy Drake via Cygwin wrote: > On Wed, 2 Apr 2025, Cedric Blancher via Cygwin wrote: > > > On Tue, 1 Apr 2025 at 13:54, Corinna Vinschen via Cygwin > > wrote: > > > No, but if we want to *better* support this driver, we need either a > > > patch (preferred) or at least info how to distinguish in > > > fs_info::update(*) between the MSFT driver and the NFSv4.2 driver. > > > > That is actually easy: Roland implemented support for > > FileRemoteProtocolInformation in > > https://github.com/kofemann/ms-nfs41-client/commit/e9f72b61494bebd9e26fefec2659a4511a05b0fd > > , and the FILE_REMOTE_PROTOCOL_INFORMATION->Protocol field can be > > used to identify the type of driver. > > > > Examples: > > Windows NFSv3 driver uses WNNC_NET_MS_NFS, > > OpenAFS uses WNNC_NET_OPENAFS, ms-nfs41-client uses > > WNNC_NET_RDR2SAMPLE, and OpenText NFS also uses WNNC_NET_RDR2SAMPLE. > > > > Plan for ms-nfs42-client is to obtain an own WNNC_NET_MSNFS42CLIENT > > tag, and also tags for DOKANY. > > I thought using WNNC to identify was brought up before and rejected. At > this point, it's querying *volume* information, and has a volume handle. > It looks like you get FileRemoteProtocolInformation by querying a *file* > handle. I could be wrong, and maybe that handle can be queried for > FileRemoteProtocolInformation, but it's not like that returns a unique > identifier either. Actually, while the Win32 API requires a volume handle, the underlying ntdll functions do not. They only require a handle to a file or directory opened on the volume which is requested for volume information. Therefore, if possible, fs_info::update() just reuses the file handle from the current path_conv check in progress. Either way, WNNC is not what we're looking for, Jeremy is entirely on the right path here: > What I'm seeing of the things that come through volume information: > FileSystemName is either NFS or DEBUG-NFS41 (the latter at least looks > unique, if not intended for public consumption). fsname NFS isn't sufficient obviously. > The VolumeLabel is > PnfsVolume, the VolumeSerialNumber is always 0xBABAFACE (that's got to > suck for the "unique per-drive/share hash" mentioned in mount.cc), the > flags are set in > https://github.com/kofemann/ms-nfs41-client/blob/bf8d343ba2465926f3bfafa8407a69e79649cf46/daemon/nfs41_superblock.c#L173 This is a bug in the nfs41 driver. It should make an effort to generate a unique serial number and/or a unique volume label. Why not generate a hash value from the remote path? If it can't create a unique VSN, the VSN should be set to 0. Otherwise, having the same non-0 VSN for all mounted NFS shares will lead to confusion. > What stands out to me in the flags is that they include > FILE_SUPPORTS_REMOTE_STORAGE. I'm not sure what that's supposed to > mean, above and beyond FILE_REMOTE_DEVICE. https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/d4bc551b-7aaf-4b4f-ba0e-3a75e7c528f0#Appendix_A_167 Setting this flag is wrong afaics. Can somebody with nfs41 installed just run $ /usr/lib/csih/getVolInfo and paste the output here? > MS NFS only seems to set > FILE_CASE_PRESERVED_NAMES (2), so the existing flag song-and-dance could > probably be extended to tell the difference here. > > They're also always setting > SSINFO_FLAGS_ALIGNED_DEVICE|SSINFO_FLAGS_PARTITION_ALIGNED_ON_DEVIVE| > SSINFO_FLAGS_NO_SEEK_PENALTY. Could be helpful to distinguish between both NFS versions, but I don't trust Windows to set the flags if the server is a Windows machine. Or something. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: unix socket hang when connect
On Apr 7 12:35, Christian Franke via Cygwin wrote: > Yuyi Wang via Cygwin wrote: > > Thank you Christian, setting SO_PEERCRED on the client socket works. It's a > > little wierd, though. Is it OK to always set it to NULL on a unix socket? > > No, this is a Cygwin extension, originally added for postfix. Background is the default mechanism to exchange credentials and a security hash, so we know we talk to the right peer. Setting SO_PEERCRED switches that off. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin SSHD session unable to access cloud/OneDrive files
On Apr 4 16:23, Shaddy Baddah via Cygwin wrote: > > On 4/04/2025 10:02 am, Shaddy Baddah via Cygwin wrote: > > Hi, > > > > On 4/04/2025 4:49 am, ASSI via Cygwin wrote: > > > Shaddy Baddah via Cygwin writes: > > > > If I connect an SSH session via the "native" OpenSSH instance > > > > integrated into Windows, I can do something like the following to a, > > > > at the time, online only, not yet downloaded file, and OneDrive will > > > > download it ahead of outputing it: > > > […] > > > > But if I connect an SSH session via the Cygwin instance running on a > > > > different port to 22, it does not trigger the provider to download the > > > > file, and I see this error: > > > […] > > > > The fact that the native SSH session is OK suggests to me that there > > > > is some newer type of security token that that service obtains that > > > > the Cygwin SSH service does not. But I've not looked into it too hard. > > > No, that suggests that you have logged into your SSh session without > > > providing a password. Windows will revoke all network access that > > > requires authentication for such sessions. > > > > Thanks for the reminder, but it's not that. I definitely logged in. And > > amazingly, it's more than 20 years ago I was helped on this list to > > understand this nuance, I think with regards to running Oracle's sqlplus > > command line. Speculating... it appears registry keys for the sync'ed > > portions of the filesystem are usually subkeys under > > HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Explorer/SyncRootManager/ > > That suggests tie-in with Explorer... which suggests the Shell > > extensions are not active somehow? I speak with complete lack of > > authority of course. One thing I could maybe do though, is profile the > > forked process layout in Process Explorer, maybe look at which DLLs are > > in scope for the relevant shell processes. I don't know how much that > > would help. -- Regards, Shaddy > > > Apologies for the formatting error. > > Thanks for the reminder, but it's not that. I defintiely logged > in. And amazingly, it's more than 20 years ago I was helped on this > list to understand this nuance, I think with regards to running > Oracle's sqlplus command line. > > Speculating... it appears registry keys for the sync'ed portions of > the filesystem are usually subkeys under > HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Explorer/SyncRootManager/ > That suggests tie-in with Explorer... which suggests the Shell > extensions are not active somehow? I speak with complete lack of > authority of course. I checked the Windows OpenSSH code and I don't see any relevant difference between the way MSFT OpenSSH uses LsaLogonUser and the way Cygwin does it. If it's using something with Shell Extensions, we're probably out. What you still can do is using password auth the good old Interix way: https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-nopasswd3 Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: ftruncate() may fail with EISDIR if other process calls open()
Corinna Vinschen via Cygwin wrote:
On Apr 8 13:50, Corinna Vinschen via Cygwin wrote:
On Apr 7 09:59, Christian Franke via Cygwin wrote:
Found because 'stress-ng --fcntl 2 --verify' reports "ftruncate failed,
errno=21":
Thanks for the testcase. I know what happens. Stay tuned for a patch.
Nice problem. That's what happens if you try to do what an OS is doing,
without actually being an OS, just a user space lib. Theoretically we
should never cache any file information but always call the matching OS
functions. Only... this adds quite a few context switches every time
accessing files. Sigh.
I just pushed 2d81f6ebe3dc ("Cygwin: open: always fix up cached DOS file
attributes after NtCreateFile").
Try the upcoming cygwin-3.7.0-0.47.g9e6193576f34.
With a local build of 3.7.0-dev-49-g42df80dad079.x86_64, both the
testcase and stress-ng no longer fail.
--
Regards,
Christian
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: ftruncate() may fail with EISDIR if other process calls open()
On Apr 8 13:50, Corinna Vinschen via Cygwin wrote:
> On Apr 7 09:59, Christian Franke via Cygwin wrote:
> > Found because 'stress-ng --fcntl 2 --verify' reports "ftruncate failed,
> > errno=21":
>
> Thanks for the testcase. I know what happens. Stay tuned for a patch.
Nice problem. That's what happens if you try to do what an OS is doing,
without actually being an OS, just a user space lib. Theoretically we
should never cache any file information but always call the matching OS
functions. Only... this adds quite a few context switches every time
accessing files. Sigh.
I just pushed 2d81f6ebe3dc ("Cygwin: open: always fix up cached DOS file
attributes after NtCreateFile").
Try the upcoming cygwin-3.7.0-0.47.g9e6193576f34.
Thanks,
Corinna
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: ftruncate() may fail with EISDIR if other process calls open()
On Apr 7 09:59, Christian Franke via Cygwin wrote: > Found because 'stress-ng --fcntl 2 --verify' reports "ftruncate failed, > errno=21": Thanks for the testcase. I know what happens. Stay tuned for a patch. Thanks, Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)
Greetings! Are there tuning variables to improve ls, ls -l, find ., find . -ls performance for very large dirs? If we have a SMB dir with 6+ entries a simple ls -l can take MANY minutes (22+mins), while cmd.exe dir just floods the terminal with results immediately. Mark -- IT Infrastructure Consultant Windows, Linux -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)
On 2025-04-08 12:05, Mark Liam Brown via Cygwin wrote: Greetings! Are there tuning variables to improve ls, ls -l, find ., find . -ls performance for very large dirs? If we have a SMB dir with 6+ entries a simple ls -l can take MANY minutes (22+mins), while cmd.exe dir just floods the terminal with results immediately. AFAIK Cygwin stat(3) is slow because of the Windows calls it has to make and alternate suffixes also have to be tried multiplying the time. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retrancher but when there is no more to cut -- Antoine de Saint-Exupéry -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: ftruncate() may fail with EISDIR if other process calls open()
On Apr 8 17:07, Christian Franke via Cygwin wrote:
> Corinna Vinschen via Cygwin wrote:
> > On Apr 8 13:50, Corinna Vinschen via Cygwin wrote:
> > > On Apr 7 09:59, Christian Franke via Cygwin wrote:
> > > > Found because 'stress-ng --fcntl 2 --verify' reports "ftruncate failed,
> > > > errno=21":
> > > Thanks for the testcase. I know what happens. Stay tuned for a patch.
> > Nice problem. That's what happens if you try to do what an OS is doing,
> > without actually being an OS, just a user space lib. Theoretically we
> > should never cache any file information but always call the matching OS
> > functions. Only... this adds quite a few context switches every time
> > accessing files. Sigh.
> >
> > I just pushed 2d81f6ebe3dc ("Cygwin: open: always fix up cached DOS file
> > attributes after NtCreateFile").
> >
> > Try the upcoming cygwin-3.7.0-0.47.g9e6193576f34.
>
> With a local build of 3.7.0-dev-49-g42df80dad079.x86_64, both the testcase
> and stress-ng no longer fail.
Thanks for checking!
Corinna
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Mounted shares not visible in bash "Run as Administrator"
On Tue, 8 Apr 2025 at 08:37, Thomas Wolff via Cygwin wrote: > > > > Am 08.04.2025 um 03:04 schrieb Jim Garrison via Cygwin: > > I just ran across this today. In a normal session all my remote > > mounts (NTFS, file server running Win10) are visible. In a session > > that is "Run as administrator", none of the mounts are visible. > > > > Is there a configuration parameter that would make mounts visible in > > the administrator session? My Google-fu must be off today because I > > can't seem to find any references to this "issue". > > > It's a Windows paradigm. Admin mounts are distinct from user mounts (and > those from each other). > Just mount again as admin. I think if you are the same user but different Logon ID, then you are not seeing mounts from the other Logon ID. The only mounts visible to all users must be mounted by user SYSTEM. You need PsExec from https://learn.microsoft.com/en-us/sysinternals/downloads/psexec to run mintty.exe as user SYSTEM. Then mount the SMB filesystem, and all users can use it. Dan -- Dan Shelton - Cluster Specialist Win/Lin/Bsd -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)
On 08/04/2025 19:05, Mark Liam Brown via Cygwin wrote: Greetings! Are there tuning variables to improve ls, ls -l, find ., find . -ls performance for very large dirs? If we have a SMB dir with 6+ entries a simple ls -l can take MANY minutes (22+mins), while cmd.exe dir just floods the terminal with results immediately. Mark By default, to simulate POSIX using local & remote MS-DOS/Windows-type mounts, Cygwin's readdir(), stat() and other filesystem functions have to jump through a lot of hoops. For a start, Windows 'Unicode' characters have to be converted to the locale, usually UTF-8, being used by Cygwin. Then either the Windows (NTFS or SMB) ACLs have to be mapped to POSIX file permissions & ACLs and/or the files' extensions have to be checked and sometimes the file opened to read the first few bytes to determine executable permissions. Windows local & remote SIDs have to be converted to POSIX UIDs & GIDs. And Cygwin then has to convert to convert the UID/GID passed back from the clients like 'ls' to Windows user/group names for display. As discussed by other replies, sixty thousand entries in a single directory is ridiculously poor practice and the best thing would be to redesign whatever is creating these directories & their contents. But if this is not possible, there are mount options that can be used in /etc/fstab to reduce the amount of overhead. Try using 'noacl' & 'notexec' in /etc/fstab and then access files from Cygwin using the mount point rather than the UNC path. This disables the advanced ACL/SID mapping and reports all files as -rw-r--r--, directories drwxr-x-rx with UID of the calling user and GID None. (Of course Windows & SMB will still restrict access based on the actual permissions available to the caller's access tokens.) If sort order isn't important, use '--sort=none' with ls. //h/s /mnt/h/s smbfs noacl,notexec,nouser 0 0 I doubt this is going to get a 60k entry remote mount directory listing down to a reasonable time but it should definitely help. See https://www.cygwin.com/cygwin-ug-net/using.html#mount-table. -- Sam Edge OpenPGP_0x8AC2CEBF54528E30.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)
On 08/04/2025 19:05, Mark Liam Brown via Cygwin wrote: Are there tuning variables to improve ls, ls -l, find ., find . -ls performance for very large dirs? If we have a SMB dir with 6+ entries a simple ls -l can take MANY minutes (22+mins), while cmd.exe dir just floods the terminal with results immediately. I just remembered noticing my /var/log/ entries jumped looking at my last Cygwin upgrade log, so before I clean it up into subdirs, let's try some stuff! If you just want a directory name dump, use \ls -U|--sort=none or \ls -f which is like -aU; command prefix \ overrides the usual aliases like -CF --color=auto: $ time \ls -1U /var/log/ | \wc -lwmcL 54049 54049 1885787 1885787 38 real0m0.336s user0m0.046s sys 0m0.295s $ time \ls -1f /var/log/ | \wc -lwmcL 54051 54051 1885792 1885792 38 real0m0.347s user0m0.140s sys 0m0.232s even with -l, it's only a few seconds: $ time \ls -lU /var/log/ | \wc -lwmcL 54050 486455 4966716 4966716 102 real0m15.891s user0m1.421s sys 0m14.560s $ time \ls -lf /var/log/ | \wc -lwmcL 54052 486473 4966835 4966835 102 real0m15.858s user0m1.405s sys 0m14.374s and skipping user and group lookup does not save anything: $ time \ls -gloU /var/log/ | \wc -lwmcL 54050 378357 3777638 3777638 80 real0m15.699s user0m1.562s sys 0m14.155s $ time \ls -gGlf /var/log/ | \wc -lwmcL 54052 378371 313 313 80 real0m15.908s user0m1.389s sys 0m14.498s -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retrancher but when there is no more to cut -- Antoine de Saint-Exupéry -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Deadlock when calling pthread_key_create in the destructor of a pthread_key
On Sun, 23 Mar 2025 20:32:44 +0800 Yuyi Wang wrote: > It's a bug when I tried to run tests of Rust std lib. The standard > library of Rust tries to create a new pthread_key in the destructor of a > key created previously. Unfortunately, List::for_each locked the mutex > before, so List_insert method cannot lock the mutex again. > > I have searched though the POSIX docs and didn't find any words that one > should not call pthread_key_create in the destructor of a pthread_key. I > think it should be a bug of cygwin. Fixed. Please try cygwin-3.7.0-0.49.g42df80dad079. -- Takashi Yano -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Mounted shares not visible in bash "Run as Administrator"
On 4/7/2025 23:35, Thomas Wolff via Cygwin wrote: Am 08.04.2025 um 03:04 schrieb Jim Garrison via Cygwin: I just ran across this today. In a normal session all my remote mounts (NTFS, file server running Win10) are visible. In a session that is "Run as administrator", none of the mounts are visible. Is there a configuration parameter that would make mounts visible in the administrator session? My Google-fu must be off today because I can't seem to find any references to this "issue". It's a Windows paradigm. Admin mounts are distinct from user mounts (and those from each other). Just mount again as admin. I suspected this was "normal" based on the lack of Google hits... thanks for the confirmation. -- Jim Garrison j...@acm.org -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)
On Tue, Apr 08, 2025 at 08:05:21PM +0200, Mark Liam Brown via Cygwin wrote: > Greetings! > > Are there tuning variables to improve ls, ls -l, find ., find . -ls > performance for very large dirs? > > If we have a SMB dir with 6+ entries a simple ls -l can take MANY > minutes (22+mins), while cmd.exe dir just floods the terminal with > results immediately. `man ls` This might help a little bit: `ls -1f` # one (-1) for single line output, and -f to disable sorting Of course, please also ask why there are 6 files in one directory rather than sharded or stored some other way. e.g. for log files, create an archive/ subfolder and rotate all non-recently, less-frequently accessed files to that subfolder so that the main folder has (many) fewer entries. For a folder that size, `ls` and `ls -l` are less appropriate tools. A web server could serve a single index.html with the directory listing, and the directory listing could be updated when the folder changes, or on a periodic basis. Various solutions depend on context and usage of the folder and its contents. Whether or not cmd.exe handles such a large directory better is immaterial to the assessment that 6+ entries in a single folder is lacking better folder organization/file management and access methods Cheers, Glenn -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
