RE: AF_UNIX/SOCK_DGRAM is dropping messages
Hi Ken > Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems > to > drop messages or at least they are not received in the same > order they are sent > > [snip] > > > Thanks for the test case. I can confirm the problem. I'm not > > familiar enough with the current AF_UNIX implementation to debug > > this easily. I'd rather spend my time on the new implementation > > (on the topic/af_unix branch). It turns out that your test case > > fails there too, but in a completely different way, due to a bug > > in sendto for datagrams. I'll see if I can fix that bug and then try > > again. > > > > Ken > > Ok, too bad it wasn't our own code base but good that the "mystery" > is verified > > I finally succeed to build topic/af_unix (after finding out what > version of zlib was needed), but not with -D__WITH_AF_UNIX to > CXXFLAGS though and thus I haven’t tested it yet > > Is it sufficient to add the define to the "main" Makefile or do you > have to add it to all the Makefile:s ? I guess I can find out > though > >>> > >>> I do it on the configure line, like this: > >>> > >>>../af_unix/configure CXXFLAGS="-g -O0 -D__WITH_AF_UNIX" -- > prefix=... > >>> > Is topic/af_unix fairly up to date with master branch ? > >>> > >>> Yes, I periodically cherry-pick commits from master to topic/af_unix. > >>> I'lldo that again right now. > >>> > Either way, I'll be glad to help out testing topic/af_unix > >>> > >>> Thanks! > >> > >> I've now pushed a fix for that sendto bug, and your test case runs > >> without error on the topic/af_unix branch. > > > > It seems like the test-case do work now with topic/af_unix in blocking > > mode, but when using non-blocking (with MSG_DONTWAIT) there are > some > > issues I think > > > > 1. When the queue is empty with non-blocking recv(), errno is set to > > EPIPE but I think it should be EAGAIN (or maybe the pipe is getting > > broken for real of some reason ?) > > > > 2. When using non-blocking recv() and no message is written at all, it > > seems like recv() blocks forever > > > > 3. Using non-blocking recv() where the "client" does send less than > > "count" messages, sometimes recv() blocks forever (as well) > > > > > > My naïve analysis of this is that for the first issue (if any) the > > wrong errno is set and for the second issue it blocks if no sendto() > > is done after the first recv(), i.e. nothing kicks the "reader thread" > > in the butt to realise the queue is empty. It is not super clear > > though what POSIX says about creating blocking descriptors and then > > using non-blocking-flags with recv(), but this works in Linux any way > > The explanation is actually much simpler. In the recv code where a bound > datagram socket waits for a remote socket to connect to the pipe, I simply > forget to handle MSG_DONTWAIT. I've pushed a fix. Please retest. > > I should add that in all my work so far on the topic/af_unix branch, I've > thought mainly about stream sockets. So there may still be things remaining > to be implemented for the datagram case. I finally got some time to test topic/af_unix in our "real" cygwin-application (casual) and unfortunately very few of our unittests pass The symptoms are that there's unexpected eternal blocking, sometimes there's unexpected EADDRNOTAVAIL, sometimes it looks like some memory corruption (and core-dumps) Of course the memory corruption etc could be our self and the core-dumps might be because of uncaught exceptions Needles to say is that all unittests pass on Linux, but of course cygwin-topic/af_unix could act according to POSIX-standard and the behaviour could be due to our own misinterpretation of how POSIX works I will try to narrow down the quite complex logic and reproduce the problems If you of some reason wanna try it with casual, I'd be glad to help you out (it should be easier now that last time (but there might be some documentation missing for Cygwin still)) https://bitbucket.org/casualcore/ Best regards, Kristian > Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: AF_UNIX/SOCK_DGRAM is dropping messages
On 4/13/2021 10:06 AM, sten.kristian.ivars...@gmail.com wrote: Hi Ken Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems to drop messages or at least they are not received in the same order they are sent [snip] Thanks for the test case. I can confirm the problem. I'm not familiar enough with the current AF_UNIX implementation to debug this easily. I'd rather spend my time on the new implementation (on the topic/af_unix branch). It turns out that your test case fails there too, but in a completely different way, due to a bug in sendto for datagrams. I'll see if I can fix that bug and then try again. Ken Ok, too bad it wasn't our own code base but good that the "mystery" is verified I finally succeed to build topic/af_unix (after finding out what version of zlib was needed), but not with -D__WITH_AF_UNIX to CXXFLAGS though and thus I haven’t tested it yet Is it sufficient to add the define to the "main" Makefile or do you have to add it to all the Makefile:s ? I guess I can find out though I do it on the configure line, like this: ../af_unix/configure CXXFLAGS="-g -O0 -D__WITH_AF_UNIX" -- prefix=... Is topic/af_unix fairly up to date with master branch ? Yes, I periodically cherry-pick commits from master to topic/af_unix. I'lldo that again right now. Either way, I'll be glad to help out testing topic/af_unix Thanks! I've now pushed a fix for that sendto bug, and your test case runs without error on the topic/af_unix branch. It seems like the test-case do work now with topic/af_unix in blocking mode, but when using non-blocking (with MSG_DONTWAIT) there are some issues I think 1. When the queue is empty with non-blocking recv(), errno is set to EPIPE but I think it should be EAGAIN (or maybe the pipe is getting broken for real of some reason ?) 2. When using non-blocking recv() and no message is written at all, it seems like recv() blocks forever 3. Using non-blocking recv() where the "client" does send less than "count" messages, sometimes recv() blocks forever (as well) My naïve analysis of this is that for the first issue (if any) the wrong errno is set and for the second issue it blocks if no sendto() is done after the first recv(), i.e. nothing kicks the "reader thread" in the butt to realise the queue is empty. It is not super clear though what POSIX says about creating blocking descriptors and then using non-blocking-flags with recv(), but this works in Linux any way The explanation is actually much simpler. In the recv code where a bound datagram socket waits for a remote socket to connect to the pipe, I simply forget to handle MSG_DONTWAIT. I've pushed a fix. Please retest. I should add that in all my work so far on the topic/af_unix branch, I've thought mainly about stream sockets. So there may still be things remaining to be implemented for the datagram case. I finally got some time to test topic/af_unix in our "real" cygwin-application (casual) and unfortunately very few of our unittests pass The symptoms are that there's unexpected eternal blocking, sometimes there's unexpected EADDRNOTAVAIL, sometimes it looks like some memory corruption (and core-dumps) Of course the memory corruption etc could be our self and the core-dumps might be because of uncaught exceptions Needles to say is that all unittests pass on Linux, but of course cygwin-topic/af_unix could act according to POSIX-standard and the behaviour could be due to our own misinterpretation of how POSIX works More likely it's due to bugs in the topic/af_unix branch. This is still very much a work in progress. I will try to narrow down the quite complex logic and reproduce the problems That would be ideal. If you of some reason wanna try it with casual, I'd be glad to help you out (it should be easier now that last time (but there might be some documentation missing for Cygwin still)) https://bitbucket.org/casualcore/ I'm going on vacation in a few days, but I might do this when I get back. Thanks for your testing. Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: cgdb fails with cygwin 3.2.0 but is OK with 3.1.6
On 4/13/2021 9:05 AM, Arthur Norman via Cygwin wrote: With the latest cygwin if I go "gcc -O0 -g hello.c -o hello" then "cgdb ./hello", break main, run then cgdb exits abruptly without sign on actually starting my code. If I use just gdb not cgdb all is OK. If I try "gcd -tui hello" I can single step my code but the display in the top half of the window is damaged - eg after a couple of steps it shows lines 5 and 6 of the source code twice each, but once normally and once in inverted video. Raw seems to gdb lets me step the code about as expected. If I revert just the cygwin package from 3.2.0-1 to 3.1.6-1 things appear to behave as I would expect. 3.1.7-1 possibly had -tui issues and led to cgdb stalling on my large and broken example code which is why I report 3.1.6-1 as the baseline. At one time it was recommended to disable a Cygwin feature, something like this: alias cgdb="env CYGWIN=disable_pcon cgdb" Not sure if this is still necesary. -- R.Berber -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: cgdb fails with cygwin 3.2.0 but is OK with 3.1.6
René Berber wrote At one time it was recommended to disable a Cygwin feature, something like this: alias cgdb="env CYGWIN=disable_pcon cgdb" Not sure if this is still necesary. Thank you for the reminder about that. I had forgotten - however ion my main Windows machine I have the CYGWIN variable set like that all the time and in the VM that I tried with it is not - and in both cases cgdb seems to behave well on the slightly old version of the cygwin package but failed with the latest. So I do not think that is the issue here and I might live advice as to whether I can now remove that setting of CYGWIN! Arthur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: cgdb fails with cygwin 3.2.0 but is OK with 3.1.6
On 4/13/2021 1:56 PM, Arthur Norman via Cygwin wrote: René Berber wrote At one time it was recommended to disable a Cygwin feature, something like this: alias cgdb="env CYGWIN=disable_pcon cgdb" Not sure if this is still necesary. Thank you for the reminder about that. I had forgotten - however ion my main Windows machine I have the CYGWIN variable set like that all the time and in the VM that I tried with it is not - and in both cases cgdb seems to behave well on the slightly old version of the cygwin package but failed with the latest. So I do not think that is the issue here and I might live advice as to whether I can now remove that setting of CYGWIN! Probably. And since you don't see any difference then that's proof that is no longer needed (in this case). Sorry I couldn't be of more help, haven't used cgdb in a while. -- R.Berber -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
dealing with a NTLM-proxy
Hi. I'm running my cygwin installation on a Windows 10 laptop without admin rights in a corporate network. To install for example pyhton moduls using pip or stuff with ansible-galaxy, I'm supposed to cross that corporate proxy working only with the proprietary NTLM protocol. cntlm isn't a working solution since when I'm executing the setup.exe, I'm required to enter an admin password. ntlmaps isn't a solution too. Python version is too old. But I've figured out is, curl is able to cope with a NTLM proxy. Is it possible to make pip or ansible-galaxy to make use of a curl command instead of trying it with the enviromment variables http_proxy and co? -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: AF_UNIX/SOCK_DGRAM is dropping messages
On 4/13/2021 10:47 AM, Ken Brown via Cygwin wrote: On 4/13/2021 10:06 AM, sten.kristian.ivars...@gmail.com wrote: Hi Ken Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems to drop messages or at least they are not received in the same order they are sent [snip] Thanks for the test case. I can confirm the problem. I'm not familiar enough with the current AF_UNIX implementation to debug this easily. I'd rather spend my time on the new implementation (on the topic/af_unix branch). It turns out that your test case fails there too, but in a completely different way, due to a bug in sendto for datagrams. I'll see if I can fix that bug and then try again. Ken Ok, too bad it wasn't our own code base but good that the "mystery" is verified I finally succeed to build topic/af_unix (after finding out what version of zlib was needed), but not with -D__WITH_AF_UNIX to CXXFLAGS though and thus I haven’t tested it yet Is it sufficient to add the define to the "main" Makefile or do you have to add it to all the Makefile:s ? I guess I can find out though I do it on the configure line, like this: ../af_unix/configure CXXFLAGS="-g -O0 -D__WITH_AF_UNIX" -- prefix=... Is topic/af_unix fairly up to date with master branch ? Yes, I periodically cherry-pick commits from master to topic/af_unix. I'lldo that again right now. Either way, I'll be glad to help out testing topic/af_unix Thanks! I've now pushed a fix for that sendto bug, and your test case runs without error on the topic/af_unix branch. It seems like the test-case do work now with topic/af_unix in blocking mode, but when using non-blocking (with MSG_DONTWAIT) there are some issues I think 1. When the queue is empty with non-blocking recv(), errno is set to EPIPE but I think it should be EAGAIN (or maybe the pipe is getting broken for real of some reason ?) 2. When using non-blocking recv() and no message is written at all, it seems like recv() blocks forever 3. Using non-blocking recv() where the "client" does send less than "count" messages, sometimes recv() blocks forever (as well) My naïve analysis of this is that for the first issue (if any) the wrong errno is set and for the second issue it blocks if no sendto() is done after the first recv(), i.e. nothing kicks the "reader thread" in the butt to realise the queue is empty. It is not super clear though what POSIX says about creating blocking descriptors and then using non-blocking-flags with recv(), but this works in Linux any way The explanation is actually much simpler. In the recv code where a bound datagram socket waits for a remote socket to connect to the pipe, I simply forget to handle MSG_DONTWAIT. I've pushed a fix. Please retest. I should add that in all my work so far on the topic/af_unix branch, I've thought mainly about stream sockets. So there may still be things remaining to be implemented for the datagram case. I finally got some time to test topic/af_unix in our "real" cygwin-application (casual) and unfortunately very few of our unittests pass The symptoms are that there's unexpected eternal blocking, sometimes there's unexpected EADDRNOTAVAIL, sometimes it looks like some memory corruption (and core-dumps) Of course the memory corruption etc could be our self and the core-dumps might be because of uncaught exceptions Needles to say is that all unittests pass on Linux, but of course cygwin-topic/af_unix could act according to POSIX-standard and the behaviour couldbe due to our own misinterpretation of how POSIX works More likely it's due to bugs in the topic/af_unix branch. This is still very much a work in progress. I will try to narrow down the quite complex logic and reproduce the problems That would be ideal. If you of some reason wanna try it with casual, I'd be glad to help you out (it should be easier now that last time (but there might be some documentation missing for Cygwin still)) https://bitbucket.org/casualcore/ I'm going on vacation in a few days, but I might do this when I get back. Thanks for your testing. By the way, if your code is using datagram sockets, then there are very serious problems with our implementation (even aside from the performance issue that we've already discussed). For example, I don't know of any reasonable way for select to test whether such a socket is ready for writing. We'll need to solve that somehow. Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: cgdb fails with cygwin 3.2.0 but is OK with 3.1.6
On Tue, 13 Apr 2021 15:05:56 +0100 (GMT Summer Time) Arthur Norman wrote: > With the latest cygwin if I go "gcc -O0 -g hello.c -o hello" then "cgdb > ./hello", break main, run then cgdb exits abruptly without sign on > actually starting my code. If I use just gdb not cgdb all is OK. If I try Thanks for the report. This bug was already fixed in cygwin git head by chance. > "gcd -tui hello" I can single step my code but the display in the top half > of the window is damaged - eg after a couple of steps it shows lines 5 and > 6 of the source code twice each, but once normally and once in inverted > video. Do you mean "gdb" by "gcd"? If so, this seems to be a bug of gdb itself. Same happens in Linux environment. -- Takashi Yano -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: cgdb fails with cygwin 3.2.0 but is OK with 3.1.6
On Tue, 13 Apr 2021 14:13:57 -0500 René Berber wrote: > On 4/13/2021 1:56 PM, Arthur Norman via Cygwin wrote: > > René Berber wrote > >> At one time it was recommended to disable a Cygwin feature, something > >> like this: > >> alias cgdb="env CYGWIN=disable_pcon cgdb" > >> Not sure if this is still necesary. > > > > Thank you for the reminder about that. I had forgotten - however ion my > > main Windows machine I have the CYGWIN variable set like that all the > > time and in the VM that I tried with it is not - and in both cases cgdb > > seems to behave well on the slightly old version of the cygwin package > > but failed with the latest. So I do not think that is the issue here and > > I might live advice as to whether I can now remove that setting of CYGWIN! > > Probably. And since you don't see any difference then that's proof that > is no longer needed (in this case). In cygwin 3.2.0, pseudo console is automatically disabled in cgdb regardless of setting CYGWIN=disable_pcon. Therefore, it is not necessary any more. Thanks. -- Takashi Yano -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
