Hi Alex, >From the log, I found the error is located at client-side. The gRPC client sends reset stream frame with error code 2 indicating internal error, and its channel state somehow becomes SHUTDOWN. Also, I tried your code in Linux, it works fine without any error. I had another failed attempt to build the VS solution with 400+ errors...
Although the root cause still remained unclear, the scope is reduced to the gRPC client behavior on Windows. To dig deeper into the bug, I think one have to trace if the "subprocess" finishes and the life cycle of channel. Lidi Zheng On Mon, Apr 1, 2019 at 7:55 AM Alejandro Villagrán <[email protected]> wrote: > Hi Lidi, > > I've set those environment variables you mentioned and I've attached the > log files (serverlog.txt and clientlog.txt). > Since I reported this issue, I've upgrade the Python version of gRPC, so > the attached log files were created with grpcio 1.19.0 > > Do they give you any hint as to what could be wrong? > > Thanks, > Alex. > > On Thu, 28 Mar 2019 at 17:07, Alejandro Villagrán <[email protected]> > wrote: > >> Hi Lidi, >> >> In the last version of the code I attached, I don't use CherryPy anymore >> so the issue cannot come from there. Eric said that fork handlers are not >> registered on Windows so he ruled out that option too. >> Unfortunately I haven't got access to a Linux machine and I'm not >> familiar enough with Linux to be able to recreate the issue there. >> >> I did use turn on the debug trace a few weeks ago but I couldn't see >> anything obvious. I'll find time to enable them again and share the logs >> with you. >> >> Thanks, >> Alex. >> >> On Thu, 28 Mar 2019 at 16:55, Lidi Zheng <[email protected]> wrote: >> >>> My apologies Alex. There are other stuff keep come up and consumed my >>> time. I'm not a Windows expert, it will take me a long time to setup the >>> compilation environment for gRPC in Windows with debugger. And I failed to >>> find a Windows expert to debug your issue. >>> Eric has mentioned that the breakage can be caused by either fork >>> handlers registration or CherryPy. Do you think you can migrate the >>> reproduce case to Linux, if the root cause is the software? It would be >>> much easier to debug. >>> Also, have you tried to turn on the debug trace in gRPC by setting >>> environmental >>> variables >>> <https://github.com/grpc/grpc/blob/master/doc/environment_variables.md> >>> "*GRPC_VERBOSITY*" to "*DEBUG*", and "*GRPC_TRACE*" to " >>> *api,channel,connectivity_state*"? They might produce useful >>> information for us to identify the problem. >>> >>> Thanks, >>> Lidi Zheng >>> >>> On Thu, Mar 28, 2019 at 3:47 AM Alejandro Villagrán <[email protected]> >>> wrote: >>> >>>> Hi Lidi, >>>> >>>> Did you manage to reproduce the issue? >>>> >>>> Thanks, >>>> Alex. >>>> >>>> On Tue, 19 Mar 2019 at 01:06, Lidi Zheng <[email protected]> wrote: >>>> >>>>> Hi Alex, >>>>> >>>>> Thank you for providing the reproduce code. I will spin up a Windows >>>>> machine to investigate this error. >>>>> If I'm able to find something useful, I'll let you know. >>>>> >>>>> Lidi Zheng >>>>> >>>>> On Mon, Mar 18, 2019 at 4:09 AM Alejandro Villagrán < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Eric/Lidi, >>>>>> >>>>>> Yes, I'm running on Windows. I have now removed the CherryPy code and >>>>>> I still get the exception. >>>>>> >>>>>> Please follow these steps to reproduce the issue: >>>>>> - Unzip ReproduceGRPCIssue.zip >>>>>> - Go to the BreakGRPC folder and compile BreakGRPC.sln. Make sure >>>>>> BreakGRPC.exe is saved in BreakGRPC/x64/Release. >>>>>> - Go to the Services/src folder and open two command prompts there. >>>>>> - Run "python GRPCserver.py" in one command prompt. >>>>>> - Run "python GRPCclient.py" in the other command prompt. >>>>>> >>>>>> You should see the exception on the client command prompt. >>>>>> >>>>>> Please let me know if you are still unable to reproduce the issue >>>>>> with this version of the code. >>>>>> >>>>>> Thanks, >>>>>> Alex. >>>>>> >>>>>> On Mon, 4 Mar 2019 at 17:04, Eric Gribkoff <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> +Lidi Zheng <[email protected]>, who will be available for any >>>>>>> follow-up questions (it will be easier for him to notice your questions >>>>>>> if >>>>>>> you include his email address on the "to:" line) >>>>>>> >>>>>>> Hi Alex, >>>>>>> >>>>>>> Sorry for the delay. I was not able to reproduce the problem; it >>>>>>> looks like you are running on Windows, in which case gRPC's fork >>>>>>> handlers >>>>>>> are not registered/run, so those shouldn't be the cause here . Since the >>>>>>> reproduction example also uses CherryPy websockets, it's quite possible >>>>>>> the >>>>>>> issue stems from that software rather than the gRPC stack - we'd likely >>>>>>> need a reproduction case that only uses gRPC, without the websockets, >>>>>>> to be >>>>>>> able to help debug this further. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Eric >>>>>>> >>>>>>> On Mon, Mar 4, 2019 at 2:36 AM Alex <[email protected]> wrote: >>>>>>> >>>>>>>> Hi Eric, >>>>>>>> >>>>>>>> Just wondering if you had time to run my attached example and >>>>>>>> managed to reproduce the problem? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Alex. >>>>>>>> >>>>>>>> On Wednesday, February 20, 2019 at 7:04:51 PM UTC, Eric Gribkoff >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Can you post the code you're using to reproduce this error? If >>>>>>>>> you're using subprocess.Popen (or otherwise using fork+exec) to start >>>>>>>>> the >>>>>>>>> C++ grpc client process, the C++ client itself cannot be interfering >>>>>>>>> with >>>>>>>>> the Python process. Something could be going wrong in the gRPC core >>>>>>>>> fork >>>>>>>>> handlers, however - you can try running with the environment variable >>>>>>>>> `GRPC_ENABLE_FORK_SUPPORT=0` to disable this feature and see if it >>>>>>>>> fixes >>>>>>>>> the issue. >>>>>>>>> >>>>>>>>> Also, in your step 5 you note that the C++ client isn't >>>>>>>>> communicating with the server. If you remove the fork+exec of a C++ >>>>>>>>> subprocess altogether, do you still see this intermittent exception >>>>>>>>> in the >>>>>>>>> Python client? >>>>>>>>> >>>>>>>>> Eric >>>>>>>>> >>>>>>>>> On Wed, Feb 20, 2019 at 6:57 AM Alex <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> I should add that the Python client application which owns the >>>>>>>>>> Python grpc client is the one that runs the C++ grpc client as a >>>>>>>>>> subprocess >>>>>>>>>> in case that makes a difference. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "grpc.io" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/grpc-io/b323fac3-978b-47c1-b1fa-555c2f62b544%40googlegroups.com >>>>>>>>>> <https://groups.google.com/d/msgid/grpc-io/b323fac3-978b-47c1-b1fa-555c2f62b544%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "grpc.io" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/grpc-io/af4d55fa-a5e0-4e3f-a5ad-9cb62378703d%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/grpc-io/af4d55fa-a5e0-4e3f-a5ad-9cb62378703d%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAMC1%3Djds-2oQth%2B7Of2rnN%3DEcenfjA-H4CyngE7Ktum4wBwKYQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
