You should verify that you haven’t saturated the network path to host - otherwise as you say you are working around it - but all you’ve done is effectively increase the timeout.
If you timestamp the messages and inspect on the receiver you may see that they are significantly delayed. > On May 16, 2022, at 8:41 AM, envee <neeraj.vaidy...@gmail.com> wrote: > > > Hi All, > I have been able to get around my issue by creating more than 1 connection > with the web server/service. I got this clue from a message I posted on > github issues page. Here is a link to the issue in case anyone is interested: > https://github.com/golang/go/issues/52845 > > It appears that the HTTP/2 client was blocked in previous requests when I > increased the TPS rate to more than ~2500. This caused those requests blocked > to get timed out as it appears that the timer starts from the time the HTTP/2 > Do method is invoked. And by the time the message being sent out gets a > chance in the queue, it does not have enough time to get written out to the > socket. This causes the "RoundTripper context deadline exceeded" errors. > > >> On Thu, May 12, 2022 at 5:58 PM envee <neeraj.vaidy...@gmail.com> wrote: >> Hi Robert, do you have any inputs about this ? >> >>> On Wednesday, 11 May 2022 at 08:31:24 UTC+10 envee wrote: >>> Ok, so you can ignore my comment about the incomplete payload being sent. >>> It is just that it is being sent in 2 frames as "partial entities". >>> But I am still wondering why there is a context deadline exceeded which >>> then results in a RST_STREAM with CANCEL being sent by my client ? >>> it's just 100ms which have elapsed since the last part of the payload was >>> sent in the DATA frame. >>> >>>> On Wednesday, 11 May 2022 at 07:47:15 UTC+10 envee wrote: >>>> Thanks for your reply, Robert. >>>> >>>> I enabled GODEBUG for http2 in my client application and noticed the >>>> following sequence of log lines for any particular stream for which I see >>>> my client sending an RST_STREAM frame. >>>> >>>> 2022/05/10 21:59:12.827474 http2: Framer 0xc23f448000: wrote HEADERS >>>> flags=END_HEADERS stream=247467 len=65 >>>> 2022/05/10 21:59:12.896018 http2: Framer 0xc23f448000: wrote DATA >>>> stream=247467 len=75 data="{\n\t\"invocationSequenceNumber\": >>>> 1,\n\t\"invocationTimeStamp\": \"2022-05-10T11:59" >>>> 2022/05/10 21:59:12.996612 RoundTrip failure: context deadline exceeded >>>> 2022/05/10 21:59:12.996692 http2: Framer 0xc23f448000: wrote RST_STREAM >>>> stream=247467 len=4 ErrCode=CANCEL >>>> >>>> Usually, the payload that I send in my DATA frame is of size ~2400 bytes. >>>> And for such "successful" requests in the logs I can see that it only >>>> prints some part and says "(2153 bytes omitted)". That is fair enough. >>>> >>>> But in this erroneous case, the DATA frame size is shown as 75 bytes and >>>> the log line does not contain any such extra information about "omitting >>>> bytes" from the log. >>>> I am wondering if my client failed to upload the entire JSON payload and >>>> hence this caused it to timeout waiting for Header. >>>> >>>> My client timeout is set to 5seconds, but with the logs being shown for >>>> the particular failing stream, why is there a timeout after just 100 >>>> milli-seconds i.e. between sending off the DATA frame and waiting for >>>> HEADER frame ? >>>> >>>>> On Tuesday, 10 May 2022 at 22:49:25 UTC+10 ren...@ix.netcom.com wrote: >>>>> Conn.Error() should give you the underlying error as well. >>>>> >>>>> >>>>>> On May 10, 2022, at 7:47 AM, robert engels <ren...@ix.netcom.com> wrote: >>>>>> >>>>>> I would add some logging to >>>>>> >>>>>> func (cs *clientStream) cleanupWriteRequest(err error) >>>>>> >>>>>> but you should be getting the underlying error when you perform the next >>>>>> write on the stream - maybe you are not processing this error properly >>>>>> (there is a code - CANCEL, but also an err). >>>>>> >>>>>>> On May 10, 2022, at 6:59 AM, Robert Engels <ren...@ix.netcom.com> wrote: >>>>>>> >>>>>>> Set a breakpoint on the RST_STREAM generation and run under the >>>>>>> debugger. >>>>>>> >>>>>>>>> On May 10, 2022, at 4:18 AM, envee <neeraj....@gmail.com> wrote: >>>>>>>>> >>>>>>>> Hi All, I am using a Go HTTP/2 client (no SSL) to send requests at >>>>>>>> the rate of around 3000 transactions per second (TPS) to a Web-server >>>>>>>> which is based on Netty. >>>>>>>> >>>>>>>> At lower rates of say around 500 TPS , I do not see this issue, but at >>>>>>>> rates as high as 2500 - 3000 TPS, I can see that my Go HTTP/2 client >>>>>>>> sends RST_STREAM frame right after transmitting the JSON payload via >>>>>>>> the DATA frame. The Error in the RST_STREAM is seen as CANCEL even >>>>>>>> though the response from the server for the Request has not been >>>>>>>> received. Also, there is no timeout that has occured which might cause >>>>>>>> this. >>>>>>>> >>>>>>>> This occurs for some requests and not all, maybe about 100 out of the >>>>>>>> 3000 per second. >>>>>>>> >>>>>>>> If you observe the screenshot attached, the HEADER is transmitted in >>>>>>>> packet 118847 in stream id 49765. The DATA frame is in packet 122113. >>>>>>>> >>>>>>>> Right after that, the client transmits RST_STREAM for stream id 49765 >>>>>>>> in packet 122277. >>>>>>>> >>>>>>>> And then this results in an error on the response from the Server in >>>>>>>> packet 122483 and 122485 which says "Request stream 49765 is not >>>>>>>> correct for server connection" >>>>>>>> >>>>>>>> Any clues what could be causing this issue of the client sending >>>>>>>> RST_STREAM before receiving the HTTP response from the server ? >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "golang-nuts" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to golang-nuts...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/golang-nuts/56766546-356f-4901-b3d1-827be5bb0947n%40googlegroups.com. >>>>>>>> <Screenshot 2022-05-10 190439.png> >>>>>> >>>>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/ebea4500-f99d-46d7-856c-67d6e01d5cdan%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CALFCev4zXUi2yrejOn5S41DHYvax%2BH35xzs0-q4LgEnxf8VieA%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7123FC85-1A71-45DD-99C0-157A1E92FAD6%40ix.netcom.com.
