I am trying this new PMTUD functionality, but it seems it doesn't work. I have configured: CONFIG_NET_ICMP_PMTU_ENTRIES = 10 CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
Again I see lots of segments being sent (all with size 1400), and all of them are responded with the same ICMP reply. A couple of retransmissions are attempted, and then the connection is reset. I try again the same procedure (now that the system had the chance to discover the maximum PMTU), but it still fails. Again all segments have a size of 1400, instead of less. I am using sendfile() in case this matters. On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <f.j.pa...@gmail.com> wrote: > I can see that there are two different problems with MTU. > They are completely independent from each other, so let's start with the > simple case first. > > I am testing on an STM32F427, using Ethernet. > > As previously noted, the following code will cause the running task to > hang. > > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500); > > int sd = socket(AF_INET, SOCK_DGRAM, 0); > > struct sockaddr_in server; > server.sin_family = AF_INET; > server.sin_port = 1000; > server.sin_addr.s_addr = inet_addr("192.168.1.235"); > > uint8_t * data = malloc(2048); > memset(data, 0xAA, 2048); > > sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server)); > > close(sd); > > As you can see, the MTU is set to 1500, and then I try to send a UDP > datagram with a larger size (2048). > Indeed `devif_send()` fails, and the aforementioned semaphore is never > posted. > > (This is without buffering in UTP, in case this is important). > > This draft PR, provides a solution to the issue. > https://github.com/apache/nuttx/pull/9423 > > If this is correct, I will also check buffered UDP, and other uses of > devif_send(). > > Alternatively, devif_send() may be changed to actually return an error > code (instead of returning void), so improved error handling can take place. > > > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <f.j.pa...@gmail.com> > wrote: > >> The failure scenario is a bit more complicated... >> >> Give me some time and I will provide a correct and reproducible example, >> with a clear explanation. >> >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <f.j.pa...@gmail.com> >> wrote: >> >>> > sendfile should return an error in this case, but senfile should only >>> be >>> > used with TCP, not UDP, since sendfile doesn't have any logic to ack or >>> > retry.. >>> >>> Sorry if this wasn't clear. This last test was with plain old `send()`... >>> >>> I opened a UDP socket, and used `send()` to transmit a buffer larger >>> than the MTU. >>> Instead of getting an error, the application hangs indefinitely. >>> `devif_send()` is called periodically, but of course it always fails. >>> >>> >>> >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <xiaoxiang781...@gmail.com> >>> wrote: >>> >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos < >>>> f.j.pa...@gmail.com> >>>> wrote: >>>> >>>> > > You need to enable IP fragmentation in this case, which is also >>>> added >>>> > > recently and disabled by default: >>>> > > https://github.com/apache/nuttx/pull/8059 >>>> > <https://github.com/apache/nuttx/pull/8059> >>>> > > Otherwise, any packet bigger than MTU will be dropped silently. >>>> > >>>> > Yes, this is the expected behavior. >>>> > But, instead of dropping the packet, the system hangs because the >>>> semaphore >>>> > is never posted. >>>> > It just tries endlessly to call devif_send() which always fails. >>>> > >>>> > >>>> sendfile should return an error in this case, but senfile should only be >>>> used with TCP, not UDP, since sendfile doesn't have any logic to ack or >>>> retry.. >>>> >>>> >>>> > >>>> > >>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao < >>>> xiaoxiang781...@gmail.com> >>>> > wrote: >>>> > >>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos < >>>> > > f.j.pa...@gmail.com> >>>> > > wrote: >>>> > > >>>> > > > While experimenting with MTU, and checking the stability of my >>>> system, >>>> > I >>>> > > > noticed the following. >>>> > > > >>>> > > > I try to send a UDP datagram that is larger than the configured >>>> MTU. >>>> > > > In this case, the offending thread seems to hang indefinitely (or >>>> at >>>> > > least >>>> > > > waiting for a very long timeout?) >>>> > > > >>>> > > >>>> > > You need to enable IP fragmentation in this case, which is also >>>> added >>>> > > recently and disabled by default: >>>> > > https://github.com/apache/nuttx/pull/8059 >>>> > > Otherwise, any packet bigger than MTU will be dropped silently. >>>> > > >>>> > > >>>> > > > The problem seems to be this line: >>>> > > > >>>> > > > >>>> > > >>>> > >>>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197 >>>> > > > `devif_send()` fails because the datagram is too large, but >>>> > > > `pstate->st_sem` is never posted (the code returns immediately). >>>> > > > >>>> > > > This leaves the sending task to be blocked here: >>>> > > > >>>> > > > >>>> > > >>>> > >>>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469 >>>> > > > >>>> > > > Shouldn't this failure also post the semaphore? >>>> > > > And let the code proceed returning an error in `send()`? >>>> > > > >>>> > > > >>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos < >>>> > > f.j.pa...@gmail.com >>>> > > > > >>>> > > > wrote: >>>> > > > >>>> > > > > >>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao < >>>> > xiaoxiang781...@gmail.com> >>>> > > > > wrote: >>>> > > > > >>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos < >>>> > > > >> f.j.pa...@gmail.com> >>>> > > > >> wrote: >>>> > > > >> >>>> > > > >> > Hello, >>>> > > > >> > >>>> > > > >> > I encounter some problems using sendfile(). >>>> > > > >> > >>>> > > > >> > I am using sendfile to... send a file to a remote server, >>>> with my >>>> > > own >>>> > > > >> > implementation of an FTP client. >>>> > > > >> > sendfile() indeed starts to transmit chunks of the file, but >>>> as I >>>> > > see >>>> > > > in >>>> > > > >> > Wireshark, I get an ICMP response "Destination unreachable >>>> > > > >> (Fragmentation >>>> > > > >> > needed)". >>>> > > > >> > I have verified that the Ethrenet MTU is correctly set to >>>> 1500. >>>> > > > >> > >>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the problem >>>> is >>>> > > > solved. >>>> > > > >> > Communication succeeds. >>>> > > > >> > >>>> > > > >> > This raises some questions, and indicates some potential >>>> bugs: >>>> > > > >> > >>>> > > > >> > 1. Why is there a problem with MTU in the first place? >>>> Shouldn't >>>> > MTU >>>> > > > be >>>> > > > >> > negotiated? (Is this functionality available in NuttX?) >>>> > > > >> > >>>> > > > >> >>>> > > > >> MTU isn't negotiated but a physical attribute of your >>>> > > transport(netdev). >>>> > > > >> On >>>> > > > >> the other hand, PMTU could be discovered from ICMP. >>>> > > > >> >>>> > > > > >>>> > > > > I am not very familiar with MTU negotiation, so it seems that it >>>> > > doesn't >>>> > > > > happen in the network layer that I thought... >>>> > > > > >>>> > > > > >>>> > > > >> >>>> > > > >> >>>> > > > >> > 2. Why is the ICMP response not handled? It seems that >>>> sendfile() >>>> > > just >>>> > > > >> > ignores it and continues to send chunks, nevertheless. >>>> > > > >> > >>>> > > > >> >>>> > > > >> It is handled by the recent addition here: >>>> > > > >> https://github.com/apachey/nuttx/pull/9254 >>>> > > > >> <https://github.com/apache/nuttx/pull/9254> >>>> > > > >> but this feature is disabled by default, you have to enable it >>>> > > > manually.. >>>> > > > >> >>>> > > > > >>>> > > > > I will definitely take a look at this. Thank you. >>>> > > > > >>>> > > > > >>>> > > > >> >>>> > > > >> >>>> > > > > >>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving any >>>> ACKs >>>> > > back? >>>> > > > >> > AFAIK, depending on the configuration, TCP allows at most two >>>> > > pending >>>> > > > >> > segments on the wire. But I see dozens of them, till sendfile >>>> > > finally >>>> > > > >> > fails. >>>> > > > >> > >>>> > > > >> > >>>> > > > >> Why only two segments? TCP can send packages until the slide >>>> window >>>> > is >>>> > > > >> full. >>>> > > > >> >>>> > > > >> Disregard this. I was confused with delayed ACKs. Which is a >>>> > > receiver's >>>> > > > > functionality, not a sender's... >>>> > > > > >>>> > > > > >>>> > > > >> >>>> > > > >> > This last point is also verified in my MQTT client. >>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP segments >>>> > without >>>> > > > >> ACKing >>>> > > > >> > the previous data. >>>> > > > >> > >>>> > > > >> > So, is there any insight on the above? >>>> > > > >> > Is my configuration wrong, or is there anything wrong with >>>> TCP? >>>> > > > >> > >>>> > > > >> > Thank you. >>>> > > > >> > >>>> > > > >> >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> >>>