[openjdk-x] Bind to a multicast address fails
Hello, it seems one of our external developers (Andre Naujoks, CC) found a bug while binding a IPv6 multicast UDP-socket for one of our projects. Since this seems to be a fundamental bug (from our perspective), we address this directly to this mailing list. (reproducible in OpenJDK 8-11, Windows and Linux) Following is the bug description from Andre: This bugs affects all currently available Java versions in Debian (7, 8, 10 and 11). If a java program tries to bind a datagram/UDP socket to a link- or node-local multicast address and also sets the needed interface index, the bind operation on the socket fails with "java.net.SocketException: Invalid argument". This is because the JVM only transfers the interface index to the scope_id field in the struct sockaddr_in6, if the address to bind to is a link local address (IN6_IS_ADDR_LINKLOCAL()). A bind to a multicast address of link- or node-local scope needs the scope_id field as well. (There might be an additional problem here, when binding to a higher scoped multicast address and trying to use the non-default-route interface) The attached patch fixes/adds this in the jvm. [Edit HDecke: Since I am not sure if attachments are allowed on this mailing list, I put the link to the corresponding debian bug log with the example and patch file at the end of the mail.] I applied the patch by copying it to an "apt-get source"d openjdk-8 packages debian/patches dir and including it in the debian/rules file in the COMMON_PATCHES variable. The resulting debian packages work as expected. Additionally attached is a very small test program that wrongly throws the mentioned exception. When run with the patched VM it works. You might have to edit the interface name in line 33 when trying this out. Regards, Andre This issue has also been reported to the debian bug logs: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907541 Best regards, Hendrik Decke Volkswagen group research Volkswagen Aktiengesellschaft Brieffach 17770 38436 Wolfsburg Tel. +49 5361 9-38363 Mobil +49-152-5-74 www.volkswagen-ag.com Volkswagen Aktiengesellschaft Registered Seat: Wolfsburg I Registration Court: Amtsgericht Braunschweig Commercial Register No.: 100484 Chairman of the Supervisory Board: Hans Dieter Pötsch Board of Management: Herbert Diess (Chairman), Oliver Blume, Jochem Heizmann, Gunnar Kilian, Andreas Renschler, Stefan Sommer, Rupert Stadler, Hiltrud D. Werner, Frank Witter Important Notice: The above information is automatically added to this e-mail. This addition does not constitute a representation that the content of this e-mail is legally relevant and/or is intended to be legally binding upon Volkswagen Aktiengesellschaft.
Re: [openjdk-x] Bind to a multicast address fails
On 07/09/2018 10:49, Decke, Hendrik (K-GERFA/A) wrote: Hello, it seems one of our external developers (Andre Naujoks, CC) found a bug while binding a IPv6 multicast UDP-socket for one of our projects. Since this seems to be a fundamental bug (from our perspective), we address this directly to this mailing list. (reproducible in OpenJDK 8-11, Windows and Linux) This bug was submitted this week on this issue: https://bugs.openjdk.java.net/browse/JDK-8210493 Have you tried joining the mullticast group, specifying the network interface, rather than binding to the multicast address? -Alan
Re: [openjdk-x] Bind to a multicast address fails
On 9/7/18 1:15 PM, Alan Bateman wrote: > On 07/09/2018 10:49, Decke, Hendrik (K-GERFA/A) wrote: >> >> Hello, >> >> it seems one of our external developers (Andre Naujoks, CC) found a >> bug while binding a IPv6 multicast UDP-socket for one of our projects. >> >> >> >> Since this seems to be a fundamental bug (from our perspective), we >> address this directly to this mailing list. >> >> (reproducible in OpenJDK 8-11, Windows and Linux) >> > This bug was submitted this week on this issue: > https://bugs.openjdk.java.net/browse/JDK-8210493 > > Have you tried joining the mullticast group, specifying the network > interface, rather than binding to the multicast address? Hi Alan. First of all, thank you for the quick reply. I was not aware, that there was actually a bug opened for that issue. The join is not the problem at this point. We need to bind the socket to the address to avoid receiving traffic from all multicast groups, that are joined on the system. Since a join joins the system (not the socket) to the group, all sockets bound to a port, which receive multicast traffic will receive all of that traffic, no matter the destination address. The bind prevents that. IP_MULTICAST_ALL sadly only works for IPv4 and the patch I tried to get IPV6_MULTICAST_ALL upstream into the kernel was even more sadly (almost) ignored. see https://marc.info/?l=linux-netdev&m=152344460530252&w=2 Andre > > -Alan
Re: [openjdk-x] Bind to a multicast address fails
On 07/09/2018 12:33, Andre Naujoks wrote: : Hi Alan. First of all, thank you for the quick reply. I was not aware, that there was actually a bug opened for that issue. The join is not the problem at this point. We need to bind the socket to the address to avoid receiving traffic from all multicast groups, that are joined on the system. Since a join joins the system (not the socket) to the group, all sockets bound to a port, which receive multicast traffic will receive all of that traffic, no matter the destination address. The bind prevents that. IP_MULTICAST_ALL sadly only works for IPv4 and the patch I tried to get IPV6_MULTICAST_ALL upstream into the kernel was even more sadly (almost) ignored. see https://marc.info/?l=linux-netdev&m=152344460530252&w=2 The JDK sets IP_MULTICAST_ALL to 0 on Linux to avoid this interference and get the behavior consistent with other platforms. It's news to me that it doesn't work on IPv6 as we always disable it, even if the socket is IPv6. Does it work when the IPv6 socket joins an IPv4 multicast group? -Alan.
Re: [openjdk-x] Bind to a multicast address fails
On 9/7/18 1:57 PM, Alan Bateman wrote: > On 07/09/2018 12:33, Andre Naujoks wrote: >> : >> Hi Alan. >> >> First of all, thank you for the quick reply. I was not aware, that there >> was actually a bug opened for that issue. >> >> The join is not the problem at this point. We need to bind the socket to >> the address to avoid receiving traffic from all multicast groups, that >> are joined on the system. Since a join joins the system (not the socket) >> to the group, all sockets bound to a port, which receive multicast >> traffic will receive all of that traffic, no matter the destination >> address. The bind prevents that. IP_MULTICAST_ALL sadly only works for >> IPv4 and the patch I tried to get IPV6_MULTICAST_ALL upstream into the >> kernel was even more sadly (almost) ignored. see >> https://marc.info/?l=linux-netdev&m=152344460530252&w=2 >> > The JDK sets IP_MULTICAST_ALL to 0 on Linux to avoid this interference > and get the behavior consistent with other platforms. It's news to me > that it doesn't work on IPv6 as we always disable it, even if the socket > is IPv6. Does it work when the IPv6 socket joins an IPv4 multicast group? I have not tried joining IPv4 groups on an IPv6 socket through java, since we do not use IPv4 at all in this particular environment. I have tried setting IP_MULTICAST_ALL to 0 in the IPv6 scenario (in a C++ project), hoping it would help, but it did not. Hence the patch for the linux kernel. Would it actually help, if I tried the IPv4 multicast group bind on an IPv6 socket? The bind to an address would be a workaround for the missing IPV6_MULTICAST_ALL handling. Andre > > -Alan. >
Re: [openjdk-x] Bind to a multicast address fails
On Fri, Sep 7, 2018 at 6:56 AM Andre Naujoks wrote: > On 9/7/18 1:15 PM, Alan Bateman wrote: > > On 07/09/2018 10:49, Decke, Hendrik (K-GERFA/A) wrote: > >> > >> Hello, > >> > >> it seems one of our external developers (Andre Naujoks, CC) found a > >> bug while binding a IPv6 multicast UDP-socket for one of our projects. > >> > >> > >> > >> Since this seems to be a fundamental bug (from our perspective), we > >> address this directly to this mailing list. > >> > >> (reproducible in OpenJDK 8-11, Windows and Linux) > >> > > This bug was submitted this week on this issue: > > https://bugs.openjdk.java.net/browse/JDK-8210493 > > > > Have you tried joining the mullticast group, specifying the network > > interface, rather than binding to the multicast address? > > Hi Alan. > > First of all, thank you for the quick reply. I was not aware, that there > was actually a bug opened for that issue. > > The join is not the problem at this point. We need to bind the socket to > the address to avoid receiving traffic from all multicast groups, that > are joined on the system. Since a join joins the system (not the socket) > to the group, all sockets bound to a port, which receive multicast > traffic will receive all of that traffic, no matter the destination > address. The bind prevents that. IP_MULTICAST_ALL sadly only works for > IPv4 and the patch I tried to get IPV6_MULTICAST_ALL upstream into the > kernel was even more sadly (almost) ignored. see > https://marc.info/?l=linux-netdev&m=152344460530252&w=2 Hi Andre, I spoke with a colleague about this kernel patch. They said first of all that multicast filtering is pretty complex in the kernel with a lot of subtle behaviors. But, they also said that it may have been ignored because of the format of the patch, perhaps even accidentally/automatically. The proposed patch has an "RFC" tag, and such patches apparently need to be in "git-format-patch" mode. Lastly, they said that since the time that the post was made, IP_MULTICAST_ALL (for IPv4 only of course) has changed a little bit in that "it only allows receipt of all multicast groups if a specific list of multicast groups has not already been set", so it may need to be updated accordingly. FWIW they didn't mention seeing any actual problems with the content of the patch, though I'm not sure how thoroughly they reviewed it. So, while I myself am not a Linux kernel contributor, I do suspect that if you reposted an updated version of the patch, in the correct format, it will enter the patch queue and may be more actively discussed. For more information, see [1]. [1] https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#the-canonical-patch-format -- - DML
Re: [openjdk-x] Bind to a multicast address fails
On Fri, Sep 7, 2018 at 11:08 AM David Lloyd wrote: > On Fri, Sep 7, 2018 at 6:56 AM Andre Naujoks wrote: > > On 9/7/18 1:15 PM, Alan Bateman wrote: > > > On 07/09/2018 10:49, Decke, Hendrik (K-GERFA/A) wrote: > > >> > > >> Hello, > > >> > > >> it seems one of our external developers (Andre Naujoks, CC) found a > > >> bug while binding a IPv6 multicast UDP-socket for one of our projects. > > >> > > >> > > >> > > >> Since this seems to be a fundamental bug (from our perspective), we > > >> address this directly to this mailing list. > > >> > > >> (reproducible in OpenJDK 8-11, Windows and Linux) > > >> > > > This bug was submitted this week on this issue: > > > https://bugs.openjdk.java.net/browse/JDK-8210493 > > > > > > Have you tried joining the mullticast group, specifying the network > > > interface, rather than binding to the multicast address? > > > > Hi Alan. > > > > First of all, thank you for the quick reply. I was not aware, that there > > was actually a bug opened for that issue. > > > > The join is not the problem at this point. We need to bind the socket to > > the address to avoid receiving traffic from all multicast groups, that > > are joined on the system. Since a join joins the system (not the socket) > > to the group, all sockets bound to a port, which receive multicast > > traffic will receive all of that traffic, no matter the destination > > address. The bind prevents that. IP_MULTICAST_ALL sadly only works for > > IPv4 and the patch I tried to get IPV6_MULTICAST_ALL upstream into the > > kernel was even more sadly (almost) ignored. see > > https://marc.info/?l=linux-netdev&m=152344460530252&w=2 > > Hi Andre, > > I spoke with a colleague about this kernel patch. They said first of > all that multicast filtering is pretty complex in the kernel with a > lot of subtle behaviors. But, they also said that it may have been > ignored because of the format of the patch, perhaps even > accidentally/automatically. The proposed patch has an "RFC" tag, and > such patches apparently need to be in "git-format-patch" mode. > Lastly, they said that since the time that the post was made, > IP_MULTICAST_ALL (for IPv4 only of course) has changed a little bit in > that "it only allows receipt of all multicast groups if a specific > list of multicast groups has not already been set", so it may need to > be updated accordingly. FWIW they didn't mention seeing any actual > problems with the content of the patch, though I'm not sure how > thoroughly they reviewed it. > > So, while I myself am not a Linux kernel contributor, I do suspect > that if you reposted an updated version of the patch, in the correct > format, it will enter the patch queue and may be more actively > discussed. For more information, see [1]. > > [1] > https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#the-canonical-patch-format I should also mention that the recommended approach would be to send a not-RFC version of the patch and see how that goes. -- - DML
