[JDK-8257080] Java does not try all DNS results when opening a socket
Hello everyone, I would like to bring up the enhancement JDK-8257080[1] on this mailing list. tl;dr symptoms: Java will use the first IP address from DNS and will fail to connect when the target host is offline, even when the other IPs would have worked. This is also true for the more recent Socket implementations from Java 14 onwards.. Proposed resolution: Implementation similar to python. Late(r) DNS resolution when the socket is being opened (i.e. a target port is known), then iterate over IP addresses returned by DNS and try a socket_open. Workarounds: None. It is not possible to write an agent which alters java.net classes, as they are protected. I would like to bring this issue to your attention. I reported it using my work mail at oracle’s bug tracker, but I think other applications might suffer from this as well. Best regards, Ben [1] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8257080
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
On 16.12.20 13:49, Alan Bateman wrote: On 16/12/2020 09:04, Benjamin Marwell wrote: Hi Alan, > I can't tell if your proposal is focused on the Socket constructors > that take a hostname or more generally changing all APIs that > connect to a SocketAddress to support connecting to an > isUnresolved InetSocketAddress Way before we even get to sockets. I am talking about InetAddress. It’s resolving logic is resolve()[0]. That’s it. But an InetAddress does not carry any port information and thus should not resolve in my opinion. --- I think I would start looking in java/net/AbstractPlainSocketImpl.java. These are the relevant lines: https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/net/AbstractPlainSocketImpl.java#L149-L164 <https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/net/AbstractPlainSocketImpl.java*L149-L164__;Iw!!GqivPVa7Brio!IQGo98vib-LqfZaVN1Y7MJ51_ZRRrljY_n6IM-btaxHD3suFHe77kiBgTeSYGhb8tw$> For the new implementation these lines seem pretty relevant: https://github.com/openjdk/jdk/blob/51d5164ca2b4801c14466e8d1420ecf27cb7615f/src/java.base/share/classes/sun/nio/ch/NioSocketImpl.java#L560-L562 <https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/51d5164ca2b4801c14466e8d1420ecf27cb7615f/src/java.base/share/classes/sun/nio/ch/NioSocketImpl.java*L560-L562__;Iw!!GqivPVa7Brio!IQGo98vib-LqfZaVN1Y7MJ51_ZRRrljY_n6IM-btaxHD3suFHe77kiBgTeQy8PBhSg$> That is where I would probably add the for loop. Here is python’s implementation as a reference: https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Lib/socket.py#L707-L719 <https://urldefense.com/v3/__https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Lib/socket.py*L707-L719__;Iw!!GqivPVa7Brio!IQGo98vib-LqfZaVN1Y7MJ51_ZRRrljY_n6IM-btaxHD3suFHe77kiBgTeTvZM9G2A$> By the way, any changes would solve a few other issues as well: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8201428 <https://urldefense.com/v3/__https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8201428__;!!GqivPVa7Brio!IQGo98vib-LqfZaVN1Y7MJ51_ZRRrljY_n6IM-btaxHD3suFHe77kiBgTeQkc7Ex1w$> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8192780 <https://urldefense.com/v3/__https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8192780__;!!GqivPVa7Brio!IQGo98vib-LqfZaVN1Y7MJ51_ZRRrljY_n6IM-btaxHD3suFHe77kiBgTeSpuxbIvA$> If I read your mail correctly, I think what you are proposing is to change the existing APIs to allow an unresolved InetSocketAddress to be specified, is that right? I suspect this approach will have a lot of implications for methods that are called after the socket is created as it would require re-creating the underlying socket when a connect attempt fails (the socket may have been explicitly bound or socket options were set). There are issues with the semantics of the timeout parameter. It might not be possible to even do this with APIs such as SocketChannel when the channel is configured non-blocking. If the starting point is the Socket constructor that takes a host name and port then it might be easier because that can do the lookup and cycle through the addresses and/or overlap the connection attempts. If you have cycles to do experiments and report back then it would be useful. I assume the attractiveness of changing the existing APIs is so that existing usages, e.g. HTTP protocol handler, could use it without changing to a new APIs. Maybe changing existing vs. new API could be part of your investigation? I think it would also be useful to know if a simple iteration through the addresses is sufficient for more cases or whether something close to Happy Eyeballs would be needed. There have been a few discussions here over the years on the latter as it is hard to get right and there may be a case of having support for that in the APIs. -Alan Huh… the APIs have already been changed which is why the issues above exist. There is no workaround anymore. It is true that I did not take into account that other methods would rely on this. The biggest issue I see (easily) is the connect timeout. It must either be distributed along all connection attempts or per connection attempt – both would change semantics. I just looked up Happy Eyeballs, and this comes very close to what I mean. I like that it implies DNS-RR. Lets see if I (or anyone else) can come up with a sample implementation. Thanks, Ben
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
On 16.12.20 16:12, Alan Bateman wrote: On 16/12/2020 13:40, Benjamin Marwell wrote: Huh… the APIs have already been changed which is why the issues above exist. There is no workaround anymore. Can you expand on this as it's not clear what you mean by "the APIs have already changed"? The code that you referenced are implementation classes, not exposed. -Alan. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8192780 Granted, it is an SPI. But it seemed to have been used.
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
Sounds good to me, but – to be honest – a simpler solution would be enough for now, at least for me. I actually need something which works on Java 8. I also want to share an excerpt from an RFC: RFC 1123 2.3 Applications on Multihomed hosts … Application protocol implementations **SHOULD** be prepared to try multiple addresses from the list until success is obtained. … SRC: https://tools.ietf.org/html/rfc1123 Java sadly does not implement this. Am Mi., 16. Dez. 2020 um 20:33 Uhr schrieb Bernd Eckenfels < e...@zusammenkunft.net>: > Hello, > > A „happy eyeballs“ implementation, not only for Multiple IPv4 addresses > but also for IPV6/IPv4 Mixed would be a good thing, however since this > > a) adds additional connection Timeouts or decreases connection deadlines > b) potentially is stateful > > I think it’s not a good idea to enable it for all connections. A new > method to specify a policy (and allow a new default policy) would probably > be better. > > I don’t think doing that in the resolving alone would work, since there is > also a question of parallelity and a endpoint specific state. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > -- > *Von:* net-dev im Auftrag von Aleks > Efimov > *Gesendet:* Wednesday, December 16, 2020 5:53:02 PM > *An:* Benjamin Marwell > *Cc:* Alan Bateman ; OpenJDK Network Dev list < > net-dev@openjdk.java.net> > *Betreff:* Re: [JDK-8257080] Java does not try all DNS results when > opening a socket > > Hi Benjamin, > > As Alan stated I'm working on adding an SPI [1] which will provide a > possibility to alter how host names and IP addresses are resolved by JDK > platform. > I believe it would be possible to use this mechanism for addressing > issue described in JDK-8257080. > > Best Regards, > Aleksei > > [1] WIP jdk-sandbox branch: > https://github.com/openjdk/jdk-sandbox/tree/JDK-8244202-nspi-stream-branch > > > On 16/12/2020 15:42, Alan Bateman wrote: > > On 16/12/2020 15:21, Benjamin Marwell wrote: > >> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8192780 > >> > >> Granted, it is an SPI. But it seemed to have been used. > >> > > > > Aleksej Efimov is working on an InetAddress SPI, I'm sure there will > > be a draft JEP at some point. It has the potential to complement the > > exploration that we discussing here. > > > > -Alan > >
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
> unless there are any objections, I will close 8257080 No objection from me. > Things have moved on a lot since then, for example I see no reason why > the Java HTTP Client, that uses non-blocking NIO socket channels, could > not do it's own multi-connect thing. Well, I am stuck with Java 8 and if any LDAP server goes down, all java applications will, too. Python, .NET, etc. are not affected. In Java 8 there is no way around it. The next LTS is Java 11 which does ont include the Ldap DNS class you mentioned. As the java.net classes are all not modifiable using a agent or such, this is still a quite unpleasant situation. Which is why I raised this issue in the first place. - Ben Am Di., 22. Dez. 2020 um 18:28 Uhr schrieb Chris Hegarty < chris.hega...@oracle.com>: > I believe that 8170568 [1] better describes the issue being discussed > here. So, unless there are any objections, I will close 8257080 as a > duplicate of 8170568. > > --- > > There was some renewed interest in this area back in mid 2019, when we > touched on it (and several other issues) relating to IPv6 [2] (yes, I > know that this is not an IPv6-only specific issue). > > This is not a new issue, but unfortunately not much progress has been > made in this area (beyond some embedded or mobile platforms doing their > own thing). I remember discussing this with some folks while at the 80th > IETF meeting in Prague, in 2011. Specifically at the time, I prototyped > an implementation of Happy Eyeballs [3] at the java.net.Socket level. If > memory serves me correct, there were some not-so-straightforward > specification issues that would need to worked out, especially relating > to exceptions and failures (I don't have the details to hand). > > Things have moved on a lot since then, for example I see no reason why > the Java HTTP Client, that uses non-blocking NIO socket channels, could > not do it's own multi-connect thing. This is clearly unrelated to > whatever, if anything, is done for java.net.Socket.connect(String,int). > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8170568 > [2] https://mail.openjdk.java.net/pipermail/net-dev/2019-April/012371.html > [3] https://tools.ietf.org/html/rfc6555 > > > > On 17 Dec 2020, at 14:39, Chris Hegarty > wrote: > > > > Looping in a prior relevant issue in JIRA: > > > > https://bugs.openjdk.java.net/browse/JDK-8170568 - > > Improve address selection for network clients > > > > -Chris. > > > >> On 17 Dec 2020, at 13:45, Daniel Fuchs wrote: > >> > >> Hi Simone, > >> > >> We are investigating introducing a Service Provider interface > >> which would allow an application to replace the default > >> built-in implementation that blocks inside the kernel. > >> > >> There is no plan to introduce any public asynchronous API to perform > >> address resolution at this point. With the advent of Loom and > >> virtual threads I'm not sure there would be much motivation for > >> that anyway. > >> > >> best regards, > >> > >> -- daniel > >> > >> On 16/12/2020 19:59, Simone Bordet wrote: > >>> Hi, > >>> On Wed, Dec 16, 2020 at 5:55 PM Aleks Efimov < > aleksej.efi...@oracle.com> wrote: > > Hi Benjamin, > > As Alan stated I'm working on adding an SPI [1] which will provide a > possibility to alter how host names and IP addresses are resolved by > JDK > platform. > I believe it would be possible to use this mechanism for addressing > issue described in JDK-8257080. > >>> Is it hopefully going to be non-blocking? > >> > > > >
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
Is there more than just one? The built-in will provide support for javax.naming.spi.LdapDnsProvider from java 12 onwards. But I was not aware of any custom implementations I could use as a drop-in replacement with Apache Shiro. The Java 8 implementation will just use the socket’s/InetAdress’s DNS resolution algo, which is just getAddresses()[0]. Am Sa., 26. Dez. 2020 um 15:53 Uhr schrieb Florian Weimer : > * Benjamin Marwell: > > > Well, I am stuck with Java 8 and if any LDAP server goes down, all > > java applications will, too. Python, .NET, etc. are not affected. > > Surely that totally depends on the LDAP client implementation in use? >
Re: [JDK-8257080] Java does not try all DNS results when opening a socket
Hi Alan, actually, I do not know, but I suspect it should work OOTB as we are using the javax.naming.ldap.InitialLdapContext [1]. I discussed the new SPI with the shiro team, there is no experience with it yet. An integration test with an ApacheDS server as might reveal if it works as expected. However, no such tests exists at this time. I will get back to you as soon as we have a working IT in place. Contributions are always welcome! ;-) - Ben [1]: https://github.com/apache/shiro/blob/0e5a4428bcaa0a4c03680f5faad5a4c897379497/core/src/main/java/org/apache/shiro/realm/ldap/JndiLdapContextFactory.java#L507-L509 Am Di., 29. Dez. 2020 um 07:39 Uhr schrieb Alan Bateman < alan.bate...@oracle.com>: > On 27/12/2020 10:05, Benjamin Marwell wrote: > > Is there more than just one? The built-in will provide support > > for javax.naming.spi.LdapDnsProvider from java 12 onwards. > > But I was not aware of any custom implementations I could use as a > > drop-in replacement with Apache Shiro. > > > Do you know if the Apache Shiro project are using the SPI when deployed > on newer Java releases? > > -Alan >
