RFR(s): Improving performance of Windows socket connect on the loopback adapter

[Nikola Grcevski]

Hello net-dev,

During a recent performance investigation of Gradle/Kotlin build daemon 
discovery, we found that Windows socket connect on a non-existent service is 
taking a lot longer than on Linux and MacOS. The socket timeout and retransmit 
defaults on Windows are quite long and it typically takes 2 seconds to discover 
that there's no listener on a given address/port. After a discussion with the 
Windows network engineering team, they suggested that we should change the 
timeout defaults when connecting to the loopback adapter to improve the 
experience. The change is only recommended to do for the loopback adapter.

Please find the webrev with this improvement here:

http://cr.openjdk.java.net/~adityam/nikola/fast_connect_loopback/

For the change itself I re-used an existing macro to detect the loopback 
adapter (IS_LOOPBACK_ADDRESS), but I had to modify it slightly to avoid a 
compilation error because of an older SDK compatibility. The macro appears to 
be unused at the moment.

For the new function I added (NET_EnableFastTcpLoopbackConnect), I followed the 
approach taken by the implementation of NET_EnableFastTcpLoopback, which is 
declared as JNICALL and stubbed out as an empty function for Unix's. It would 
be simpler to implement this new utility function as a static for Windows only, 
but I wasn't sure if there are any specific guidelines when implementing net 
helper functions. I would appreciate some feedback on the best way to implement 
this in the net code.

The SIO_TCP_INITIAL_RTO flag was introduced in Windows 8, however I was told 
that there are no side-effects on calling WSAIoctl with it on older versions of 
Windows. The call will simply not do anything and return SOCKET_ERROR with 
WSAEOPNOTSUPP as last error.

I have tested the change for correctness on Windows 10 with jtreg:tier1 and 
full jtreg:net and jtreg:nio. I'm expanding the testing to rmi and other 
platforms to make sure everything works well.

The fix, when applied, significantly speeds up Gradle/Kotlin compilation time 
on Windows in certain scenarios.

I'm also providing here a simple benchmark (pasted at the end of this email) 
which tests the performance improvement on Windows:

Windows 10 before the change:

Benchmarking for address 127.0.0.1, elapsed time 20687 ms
Benchmarking for address 192.168.1.1, elapsed time 20574 ms

Windows 10 after the change:

Benchmarking for address 127.0.0.1, elapsed time 356 ms
Benchmarking for address 192.168.1.1, elapsed time 21113 ms

Ubuntu 18.04 for comparison:

Benchmarking for address 127.0.0.1, elapsed time 42 ms
Benchmarking for address 192.168.1.1, elapsed time 59 ms


Thanks in advance,
Nikola Grcevski
Microsoft

Benchmark program follows (parts of it converted from the actual Kotlin compile 
daemon):

import java.io.IOException;
import java.net.Inet6Address;
import java.net.InetAddress;
import java.rmi.ConnectException;
import java.rmi.ConnectIOException;
import java.rmi.registry.LocateRegistry;
import java.rmi.server.RMISocketFactory;

public class Test {

    final String IPV4_LOOPBACK_INET_ADDRESS = "127.0.0.1";
    final String IPV6_LOOPBACK_INET_ADDRESS = "::1";

    public static void main(String[] args) {
        Test test = new Test();

        test.measureFailedConnectToAddress(test.loopbackInetAddressName());
        test.measureFailedConnectToAddress("192.168.1.1");
    }

    void measureFailedConnectToAddress(String address) {
        System.out.print("Benchmarking for address " + address);
        long startTime = System.currentTimeMillis();
        for (int i = 0; i < 10; i++) {
            tryConnectToDaemon(address, 12345 + i);
        }

        System.out.println(", elapsed time " + (System.currentTimeMillis() - 
startTime) + " ms");
    }

    String loopbackInetAddressName() {
        try {
            if (InetAddress.getByName(null) instanceof Inet6Address) {
                return IPV6_LOOPBACK_INET_ADDRESS;
             } else {
                return IPV4_LOOPBACK_INET_ADDRESS;
             } 
        } catch (IOException e) {
            // getLocalHost may fail for unknown reasons in some situations, 
the fallback is to assume IPv4 for now
            return IPV4_LOOPBACK_INET_ADDRESS;
        }
    }

    private void tryConnectToDaemon(String address, int port) {
        RMISocketFactory defaultFactory =
                RMISocketFactory.getDefaultSocketFactory();
        try {
            LocateRegistry.getRegistry(
                    address,
                    port,
                    defaultFactory).lookup("KotlinJvmCompilerService");
        } catch (ConnectException | ConnectIOException e) {
        } catch (Exception x) {
            throw new RuntimeException(x);
        }
    }
}