Hi Damjan,

I've come full circle on this and created a new webrev based on your original proposal. Using the current HTTP protocol handler that already implements the tunneling and authentication. See the updated webrev:

  http://cr.openjdk.java.net/~chegar/6370908/webrev.01/

I've used "more" reflection to avoid creating static dependencies on the internal http implementation. The addition of a test will ensure that any changes to the implementation will be picked up very quickly, i.e. the test will break.

-Chris.

On 04/03/2010 11:29, Damjan Jovanovic wrote:
Hi Chris

Proxies are on the application layer, TCP is a transport layer
protocol below. Using the application layer to establish in-band
transport layer connections is dodgy by its very nature. The multiple
sockets to get through an HTTP proxy issue is not unique to Java.
Here's a quick look at how several open source applications have dealt
with the problem:

* prtunnel (http://joshbeam.com/software/prtunnel.php), a C
application for tunneling through several proxy types, only supports
no authentication and BASIC HTTP authentication. proxychains does the
same. XChat does the same.
* ntlmaps.sourceforge.net and similar apps work by acting as another
proxy themselves. They can use as many sockets as necessary on the
outgoing end to the second proxy.
* pidgin (www.pidgin.im), an instant messaging app, has very good
proxy support including NTLM authentication for HTTP proxies. It only
uses a single socket and requires the proxy to use a persistent
connection. proxytunnel (proxytunnel.sourceforge.net) does the same.

I like pidgin's solution the best, and since it even works with NTLM,
the other authentication types should be easy. Most HTTP 1.1 proxies
should support persistent connections - after all, that feature
benefits proxies the most. So we should be able to get away with one
socket in most cases, maybe even for all commonly used HTTP proxies.
I'll try to do some tests this weekend.

Also, nothing stops a Socket from changing its impl. We would need a
very clever impl that somehow remembers options that have been set and
reapplies them to the new impl, but there's no theoretical reason why
connect() cannot create as many impls as are necessary to get through
the proxy.

On the other hand, all Java's solutions for connecting through proxies
are poor: SOCKS 4 is broken
(http://mail.openjdk.java.net/pipermail/net-dev/2010-February/001580.html),
server sockets are unsupported, datagram sockets are unsupported, and
all java.nio channels are unsupported. But then proxies aren't so
common any more: ADSL routers typically use NAT instead, and we don't
provide any NAT traversal support for server sockets and datagram
sockets using UPnP's IGD or Bonjour's NAT-PMP protocols. Maybe we
should?

Damjan

On Wed, Mar 3, 2010 at 3:55 PM, Christopher Hegarty -Sun Microsystems
Ireland<christopher.hega...@sun.com>  wrote:
Hi Damjan,

After spending more time looking at this, I still haven't found an elegant
solution for supporting authentication.

You've correctly identified an issue with the fact we have no guaranteed of
a persistent connection with the proxy. Changing the socket identity is not
as simple as overriding getIn/OutputStream since other methods like
setting/getting options, close, shutdown, etc use the impls fd field. This
field needs to refer to the correct file descriptor object.

There is quite a lot of logic in the http protocol handler to support
authentication and looking at it again it is closely tied to the handler
implementation itself. I would prefer to reuse this code if possible.

Have you made any progress on this? If we want authentication it may be more
straight forward to revert back to your original proposal of using
HttpURLConnection directly and having its socket exposed. Though, I'm still
not convinced of this solution either.

-Chris

Damjan Jovanovic wrote:

Hi Chris

I think that without authentication, you'll just get another RFE
asking for authentication. Many corporate proxies require
authentication, especially for a liberal request like CONNECT.

java.net.Socket seems to delegate its impl field's methods. If we have
another, internal socket in HttpConnectSocketImpl, and we override
getInputStream() and getOutputStream() in HttpConnectSocketImpl to
return this inner socket's streams, we can open as many sockets as we
like during authentication, and then just store the last one that
connected as this internal socket? This seems like a way to "change
socket identities" like I suggested.

Thank you
Damjan

On Mon, Mar 1, 2010 at 4:29 PM, Christopher Hegarty - Sun Microsystems
Ireland<christopher.hega...@sun.com>  wrote:

Hi Damjan,

Sorry for the delayed response, I'm juggling too many things!

I'm not sure how important Authentication is here. I initially pointed it
out, but hadn't done sufficient scoping so didn't encounter these issues.
Let me take another look at it, but I'm wondering if we should just
proceed
with it as is for now. Authentication is the reason that this RFE never
made
it back yet and has been sitting for a few years now.


-Chris.

On 24/02/2010 17:28, Damjan Jovanovic wrote:

On Mon, Feb 22, 2010 at 5:02 PM, Christopher Hegarty - Sun
Microsystems Ireland<christopher.hega...@sun.com>    wrote:

Hi Damjan,

Actually, I did some work on this back in 2006 (!), but never finished
it. I
brought the changes into a mercurial repository and created a webrev:

  http://cr.openjdk.java.net/~chegar/6370908/webrev.00/webrev/

Basically, this change provides the basic functionality, without any
frills,
authentication, etc. I think for tunneling sockets through a HTTP proxy
it
should be sufficient. Do you require authentication in your
environment?

To have authentication supported we would need to restructure the HTTP
protocol handler in sun.net.www.protocol.http.HttpURLConnection, so
that
we
can take advantage of the authentication schemes it already supports.
Not
a
big deal, just needs to be done.

The major problem with authentication is that there is no guarantee
the HTTP connection will be persistent. Authentication generally
requires making additional requests to the proxy, so we may need to
connect to the proxy more than once. Each socket can only connect
once, and the HttpConnectSocketImpl is a single immutable socket, so
it cannot directly be used for authentication.

A workaround may be to connect with a different socket for the initial
request, and then connect through the HttpConnectSocketImpl's own
socket for the final request. But then the problem becomes that if the
proxy doesn't require authentication, the initial socket will succeed
in connecting to the target host, and it will be too late to stop it
and use HttpConnectSocketImpl's own socket...

So we have the following options:
1. Only support proxies for which the entire request chain can be made
over a single persistent connection.
2. Connect to each target host up to twice, once when we believe the
proxy needs authentication but doesn't, and then again later with the
real socket. This will probably kill protocols that use one shot
connections, like FTP.
3. Find some way to change socket identities, then use as many sockets
as it takes and somehow adopt the final socket into
HttpConnectSocketImpl's own.

Any suggestions?

-Chris.

Damjan

On 21/02/2010 13:09, Damjan Jovanovic wrote:

Hi

  From http://bugs.sun.com/view_bug.do?bug_id=6370908

This RFE is basically about getting a TCP socket to tunnel through an
HTTP proxy using the HTTP CONNECT request.

I've found a hack to get this feature to work, using sun.net.*
packages and lots of reflection. Would it be acceptable to use this
solution (with some way to change socket identity) in a patch that
adds a java.net.HttpSocketImpl class similar to the
java.net.SocksSocketImpl class that's already used to tunnel through
SOCKS proxies? If not, in what other way should such a patch be done?

Thank you
Damjan Jovanovic

import java.net.*;
import java.io.*;
import java.lang.reflect.*;

public class TunnelProxy {
       private static Socket connectThroughHTTPProxy(String proxyHost,
int
proxyPort, String destinationHost, int destinationPort) throws
Exception
       {
               URL destinationURL = new URL("http://"; + destinationHost
+
":" +
destinationPort);
               sun.net.www.protocol.http.HttpURLConnection conn =
                       new sun.net.www.protocol.http.HttpURLConnection(
                               destinationURL, new
java.net.Proxy(java.net.Proxy.Type.HTTP, new
InetSocketAddress(proxyHost, proxyPort)));
               conn.setDoInput(true);
               conn.setDoOutput(true);
               conn.connect();
               conn.doTunneling();
               Field httpField =
conn.getClass().getDeclaredField("http");
               httpField.setAccessible(true);
               sun.net.www.http.HttpClient httpClient =
(sun.net.www.http.HttpClient) httpField.get(conn);
               Field serverSocketField =
sun.net.NetworkClient.class.getDeclaredField("serverSocket");
               serverSocketField.setAccessible(true);
               Socket socket = (Socket)
serverSocketField.get(httpClient);
               return socket;
       }

       public static void main(String[] args) throws Exception {
               System.setProperty("java.net.useSystemProxies", "true");
               InputStream in = connectThroughHTTPProxy(args[0],
Integer.parseInt(args[1]), args[2],
Integer.parseInt(args[3])).getInputStream();
               byte[] bytes = new byte[1024];
               int bytesRead;
               while ((bytesRead = in.read(bytes)) != -1) {
                       System.out.print(new String(bytes));
               }
       }
}

Reply via email to