RE: Backporting the TCP loopback fast path (Windows) improvement to OpenJDK v7

[Martin Sawicki (MS OPEN TECH)]

Thank you all for the help!

From: Seán Coffey [mailto:sean.cof...@oracle.com]
Sent: Friday, January 30, 2015 12:07 PM
To: Alan Bateman; Valery Kopylov (Akvelon); Martin Sawicki (MS OPEN TECH); 
jdk7u-...@openjdk.java.net
Cc: OpenJDK Network Dev list; Kirk Shoop (MS OPEN TECH)
Subject: Re: Backporting the TCP loopback fast path (Windows) improvement to 
OpenJDK v7

Thanks all - Agreed changes pushed to 7u85 and I'll make a request to see if we 
can include this in 7u80 code base.

regards,
Sean.
On 30/01/2015 16:00, Alan Bateman wrote:
On 30/01/2015 12:18, Seán Coffey wrote:
Alan,

hadn't spotted the race issue. I guess we could delay setting ver_initialized 
until OSVERSIONINFO is populated.
Alternatively - like you say, I think we can revert back to the simpler edit :

http://cr.openjdk.java.net/~coffeys/webrev.8060170.7u.v2/webrev/
I'll run with this version unless I hear objections.
I think this better so I'd suggest go with this version.

-Alan

Re: RFR 8064924: Update java.net.URL to work with modules

[Peter Levart]

Hi Chris,

I looked at your solution to make URLStreamHandlerFactory interface a 
service provider using ServiceLoader API and in addition adding new URL 
static method to programmaticaly register URLStreamHandlerFactories. 
There are a couple of issues with your approach I'd like to discuss.


The programmatic approach using static URL method does give you some 
means of order in which registered URLStreamHandlerFactories are tried 
to create URLStreamHandler for particular protocol - the registration 
order. It means that this method should only be called by one "party" in 
the VM or else it is hard to control the order of registration.


ServiceLoader is a declarative approach, but does not give you order by 
default. Also, your ServiceLoader approach gives a way for 
URLStreamHandlerFactories to register in the system without security 
checks. If a particular 
META-INF/services/java.net.URLStreamHandlerFactory is found, it's 
content is used to load a class and instantiate a factory which is used 
in URL constructors then. Previously, one had to have a "setFactory" 
permission to set the URLStreamHandlerFactory or appropriate 
PropertyPermission for seting the package prefix property. This can be 
fixed.


The java.protocol.handler.pkgs system property manipulation in 
HttpSOAPConnection has a couple of issues as it is now (without your 
changes):


- first, the property can be set too late, when https protocol has 
already been used in URL constructions before any HttpSOAPConnection is 
used. The URLStreamHandler cached in URL will be used for ever 
regardless of HttpSOAPConnection setting the property later.
- second, the HttpSOAPConnection.initHttps is racy - it bases it's 
decision to instantiate and add a security Provider on the 
presence/absence of a particular package in the value of 
java.protocol.handler.pkgs system property, which it modifies 
afterwards. Concurrent calls to initHttps (which is called from 
HttpSOAPConnection.doGet and .post) can lead to multiple security 
Providers instantiations and adds...


Your question "why is jaxws using the old https client ??" is right on 
the spot. Is it just because nobody has updated it yet? Why does it have 
to choose the URLStreamHandlers based on JVM it is executing on (doesn't 
IBM JVM already arrange for it's own HTTPS support by default?). Is it 
maybe the security Provider that is dependent on these *old* HTTPS 
implementations because it is using their internal APIs? This needs to 
be cleared, I think.


Anyway, I think there is an alternative to programmatic registration 
approach of URLStreamHandlerFactories. Using just ServiceLoader and a 
new default method on URLStreamHandlerFactory interface to provide 
order. Here's what I'm thinking about:


http://cr.openjdk.java.net/~plevart/jdk9-dev/URLStreamHandlerFactory/webrev.01/

You see, even tests can be made to use this approach. I think 
declarative approach is better because it doesn't depend on 
initialization order.


Additional ideas you can take from this for your patch are the following:
- the way URLStreamHandlerFactories obtained from ServiceLoader are 
security-checked before used. The implementation class / code source has 
to be given "setFactory" permission if such factory is to be considered 
at all.
- the way URLStreamHandlers are cached using ConcurrentHashMap instead 
of Hashtable which simplifies caching "atomicity" checks and does not 
use lock(s) on fast-path.


So what do you think of ServiceLoader-only approach?


Regards, Peter


On 01/30/2015 02:36 PM, Chris Hegarty wrote:

This is phase 1, of getting java.net.URL work with modules.

Being able to effectively specify URL protocol handler factories as fully 
qualified class names, through the 'java.protocol.handler.pkgs' system property 
is problematic. It requires the protocol handler factory implementation class 
to be public and accessible, as the current implementation tries to instantiate 
it through core reflection. Since the protocol handler factory must be an 
implementation of a URLStreamHandlerFactory, it lends itself to being 
retrofitted with a ServiceLoader lookup. Note, the 'java.protocol.handler.pkgs' 
system property mechanism predates ServiceLoader. URL protocol handlers would 
most likely have used service loader if it were available at the time.

Some URL protocol handlers are fundamental to the platform itself, like 'file' 
and 'jar', it is not appropriate to attempt a service loader lookup for these, 
as they may lead to recursive initialization issues. However, Java Plugin, 
Webstart, and possibly other containers, do override the 'jar' handler. A new 
API will be provided for this purpose. Providing an API solution should not 
interfere with system initialization as it will only be called after the system 
comes up and user code is executing.

The 'file' protocol handler factory will no longer be overridable, and the 
system property will no longer be consulted.

Webrev and spec diffs:
   h

Re: RFR 8064924: Update java.net.URL to work with modules

[Alan Bateman]

On 31/01/2015 19:42, Peter Levart wrote:

Hi Chris,

I looked at your solution to make URLStreamHandlerFactory interface a 
service provider using ServiceLoader API and in addition adding new 
URL static method to programmaticaly register 
URLStreamHandlerFactories. There are a couple of issues with your 
approach I'd like to discuss.


The programmatic approach using static URL method does give you some 
means of order in which registered URLStreamHandlerFactories are tried 
to create URLStreamHandler for particular protocol - the registration 
order. It means that this method should only be called by one "party" 
in the VM or else it is hard to control the order of registration.


ServiceLoader is a declarative approach, but does not give you order 
by default. Also, your ServiceLoader approach gives a way for 
URLStreamHandlerFactories to register in the system without security 
checks. If a particular 
META-INF/services/java.net.URLStreamHandlerFactory is found, it's 
content is used to load a class and instantiate a factory which is 
used in URL constructors then. Previously, one had to have a 
"setFactory" permission to set the URLStreamHandlerFactory or 
appropriate PropertyPermission for seting the package prefix property. 
This can be fixed.


:

Anyway, I think there is an alternative to programmatic registration 
approach of URLStreamHandlerFactories. Using just ServiceLoader and a 
new default method on URLStreamHandlerFactory interface to provide 
order. Here's what I'm thinking about:


http://cr.openjdk.java.net/~plevart/jdk9-dev/URLStreamHandlerFactory/webrev.01/


I don't think we should we expose ordering values in 
URLStreamHandlerFactory, it looks a bit odd and not clear how an 
implementation can choose a useful value. There is a general issue that 
ServiceLoader doesn't currently support a means to order service 
providers but I think we can re-examine that when we move to modules and 
and linking. That is, have a consistent way to configure ordering that 
we can use everywhere that ServiceLoader is used rather than doing 
one-off solutions.


The other thing is that it's not clear how this would work for a factory 
for the jar protocol that is itself deployed in a JAR file on the class 
path. This is important for containers that want to do their own caching 
and they want their jar protocol handler configured system-wide before 
starting any applications. It's also part of the motive for the 
addURLStreamHandlerFactory in the original proposal.


I think you have good point with the setFactory permission, that does 
need to be looked at.


-Alan.

Re: RFR 8064924: Update java.net.URL to work with modules

[Peter Levart]

Hi Alan,

On 01/31/2015 10:33 PM, Alan Bateman wrote:

On 31/01/2015 19:42, Peter Levart wrote:

Hi Chris,

I looked at your solution to make URLStreamHandlerFactory interface a 
service provider using ServiceLoader API and in addition adding new 
URL static method to programmaticaly register  
URLStreamHandlerFactories. There are a couple of issues with your 
approach I'd like to discuss.


The programmatic approach using static URL method does give you some 
means of order in which registered URLStreamHandlerFactories are 
tried to create URLStreamHandler for particular protocol - the 
registration order. It means that this method should only be called 
by one "party" in the VM or else it is hard to control the order of 
registration.


ServiceLoader is a declarative approach, but does not give you order 
by default. Also, your ServiceLoader approach gives a way for 
URLStreamHandlerFactories to register in the system without security 
checks. If a particular 
META-INF/services/java.net.URLStreamHandlerFactory is found, it's 
content is used to load a class and instantiate a factory which is 
used in URL constructors then. Previously, one had to have a 
"setFactory" permission to set the URLStreamHandlerFactory or 
appropriate PropertyPermission for seting the package prefix 
property. This can be fixed.


:

Anyway, I think there is an alternative to programmatic registration 
approach of URLStreamHandlerFactories. Using just ServiceLoader and a 
new default method on URLStreamHandlerFactory interface to provide 
order. Here's what I'm thinking about:


http://cr.openjdk.java.net/~plevart/jdk9-dev/URLStreamHandlerFactory/webrev.01/


I don't think we should we expose ordering values in 
URLStreamHandlerFactory, it looks a bit odd and not clear how an 
implementation can choose a useful value. There is a general issue 
that ServiceLoader doesn't currently support a means to order service 
providers but I think we can re-examine that when we move to modules 
and and linking. That is, have a consistent way to configure ordering 
that we can use everywhere that ServiceLoader is used rather than 
doing one-off solutions.


I agree. Putting the order on the SPI API is not the right solution. The 
order should be configured in one place. But there needs to be some kind 
of handle each service exposes for order configuration to reference. So 
one idea how to extend the ServiceLoader mechanism is this:


create a special class-scope runtime annotation...

public @interface ServiceProvider {
String name();
}

...with which service implementation classes can optionally be 
annotated. This could enable ServiceLoader API extension like:


public static  ServiceLoader load(Class service, String 
serviceProviderName)


that would return an Iterable over implementations that are annotated 
with a particular @ServiceProvider(name = ...) annotation (similar to 
security Providers but simpler).


In addition one could specify a system property with the key being 
prefixed by service interface FQ class name, like:


java.net.URLStreamHandlerFactory.serviceLoaderOrder=providerName1,providerName2,providerName3,...





The other thing is that it's not clear how this would work for a 
factory for the jar protocol that is itself deployed in a JAR file on 
the class path. This is important for containers that want to do their 
own caching and they want their jar protocol handler configured 
system-wide before starting any applications. It's also part of the 
motive for the addURLStreamHandlerFactory in the original proposal.


I see. But isn't URL.setURLStreamHandlerFactory() enough for that 
purpose? It can only be set once, but there can only be *one* container 
that wants it's jar protocol handler configured system-wide.


Regards, Peter



I think you have good point with the setFactory permission, that does 
need to be looked at.


-Alan.

Re: RFR 8064924: Update java.net.URL to work with modules

[Bernd]

Hello,

What about simply rejecting overlapping programmatic registrations, and for
the service loader using the natural order (class loader search order). I
guess most extensions register new schemes only, as it was not easy before
in such a central shared system component. an ordering api might not really
help. Its might be better to allow specific versions like a factory or even
something thread local? (Similar to jndi enc)

Bernd
Am 01.02.2015 00:48 schrieb "Peter Levart" :

>  Hi Alan,
>
> On 01/31/2015 10:33 PM, Alan Bateman wrote:
>
> On 31/01/2015 19:42, Peter Levart wrote:
>
> Hi Chris,
>
> I looked at your solution to make URLStreamHandlerFactory interface a
> service provider using ServiceLoader API and in addition adding new URL
> static method to programmaticaly register  URLStreamHandlerFactories. There
> are a couple of issues with your approach I'd like to discuss.
>
> The programmatic approach using static URL method does give you some means
> of order in which registered URLStreamHandlerFactories are tried to create
> URLStreamHandler for particular protocol - the registration order. It means
> that this method should only be called by one "party" in the VM or else it
> is hard to control the order of registration.
>
> ServiceLoader is a declarative approach, but does not give you order by
> default. Also, your ServiceLoader approach gives a way for
> URLStreamHandlerFactories to register in the system without security
> checks. If a particular META-INF/services/java.net.URLStreamHandlerFactory
> is found, it's content is used to load a class and instantiate a factory
> which is used in URL constructors then. Previously, one had to have a
> "setFactory" permission to set the URLStreamHandlerFactory or appropriate
> PropertyPermission for seting the package prefix property. This can be
> fixed.
>
> :
>
> Anyway, I think there is an alternative to programmatic registration
> approach of URLStreamHandlerFactories. Using just ServiceLoader and a new
> default method on URLStreamHandlerFactory interface to provide order.
> Here's what I'm thinking about:
>
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/URLStreamHandlerFactory/webrev.01/
>
>
> I don't think we should we expose ordering values in
> URLStreamHandlerFactory, it looks a bit odd and not clear how an
> implementation can choose a useful value. There is a general issue that
> ServiceLoader doesn't currently support a means to order service providers
> but I think we can re-examine that when we move to modules and and linking.
> That is, have a consistent way to configure ordering that we can use
> everywhere that ServiceLoader is used rather than doing one-off solutions.
>
>
> I agree. Putting the order on the SPI API is not the right solution. The
> order should be configured in one place. But there needs to be some kind of
> handle each service exposes for order configuration to reference. So one
> idea how to extend the ServiceLoader mechanism is this:
>
> create a special class-scope runtime annotation...
>
> public @interface ServiceProvider {
> String name();
> }
>
> ...with which service implementation classes can optionally be annotated.
> This could enable ServiceLoader API extension like:
>
> public static  ServiceLoader load(Class service, String
> serviceProviderName)
>
> that would return an Iterable over implementations that are annotated with
> a particular @ServiceProvider(name = ...) annotation (similar to security
> Providers but simpler).
>
> In addition one could specify a system property with the key being
> prefixed by service interface FQ class name, like:
>
>
> java.net.URLStreamHandlerFactory.serviceLoaderOrder=providerName1,providerName2,providerName3,...
>
>
>
>
> The other thing is that it's not clear how this would work for a factory
> for the jar protocol that is itself deployed in a JAR file on the class
> path. This is important for containers that want to do their own caching
> and they want their jar protocol handler configured system-wide before
> starting any applications. It's also part of the motive for the
> addURLStreamHandlerFactory in the original proposal.
>
>
> I see. But isn't URL.setURLStreamHandlerFactory() enough for that purpose?
> It can only be set once, but there can only be *one* container that wants
> it's jar protocol handler configured system-wide.
>
> Regards, Peter
>
>
> I think you have good point with the setFactory permission, that does need
> to be looked at.
>
> -Alan.
>
>
>