Alan Bateman wrote:
Joe Darcy wrote:
Greetings.
Recently Mark Wielaard started an effort to run the regression tests
on OpenJDK builds and publish the results
(http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001921.html).
Certain tests in the nio and networking area assume Sun-internal
hosts like javaweb are available; the shell test
test/java/net/InetAddress/ptr/lookup.sh
has a string for javaweb and
test/java/nio/channels/TestUtil.java
has the following constants used by other tests:
// Test hosts used by the channels tests - change these when
// executing in a different network.
public static final String HOST = "javaweb.sfbay.sun.com";
public static final String REFUSING_HOST = "jano.sfbay.sun.com";
public static final String FAR_HOST = "theclub.ireland.sun.com";
public static final String UNRESOLVABLE_HOST =
"blah-blah.blah-blah.blah";
Mark observed these three hosts are needed:
With daytime and echo available. The only wrinkle is that some tests
expect to be able to use quick timeouts from HOST, but to need long
timeouts from FAR_HOST. I think we could provide something public for
this. But I might have missed some tests that need other services. Do
you have a full overview?
(http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2008-May/001972.html)
First, is this all the networking configuration information that is
used in the regression tests?
For the tests in the java/nio and sun/nio tree then the dependencies
are centralized in TestUtil.java. When we moved these tests from
closed->open we did talk about moving this configuration into a
properties/configuration file but figured there wasn't much difference
between editing a config file vs. a source file.
I think being able to pass in your own config file would be useful for
this use case; it is awkward to have to edit the source to the tests to
get them to pass :-) If -Dnetworking.test.config.file were defined, the
test could use those values instead of the defaults...
For the classic networking, the dependencies are scattered. I believe
some of the tests weren't moved from closed->open for this reason.
Some of the tests uses an embedded HTTP or FTP server but there are
others that depend on DNS, proxy, etc. Michael or Jessie may have more
to say on this.
Could the classic networking tests be refactored a bit to be centralized
like the nio tests? Would it be natural for classic networking and nio
to get their host information from the same place?
BTW: I briefly scanned the summary and most of the java/nio failures
appear to be *HOST related. Now that folks are running these tests
outside Sun it is time to own up to a few intermittent/timing related
failures with 4 or 5 of these tests. I've been meaning to fix these
test issues for a while but I haven't had the cycles. I'll dig up my
notes on these. One other thing - I notice
test/java/nio/channels/DatagramChannel/NotBound.java on the list -
that test is failing because it is missing the public modifier. The
test is bogus anyway so it might be best to just remove this test from
jdk6 and open6. For jdk7/OpenJDK I have spec clarification in the
works to fix the spec hole and as part of that work the test will be
replaced (see 6621689).
Yes, fixing the intermittent failures and removing the invalid test in
OpenJDK 6 sounds good.
Second, it would be helpful is this host information could be
configured without changing the sources of the test each time. I've
spoken briefly to Jon about ways this kind of information can be
passed into a jtreg run. Environment variables can be used, as can
system properties. A system property could also be used to specify a
file from which the information was read. With a bit of jtreg
hacking, it is feasible portions of the jtharness interview procedure
could be exposed to configure information that way too.
I would be concerned with a solution that requires environment
variables or jtreg requiring configuration from somewhere other than
the test suite. At least in Sun (sorry for being selfish), these tests
are run by various teams on hundreds of machines. I don't think we'd
want to require the javatest interview be run or have .jti files
stored in various places with configuration information that will
quickly go stale. If you mean that the configuration can be overridden
via some input to jtreg that might be okay but would require the
distributions and others to maintain that configuration. Anyway, I'm
interested to hear the outcome of the discussion.
Even if the Sun-internal settings remain baked in, I think an easy way
to pass in a limited number of configuration settings is appropriate
and shouldn't be too hard to accommodate in the test code. Presumably
an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be
too awkward to use.
-Joe
