Hi Ryan,
Thanks for trying out JDK 21 early access.
The issue you raise is indeed an inconsistency, but it's not clear which way it
should be resolved, or even whether it needs to be resolved, as the consequences are
quite minor.
Specifically, when the Sequenced Collections JEP was integrated, it included these
changes (edited for brevity):
- public interface List<E> extends Collection<E> {
+ public interface List<E> extends SequencedCollection<E> {
- public interface SortedSet<E> extends Set<E> {
+ public interface SortedSet<E> extends Set<E>, SequencedSet<E> {
As you observed, on List, SequencedCollection has replaced Collection, but on
SortedSet, SequencedSet was added alongside Set. Which way is correct? Should
SequencedCollection have been added to List, instead of replacing Collection? Or on
SortedSet, should SequencedSet have replaced Set instead of simply being added?
(I have to admit my first inclination was that the SortedSet change was a mistake,
and that SequencedSet ought to have replaced Set. I think the reason it turned out
the way it did was that, my earliest prototype, before I published anything, had a
single interface OrderedCollection. This was retrofitted onto SortedSet as
public interface SortedSet<E> extends Set<E>, OrderedCollection<E> {
Only later did I decide to add OrderedSet, and so I changed the declaration to
public interface SortedSet<E> extends Set<E>, OrderedSet<E> {
and I didn't notice that the interfaces could be minimized. And of course there were
a couple rounds of renaming subsequent to this.)
In any case, this inconsistency is of little consequence, as the members available
to users of the interfaces in question are the same regardless of the way the
interfaces are declared. This is true from both a source and binary compatibility
standpoint.
Joe's point about compatibility is that even though such changes are *visible* to
reflective code, they aren't *incompatible*. There are many compatible changes
possible, such as moving the declaration of a method upward in the hierarchy. From
the standpoint of reflection, that method may appear to have been removed from some
interface; but that doesn't mean it's incompatible. Reflective code needs to be
adjusted to take such things into account. The presence or absence of Collection as
a superinterface of List is a similar case.
Perhaps in some sense the JDK itself ought to be more consistent about this. We have
for example this:
class HashSet extends AbstractSet implements Set, Cloneable, Serializable
but also this:
class EnumSet extends AbstractSet implements Cloneable, Serializable
(That is, EnumSet is "missing" Set.)
Questions about this have been asked on Stack Overflow, and various answers there
have made up a rationale about the possibly-redundant interfaces being included
explicitly because it expresses intent more clearly, or some such. My own guess is
that nobody has paid much attention to this issue because resolving it one way or
another hardly makes any practical difference.
s'marks
On 7/7/23 7:25 AM, Ryan Ernst wrote:
Thanks for laying out your thinking, Joe. I will watch your talks.
If I understood your response correctly, you are ok making such a change,
especially since it is semantically equivalent? If that’s the case, is JDK 21
past the point of feature release, or should the change target only 22? It
doesn’t matter much to me, but I thought since SequencedCollection is added in
21 it would be good to “fix” this there so as to avoid a gap in behavior.
On Jun 30, 2023, at 6:13 PM, Joseph D. Darcy <joe.da...@oracle.com> wrote:
Hi Ryan,
Apropos of this discussion, I happened to recently give a talk to the JCP that
in part covered behavioral compatibility in the JDK:
https://jcp.org/aboutJava/communityprocess/ec-public/materials/2023-06-13/JCP-EC-Public-Agenda-June-2023.html
https://jcp.org/aboutJava/communityprocess/ec-public/materials/2023-06-13/Contributing_to_OpenJDK_2023_04_12.pdf
There are many observable details of the JDK implementation, including
reflective details, timing, etc. there are _not_ part of the interfaces users
should rely on.
My current evaluation here is that it would set an unfortunate precedent to
preclude making the sort of source changes in questions because of (potential)
compatibility concerns in reflective modeling, especially in a feature release.
(IMO, there is a stronger argument for not making such a change in an update
release, but even there as the change is a semantically equivalent refactoring,
I think it is generally fine.)
HTH,
-Joe
On 6/29/2023 11:30 AM, Ryan Ernst wrote:
Thanks for replying, Joe. First, let me reiterate, we fully admit
there was a bug in painless, we stopped short in walking the class
hierarchy. There is no bug in the SequencedCollection hierarchy. But I
do think there is an inconsistency.
The two definition are semantically equivalent
...
The JDK 22 javadoc for List shows:
All Superinterfaces:
Collection<E>, Iterable<E>, SequencedCollection<E>
While that is true, in practice the reflection API does not give the
same collapsed view that javadocs do. Calling getInterfaces() on a
class only returns direct super interfaces, so
List.class.getInterfaces() no longer returns Collection.class in JDK
21.
The inconsistency I see is that SortedSet.class.getInterfaces() *does*
still return Set.class. Was that intentional? It seems like either
SortedSet does not need to extend Set directly, or List should still
extend Collection directly. And doing the latter would provide an
extra layer of "compatibility" for applications that may look at
direct super interfaces, and be surprised that Collection disappeared
across JDK versions for List.
On Wed, Jun 28, 2023 at 6:31 PM Joseph D. Darcy <joe.da...@oracle.com> wrote:
Hello,
What is Painless doing that would fail under
List extends SequencedCollection ...
but work under
List extends SequencedCollection, Collection ...
The two definition are semantically equivalent since SequencedCollection
itself extends Collection.
The JDK 22 javadoc for List shows:
All Superinterfaces:
Collection<E>, Iterable<E>, SequencedCollection<E>
There are certainly implementation artifacts concerning the details of
how a type was declared that exposed via core reflection (for the
javax.lang.model API during annotation processing at compile time), but
it is a bug for third party programs to rely on such details.
HTH,
-Joe
On 6/27/2023 7:37 AM, Ryan Ernst wrote:
Hi core-libs-dev,
I know various threads have existed over the past few months on
SequencedCollection and its implications on compatibility. I wanted to
raise this semi-related issue we noticed recently after beginning
testing against Java 21.
Elasticsearch began testing against Java 21, and noticed a series of
failures in Painless (our builtin scripting language, which
dynamically compiles to bytecode). Most tests that used List failed to
compile, with several unknown methods (eg add). Painless builds a
hierarchy of classes it knows about for method resolution. This
hierarchy didn't know anything about SequencedCollection since we
currently compile against Java 17. Now, this was ultimately a bug in
Painless, because we knew there could be cases where we encounter
unknown classes in the hierarchy (eg a class which is not blessed to
be visible in the language). However, I think the reason we found that
bug (List extending from SequencedCollection) might still cause issues
for some.
While debugging the issue, something that struck me as interesting in
the SequencedCollection hierarchy is the difference between List and
SortedSet. Both of these classes were made to be compatible with
sequenced classes by adding the new classes as super interfaces,
SequencedCollection and SequencedSet, respectively. However, while
SortedSet was *added* as a super interface, List had its super
interface Collection *replaced* by SequencedCollection.
I don't know enough about the rampdown process to know if this is
still fixable in Java 21. It is probably an extreme edge case that
doesn't matter, since eg instanceof Collection will still work in
existing code. But for consistency, it would seem better to maintain
Collection as a direct super interface of List. Is there any reason
such a change doesn't fit with the collections architecture going
forward?
