Hi Joe and Stuart,
Given the inconsistencies mentioned, I see how this change may not be worth the
hassle, so I’ll drop it. I appreciate the thoughtful responses to explain your
reasoning.
Thanks!
Ryan
> On Jul 7, 2023, at 4:21 PM, Stuart Marks <stuart.ma...@oracle.com> wrote:
>
> Hi Ryan,
>
> Thanks for trying out JDK 21 early access.
>
> The issue you raise is indeed an inconsistency, but it's not clear which way
> it should be resolved, or even whether it needs to be resolved, as the
> consequences are quite minor.
>
> Specifically, when the Sequenced Collections JEP was integrated, it included
> these changes (edited for brevity):
>
> - public interface List<E> extends Collection<E> {
> + public interface List<E> extends SequencedCollection<E> {
>
> - public interface SortedSet<E> extends Set<E> {
> + public interface SortedSet<E> extends Set<E>, SequencedSet<E> {
>
> As you observed, on List, SequencedCollection has replaced Collection, but on
> SortedSet, SequencedSet was added alongside Set. Which way is correct? Should
> SequencedCollection have been added to List, instead of replacing Collection?
> Or on SortedSet, should SequencedSet have replaced Set instead of simply
> being added?
>
> (I have to admit my first inclination was that the SortedSet change was a
> mistake, and that SequencedSet ought to have replaced Set. I think the reason
> it turned out the way it did was that, my earliest prototype, before I
> published anything, had a single interface OrderedCollection. This was
> retrofitted onto SortedSet as
>
> public interface SortedSet<E> extends Set<E>, OrderedCollection<E> {
>
> Only later did I decide to add OrderedSet, and so I changed the declaration to
>
> public interface SortedSet<E> extends Set<E>, OrderedSet<E> {
>
> and I didn't notice that the interfaces could be minimized. And of course
> there were a couple rounds of renaming subsequent to this.)
>
> In any case, this inconsistency is of little consequence, as the members
> available to users of the interfaces in question are the same regardless of
> the way the interfaces are declared. This is true from both a source and
> binary compatibility standpoint.
>
> Joe's point about compatibility is that even though such changes are
> *visible* to reflective code, they aren't *incompatible*. There are many
> compatible changes possible, such as moving the declaration of a method
> upward in the hierarchy. From the standpoint of reflection, that method may
> appear to have been removed from some interface; but that doesn't mean it's
> incompatible. Reflective code needs to be adjusted to take such things into
> account. The presence or absence of Collection as a superinterface of List is
> a similar case.
>
> Perhaps in some sense the JDK itself ought to be more consistent about this.
> We have for example this:
>
> class HashSet extends AbstractSet implements Set, Cloneable, Serializable
>
> but also this:
>
> class EnumSet extends AbstractSet implements Cloneable, Serializable
>
> (That is, EnumSet is "missing" Set.)
>
> Questions about this have been asked on Stack Overflow, and various answers
> there have made up a rationale about the possibly-redundant interfaces being
> included explicitly because it expresses intent more clearly, or some such.
> My own guess is that nobody has paid much attention to this issue because
> resolving it one way or another hardly makes any practical difference.
>
> s'marks
>
>> On 7/7/23 7:25 AM, Ryan Ernst wrote:
>> Thanks for laying out your thinking, Joe. I will watch your talks.
>> If I understood your response correctly, you are ok making such a change,
>> especially since it is semantically equivalent? If that’s the case, is JDK
>> 21 past the point of feature release, or should the change target only 22?
>> It doesn’t matter much to me, but I thought since SequencedCollection is
>> added in 21 it would be good to “fix” this there so as to avoid a gap in
>> behavior.
>>>> On Jun 30, 2023, at 6:13 PM, Joseph D. Darcy <joe.da...@oracle.com> wrote:
>>>
>>> Hi Ryan,
>>>
>>> Apropos of this discussion, I happened to recently give a talk to the JCP
>>> that in part covered behavioral compatibility in the JDK:
>>>
>>> https://jcp.org/aboutJava/communityprocess/ec-public/materials/2023-06-13/JCP-EC-Public-Agenda-June-2023.html
>>> https://jcp.org/aboutJava/communityprocess/ec-public/materials/2023-06-13/Contributing_to_OpenJDK_2023_04_12.pdf
>>>
>>> There are many observable details of the JDK implementation, including
>>> reflective details, timing, etc. there are _not_ part of the interfaces
>>> users should rely on.
>>>
>>> My current evaluation here is that it would set an unfortunate precedent to
>>> preclude making the sort of source changes in questions because of
>>> (potential) compatibility concerns in reflective modeling, especially in a
>>> feature release. (IMO, there is a stronger argument for not making such a
>>> change in an update release, but even there as the change is a semantically
>>> equivalent refactoring, I think it is generally fine.)
>>>
>>> HTH,
>>>
>>> -Joe
>>>
>>>> On 6/29/2023 11:30 AM, Ryan Ernst wrote:
>>>> Thanks for replying, Joe. First, let me reiterate, we fully admit
>>>> there was a bug in painless, we stopped short in walking the class
>>>> hierarchy. There is no bug in the SequencedCollection hierarchy. But I
>>>> do think there is an inconsistency.
>>>>
>>>>> The two definition are semantically equivalent
>>>>> ...
>>>>> The JDK 22 javadoc for List shows:
>>>>>> All Superinterfaces:
>>>>>> Collection<E>, Iterable<E>, SequencedCollection<E>
>>>> While that is true, in practice the reflection API does not give the
>>>> same collapsed view that javadocs do. Calling getInterfaces() on a
>>>> class only returns direct super interfaces, so
>>>> List.class.getInterfaces() no longer returns Collection.class in JDK
>>>> 21.
>>>>
>>>> The inconsistency I see is that SortedSet.class.getInterfaces() *does*
>>>> still return Set.class. Was that intentional? It seems like either
>>>> SortedSet does not need to extend Set directly, or List should still
>>>> extend Collection directly. And doing the latter would provide an
>>>> extra layer of "compatibility" for applications that may look at
>>>> direct super interfaces, and be surprised that Collection disappeared
>>>> across JDK versions for List.
>>>>
>>>>> On Wed, Jun 28, 2023 at 6:31 PM Joseph D. Darcy <joe.da...@oracle.com>
>>>>> wrote:
>>>>> Hello,
>>>>>
>>>>> What is Painless doing that would fail under
>>>>>
>>>>> List extends SequencedCollection ...
>>>>>
>>>>> but work under
>>>>>
>>>>> List extends SequencedCollection, Collection ...
>>>>>
>>>>> The two definition are semantically equivalent since SequencedCollection
>>>>> itself extends Collection.
>>>>>
>>>>> The JDK 22 javadoc for List shows:
>>>>>
>>>>>> All Superinterfaces:
>>>>>> Collection<E>, Iterable<E>, SequencedCollection<E>
>>>>> There are certainly implementation artifacts concerning the details of
>>>>> how a type was declared that exposed via core reflection (for the
>>>>> javax.lang.model API during annotation processing at compile time), but
>>>>> it is a bug for third party programs to rely on such details.
>>>>>
>>>>> HTH,
>>>>>
>>>>> -Joe
>>>>>
>>>>> On 6/27/2023 7:37 AM, Ryan Ernst wrote:
>>>>>> Hi core-libs-dev,
>>>>>>
>>>>>> I know various threads have existed over the past few months on
>>>>>> SequencedCollection and its implications on compatibility. I wanted to
>>>>>> raise this semi-related issue we noticed recently after beginning
>>>>>> testing against Java 21.
>>>>>>
>>>>>> Elasticsearch began testing against Java 21, and noticed a series of
>>>>>> failures in Painless (our builtin scripting language, which
>>>>>> dynamically compiles to bytecode). Most tests that used List failed to
>>>>>> compile, with several unknown methods (eg add). Painless builds a
>>>>>> hierarchy of classes it knows about for method resolution. This
>>>>>> hierarchy didn't know anything about SequencedCollection since we
>>>>>> currently compile against Java 17. Now, this was ultimately a bug in
>>>>>> Painless, because we knew there could be cases where we encounter
>>>>>> unknown classes in the hierarchy (eg a class which is not blessed to
>>>>>> be visible in the language). However, I think the reason we found that
>>>>>> bug (List extending from SequencedCollection) might still cause issues
>>>>>> for some.
>>>>>>
>>>>>> While debugging the issue, something that struck me as interesting in
>>>>>> the SequencedCollection hierarchy is the difference between List and
>>>>>> SortedSet. Both of these classes were made to be compatible with
>>>>>> sequenced classes by adding the new classes as super interfaces,
>>>>>> SequencedCollection and SequencedSet, respectively. However, while
>>>>>> SortedSet was *added* as a super interface, List had its super
>>>>>> interface Collection *replaced* by SequencedCollection.
>>>>>>
>>>>>> I don't know enough about the rampdown process to know if this is
>>>>>> still fixable in Java 21. It is probably an extreme edge case that
>>>>>> doesn't matter, since eg instanceof Collection will still work in
>>>>>> existing code. But for consistency, it would seem better to maintain
>>>>>> Collection as a direct super interface of List. Is there any reason
>>>>>> such a change doesn't fit with the collections architecture going
>>>>>> forward?
