Hi Cay, No problems at all! 🙂 Sharing the background is interesting because people rarely can see "what didn't work" or "what led to this".
>I don't understand why a short-circuiting integrator is "more capable" than a >greedy one. And why the "less capable" one would then be the subclass. Being able to consume Set(0..N) elements is a much larger set than Set(N), or in other terms, being able to (but not necessarily having to) terminate before consuming an entire stream makes the operation able to execute on top of "infinite" streams, greedy operations wouldn't terminate in such cases. So if a "short-circuiting" operation can consume potentially any number of elements, and a "greedy" operation only can consume all elements, and all-elements is a length subset of "any prefix"—so it is "less capable". >But what about implementing a gatherer that never short-circuits, like all >three gatherers in the JEP? Here, "ofGreedy" is an indication that an >optimization can be made (pushing vs. pulling), right? And that's what we want >the implementor to choose. Not necessarily push-vs-pull (Java Streams are by nature push-streams) but rather the cost of having to track/inspect short-circuiting across N stages of processing turns out to be a non-negligible cost so if you don't want to pay for it, you can opt out. For short-circuiting operations it's a cost you're willing to pay in order to be able to only consume a prefix of the stream. >That's where my struggle comes in. How can I phrase my advice in a symmetric >way? Right now, my advice is "if the integrator doesn't short-circuit, >advertise that benefit by using ofGreedy" and "if the gatherer can't combine >results, advertise that drawback by using ofSequential". Ah, I see what you're thinking there. I tend to try to want to frame things like that from multiple vantage points, so to me it would look something like this: Implementor: "use ofGreedy to signal that this operation does not itself initiate short-circuiting" Reviewer: "when you see ofGreedy verify that this operation does not itself initiate short-circuiting" Debugger: "when the expectation was that the stream should be short-circuiting and isn't, one thing to look for is ofGreedy" Implementor: "use ofSequential to signal that this operation is intended to always be executed sequentially" Reviewer: "when you see ofSequential, understand why the operation can't/shouldn't be parallelized for parallel streams" Debugger: "if the parallel stream is not performing as well as expected, one thing to look for is ofSequential! Many times, it's the same person in the different roles at different times, so understanding what the signal indicates to each role is good. >Put the factory method first in the JEP :-) I'd expect most Java developers to encounter this via the Javadoc and I put the factory as the main example: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/util/stream/Gatherer.html >1. The way to tell whether a gatherer is sequential is very unusual. To >provide a sequential gatherer I must fail to override a default method. I >don't recall seeing anything like this in the Java API. To be fair, the "interface with only methods returning functional interfaces" is a pretty rare sight as well? Using sentinel values turned out to be the low-energy state solution: By default a Gatherer is stateless, if you need state, you need to provide a Supplier which creates it. By default a Gatherer is sequential, if you need parallelization, you need to provide a BinaryOperator which provides it. By default a Gatherer does nothing on end-of-stream, if you need to do something then, you need to provide a BiFunction which does it. >2. The way to tell whether an integrator is greedy is through a subinterface. >That's fine, except it's a marker interface with no new methods. And then why >not use a subinterface for parallelizable gatherers, where one could make >combiner a subinterface method? Too OO for 2024? What about stateless Gatherers? Is that a marker interface too? I've been down this rabbit hole a couple of times and have the scars to prove it. 🙂 >3. Here we have two different mechanisms for what streams and collectors >address with a simple and unified approach: flags. I guess it bears repeating: Please see the discussion around why Characteristics didn't make it. 🙂 >I don't strongly care about this, just answering your question. Fair enough! Cheers, √ Viktor Klang Software Architect, Java Platform Group Oracle ________________________________ From: Cay Horstmann <cay.horstm...@gmail.com> Sent: Tuesday, 15 October 2024 16:07 To: Viktor Klang <viktor.kl...@oracle.com>; core-libs-dev@openjdk.org <core-libs-dev@openjdk.org> Subject: [External] : Re: Fw: New candidate JEP: 485: Stream Gatherers Hi Viktor, thanks again for sharing your design decisions. I agree this is a minor matter because not many programmers will implement gatherers. Let me try to explain my confusion. On 15/10/2024 13:46, Viktor Klang wrote: > Making of() the short-name is because it is more capable than ofGreedy() > (which does not support short-circuiting). So using it signals a choice of > less capability. I don't understand why a short-circuiting integrator is "more capable" than a greedy one. And why the "less capable" one would then be the subclass. What is the capability? To never short-circuit? To signal short-circuiting? > The reason for ofSequential/of instead of ofParallel/of is that a > parallel-capable Gatherer covers more scenarios than a sequential-only > versison, so giving the shorter name to the less-capable version would not > stand out in the same way when reviewing, and Collector.of is > parallel-capable, which seemed more consistent to make Gatherer.of > parallel-capable as well. Again, what is "less capable"? When you write "not stand out in the reviewing", I interpret that as follows: "of" means the mechanism providing maximum efficiency, that we want implementors to use. And "ofXXX" means they didn't go all the way, either because of the nature of the processing or because they didn't take the trouble to do all the work. With of/ofSequential, I can see that point of view. "ofSequential" is a warning that, when used with a parallel stream, there is the potential for disappointment. But what about implementing a gatherer that never short-circuits, like all three gatherers in the JEP? Here, "ofGreedy" is an indication that an optimization can be made (pushing vs. pulling), right? And that's what we want the implementor to choose. That's where my struggle comes in. How can I phrase my advice in a symmetric way? Right now, my advice is "if the integrator doesn't short-circuit, advertise that benefit by using ofGreedy" and "if the gatherer can't combine results, advertise that drawback by using ofSequential". Or maybe I misunderstand greedy? > > >First off, I think factory methods should be the favored approach. > > Is there something which could be made clearer around that, because that's > how I already view it. Put the factory method first in the JEP :-) > >The details are fussy, with the marker interface and the magic default > combiner. > > I think this is worth expanding a bit on, in what way are they fussy? > Maybe fussy is not a good term. Baroque? This is what went through my head when I studied the design. 1. The way to tell whether a gatherer is sequential is very unusual. To provide a sequential gatherer I must fail to override a default method. I don't recall seeing anything like this in the Java API. 2. The way to tell whether an integrator is greedy is through a subinterface. That's fine, except it's a marker interface with no new methods. And then why not use a subinterface for parallelizable gatherers, where one could make combiner a subinterface method? Too OO for 2024? 3. Here we have two different mechanisms for what streams and collectors address with a simple and unified approach: flags. I don't strongly care about this, just answering your question. Cheers, Cay -- Cay S. Horstmann | https://urldefense.com/v3/__https://horstmann.com__;!!ACWV5N9M2RV99hQ!NoHCnee1zi29L19LosVsyGdjFa1TJ9Ioszv2U02jMGnoWjQbyoGMCq4d8tv7O--v6s6gLgqOGfNr1RwC2sOUl5MfHQ$
