Thanks for the feedback, Svein -- it is much appreciated! In my mind, the default expectation on Java interfaces is that there is no specified order of invocation around the exposed methods, and unless explicitly stated otherwise it has to be presumed that any order is fine.
Given that, and (as you noted) the specified behavior that repeated invocations must return semantically equivalent results, an implementation of an evaluator of a Gatherer must be able to call the Gatherer's initializer(), integrator(), combiner(), and finisher()-methods in any order and 1..N times. That being said, there's an established order upon which the returned functions are to be invoked -- starting with the initializer (presuming initializer != defaultInitializer()), via the integrator, then potentially the combiner, (presuming combiner != defaultCombiner() && we want parallel evaluation), then the finisher (presuming finisher != defaultFinisher()). Getting back to your example output, there's a lot of different things at play, such as (but not limited to) composition (both explicit and implicit), runtime information gathering, hoisting/caching, and more. If you're interested in diving even deeper, having a look at GathererOp<https://github.com/openjdk/jdk/blob/ec841740eb4cb6304243dd1b29a59acf9526bb79/src/java.base/share/classes/java/util/stream/GathererOp.java> is recommended! ? Cheers, ? Viktor Klang Software Architect, Java Platform Group Oracle ________________________________ From: core-libs-dev <core-libs-dev-r...@openjdk.org> on behalf of Svein Otto Solem <svein.otto.so...@gmail.com> Sent: Friday, 24 November 2023 20:31 To: core-libs-dev@openjdk.org <core-libs-dev@openjdk.org> Subject: RFR: 8319123: Implement JEP 461: Stream Gatherers (Preview) [v13] Tested the Gatherer implementation, and it is a very nice work. I have always wanted a way of writing my own stream operations and also have incomplete composable pipelines which could be combined in different ways. Very interesting. The four Gatherer functions - initializer/integrator/combiner/finisher - all return function objects. In the javadoc for these Gatherer functions, no requirements are described for these functions. (though API note on class level says "Each invocation of initializer(), integrator(), combiner(), and finisher() must return a semantically identical result.") It would be nice to have the requirements stated on the function level also. Looking at the class level javadoc is not always done. How often and in which sequence these functions are called are of interest, especially for stateful Gatherers. I created a simple gatherer which prints some trace info for these functions (see below). It is tested using branch "pr/16420" of the jdk pr. 21.nov. - The "initializer()" function are in some cases called twice from the library. - The "initializer().get()" function is called only once in the tested scenarios. This is expected since the method is called "initializer", but this behaviour could have been documented. - The sequence these functions are called varies from use case to use case, sometimes the "integrator()" is called before "initializer()". Sometimes the "finisher()" are called before first call to "integrator()", sometimes after. I've marked the unexpected/unneccesary/repeated calls with ? marks. All this may be correct, but is it ? ------------------- Trace -------------------- Using andThen Using separate gathers Using separate gathers with map(identity) Creates WithIndex(A) Creates WithIndex(A) Creates WithIndex(A) Creates WithIndex(B) Creates Integrator(A) Creates Integrator(A) Creates WithIndex(C) Creates WithIndex(B) Creates WithIndex(B) Creates Initializer(A) Creates Initializer(A) Creates Integrator(B) Creates Integrator(A) ?Creates Integrator(A) Creates WithIndex(C) Creates Finisher(A) Creates Finisher(A) Creates Integrator(C) Creates Initializer(B) Creates Initializer(B) Calls Collectors.toList Creates Integrator(B) Creates Integrator(B) Finished Collectors.toList Creates Finisher(B) Creates Finisher(B) ?Creates Integrator(C) Creates Initializer(C) Creates WithIndex(C) ?Creates Integrator(B) Creates Integrator(C) ?Creates Initializer(A) ?Creates Integrator(A) Creates Finisher(C) ?Creates Integrator(A) Creates Initializer(A) Calls Collectors.toList ?Creates Finisher(A) Invokes Initializer(A) Finished Collectors.toList ?Creates Initializer(B) Creates Initializer(B) Invokes Initializer(A) ?Creates Integrator(B) Invokes Initializer(B) Invokes Initializer(B) ?Creates Finisher(B) Creates Initializer(C) Invokes Initializer(C) Creates Initializer(C) Invokes Initializer(C) Invokes Integrator(A) Creates Integrator(C) Invokes Integrator(A) Invokes Integrator(B) Creates Finisher(C) Invokes Integrator(B) Invokes Integrator(C) Calls Collectors.toList Invokes Integrator(C) Invokes Integrator(A) Finished Collectors.toList Invokes Integrator(A) Invokes Integrator(B) Invokes Initializer(A) Invokes Integrator(B) Invokes Integrator(C) Invokes Initializer(B) Invokes Integrator(C) Invokes Integrator(A) Invokes Initializer(C) Invokes Integrator(A) Invokes Integrator(B) Invokes Integrator(A) Invokes Integrator(B) Invokes Integrator(C) Invokes Integrator(B) Invokes Integrator(C) Invokes Finisher(A) Invokes Integrator(C) Creates Finisher(A) Invokes Finisher(B) Invokes Integrator(A) Invokes Finisher(A) Invokes Finisher(C) Invokes Integrator(B) Creates Finisher(B) Invokes Integrator(C) Invokes Finisher(B) Invokes Integrator(A) Creates Finisher(C) Invokes Integrator(B) Invokes Finisher(C) Invokes Integrator(C) Invokes Finisher(A) Invokes Finisher(B) Invokes Finisher(C) ----- Java code -------------------- package org.example; import java.util.List; import java.util.concurrent.atomic.AtomicInteger; import java.util.function.BiConsumer; import java.util.function.Supplier; import java.util.stream.Collector; import java.util.stream.Collectors; import java.util.stream.Gatherer; import java.util.stream.Stream; import static java.lang.StringTemplate.STR; import static java.util.function.Function.identity; public class GatherEx { static <T> Gatherer<T, AtomicInteger, Indexed<T>> withIndex(String name) { return new WithIndexGatherer<>(name); } static class WithIndexGatherer<T> implements Gatherer<T, AtomicInteger, Indexed<T>> { final String name; public WithIndexGatherer(String name) { System.out.println(STR. "Creates WithIndex(\{ name })" ); this.name = name; } @Override public Supplier<AtomicInteger> initializer() { System.out.println(STR. "Creates Initializer(\{ name })" ); return () -> { System.out.println(STR. "Invokes Initializer(\{ name })" ); return new AtomicInteger(0); }; } @Override public Integrator<AtomicInteger, T, Indexed<T>> integrator() { System.out.println(STR. "Creates Integrator(\{ name })" ); return (state, element, downstream) -> { System.out.println(STR. "Invokes Integrator(\{ name })" ); return downstream.push(new Indexed<>(state.getAndIncrement(), element)); }; } @Override public BiConsumer<AtomicInteger, Downstream<? super Indexed<T>>> finisher() { System.out.println(STR. "Creates Finisher(\{ name })" ); return (_, _) -> { System.out.println(STR. "Invokes Finisher(\{ name })" ); Gatherer.super.finisher(); }; } } record Indexed<T>(int i, T data) { } public static void main(String[] args) { System.out.println("Using andThen"); System.out.println(Stream.of(1, 2, 3) .gather(withIndex("A") .andThen(withIndex("B")) .andThen(withIndex("C"))) .collect(createCollector())); System.out.println("Using separate gathers"); System.out.println(Stream.of(1, 2, 3) .gather(withIndex("A")) .gather(withIndex("B")) .gather(withIndex("C")) .collect(createCollector())); System.out.println("Using separate gathers with map(identity)"); System.out.println(Stream.of(1, 2, 3) .map(identity()) .gather(withIndex("A")) .map(identity()) .gather(withIndex("B")) .map(identity()) .gather(withIndex("C")) .map(identity()) .collect(createCollector())); } private static Collector<? super Indexed<?>, ?, List<Indexed<?>>> createCollector() { System.out.println("Calls Collectors.toList"); try { return Collectors.toList(); } finally { System.out.println("Finished Collectors.toList"); } } }
