On Wed, 8 Nov 2023 16:26:56 GMT, Tagir F. Valeev <tval...@openjdk.org> wrote:
>> @amaembo pairMap should be straight-forward to implement. And regarding >> parallelization I don't think that will necessarily work since it would not >> be able to run incrementally (depth-first). > > Sorry, I'm not sure what do you mean by incrementally. But I've realized that > to implement this you may need to push to downstream during combining. E.g., > imagine the stream of numbers 1...1000, and you want to make > `parallel().gather(pairMap((left, right) -> > left+"->"+right)).collect(toList())`, creating a list of "1->2", "2->3", ..., > "999->1000" strings. Now, let's assume that the stream was split into four > parts during the parallelization: `1..250`, `251..500`, `501..750`, > `751..1000`. So you create downstream accumulators and push the pairs there > with integrator, like: > "1->2", ..., "249->250" into first toList() accumulator, > "251->252", ..., "499->500" into second toList() accumulator > "501->502", ..., "749->750" into third toList() accumulator > "751->752", ..., "999->1000" into fourth toList() accumulator > At this point we have four independent gatherer states, and they keep the > first and last element from the upstream. Now, you combine for example, first > and second gatherer states. The first one is [first=1, last=250], and the > second is [first=251, last=500]. The combined gatherer should be [first=1, > last=500] and at the same time during the combining it should push the > "250->251" object to the downstream, which will land into the first toList() > accumulator. > > Hopefully my explanation is clear. Will it be possible to extend the combiner > function and provide the Downstream there as well, so combiner can also work > as a finisher for the first combined chunk? Ah, now I see what you mean. In this specific case you might be able to create a combiner (which would also need a Downstream) such that you can add the missing combinations (the end of the "left" and the beginning of the "right"). But imagine something like `windowFixed(3)`, where the presence of 1 or 2 elements to the "left" will skew all the elements to the "right", potentially rippling through the entire stream. I think the Preview is a perfect place to see if the combiner would be made even better with a Downstream reference or not. Logically, the output of said Downstream would be placed *after* the "left"'s output, and *before* the "right"'s output (so in the middle of the two). But then someone might want to be able to inject elements outside of that order, but I guess that remains to be seen. Great thoughts, @amaembo 👍 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16420#discussion_r1386910345
