Lol, that is so creative. I never would have thought to use
anonymous classes on the java. util. function interfaces lol.
That's like choosing the horse when you have a motorcycle right
in front of you. But lo and behold, it genuinely is the
Lol, that is so creative.
I never would have thought to use anonymous classes on the
java.util.function interfaces lol. That's like choosing the horse
when you have a motorcycle right in front of you. But lo and
behold, it genuinely is the tersest way forward using normal Java.
How did you come up with this? lol
Anyways, creativeness aside, this one had about the same
performance as the Gatherer that I came up with. I am wondering
if there might be too much hand off, or something performance
wise that I might be doing wrong here. Long story short is that,
I want to gather the files as fast as possible, but I don't see a
way to get the full extent of the parallelism out without keeping
it all on one pipeline. At the end of the day, the solution that
you made seems to be making all of these instances of Stream over
and over again, and I worry if that might be making things slower
than needed. The instance of Gatherer I made doesn't have this
problem, but that's because it has the opposite problem of not
being able to parallelize easily. Yours looks like parallelizes
easily enough, but also does a lot of work to make that happen.
Maybe this is imaginative of me, but I am thinking of some sort
of Collection type, which doesn't suffer from
ConcurrentModificationException, where the Stream can just keep
pulling from it until it gets told that no more elements are
coming through, and it can pull and process the elements from
that Collection type in parallel. This is my dream object, if you
will. You just keep appending to the end of it as you come across
directories, and you keep pulling from the start of it until the
collection object finally becomes empty, in which case, you are
finally done.
Does any of that make sense to you?
On Wed, Aug 26, 2026, 11:26 AM Viktor Klang
<[email protected]> wrote:
Hello David,
You mean something like this?
void main() throws Exception {
final Path root = Path.of(System.getProperty("user.home"));
IO.println(root);
IO.println(root.toAbsolutePath());
Stream
.of(root)
.flatMap(new Function<Path, Stream<Path>>() {
@Override public Stream<Path> apply(Path p) {
if (Files.isRegularFile(p)) {
return Stream.of(p);
} else {
try {
return Files.list(p).flatMap(this::apply);
} catch (IOException _) {
return Stream.empty();
}
}
}
})
.limit(5)
.forEach(IO::println);
}
On 2026-08-26 16:24, David Alayachew wrote:
> Hello Viktor and Rémi,
>
> First off, sorry for the horrifically delayed response.
Juggling disasters
> and emergencies.
>
> Rémi, thanks for the example with mapMulti. That has been
my temporary
> workaround for now, and while it is still not ideal, it is
better than
> where I was before.
>
> Viktor, sure, here is a simple example -- traversing a
directory tree, and
> only passing the files down the stream.
>
> I actually made a post on StackOverflow --
>
https://softwareengineering.stackexchange.com/questions/461442/
<https://urldefense.com/v3/__https://softwareengineering.stackexchange.com/questions/461442/__;!!ACWV5N9M2RV99hQ!JsL3fQJ1ult_TN0D6BCiRDGewOWQs2G_97ksDE0txV_GsNtGvjC1XLvjm8-rVEgGNQKIOX_5Xz7VRezKpGpwi9bbnPg$>
>
> But anyways, when attempting to do this with streams, this
is where I
> started.
>
>
> import module java.base;
>
> void main() throws Exception
> {
>
> final Path root = Path.of(System.getProperty("user.home"));
>
> IO.println(root);
> IO.println(root.toAbsolutePath());
>
> Stream
> .of(root)
> .mapMulti(this::recursiveDescent)
> .limit(5)
> .forEach(IO::println)
> ;
>
> }
>
> private void recursiveDescent(final Path rootPath, final
Consumer<Path>
> downstream)
> {
>
> final Stack<Path> stack = new Stack<>();
> stack.push(rootPath);
>
> while (!stack.empty())
> {
>
> final Path path = stack.pop();
>
> if (Files.isRegularFile(path))
> {
>
> downstream.accept(path);
>
> }
>
> else
> {
>
> try (final Stream<Path> folderContents =
Files.list(path))
> {
>
> folderContents.forEach(stack::push);
>
> }
>
> catch (final Exception exception)
> {
>
> throw new IllegalStateException("Failed for "
+ path,
> exception);
>
> }
>
> }
>
> }
>
> }
>
> But this has at least 2 major downsides.
>
> 1 - This is not easy to turn parallel (comparatively).
>
> 2 - This does not short-circuit when the downstream no
longer accepts
> elements.
>
> Ok, I can at least solve problem 2 by becoming a Gatherer
instead.
>
> Here is my gatherer attempt.
>
>
> import module java.base;
>
> void main() throws Exception
> {
>
> final Path root = Path.of(System.getProperty("user.home"));
>
> IO.println(root);
> IO.println(root.toAbsolutePath());
>
> final Gatherer<Path, Stack<Path>, Path> gatherer =
> Gatherer
> .of
> (
> Stack<Path>::new,
> (stack, rootPath, downstream) ->
> {
>
> stack.push(rootPath);
>
> while (!stack.isEmpty())
> {
>
> final Path path = stack.pop();
>
> if (Files.isRegularFile(path))
> {
>
> final boolean acceptingMoreElements =
> downstream.push(path);
>
> if (!acceptingMoreElements)
> {
>
> return false;
>
> }
>
> }
>
> else
> {
>
> try (final Stream<Path> folderContents =
> Files.list(path))
> {
>
> folderContents.forEach(stack::push);
>
> }
>
> catch (final Exception exception)
> {
>
> throw new
IllegalStateException("Failed for " +
> path, exception);
>
> }
>
> }
> }
>
> return true;
>
> },
> (s1, s2) ->
> {
>
> s1.addAll(s2);
> return s1;
>
> },
> (stack, downstream) ->
> {
>
> for (final Path path : stack)
> {
>
> if (!downstream.push(path))
> {
>
> return;
>
> }
>
> }
>
> }
> )
> ;
>
> Stream
> .of(root)
> .gather(gatherer)
> .limit(5)
> .forEach(IO::println)
> ;
>
> }
>
> So, problem 2 is solved, but problem 1 is not really. Sure,
I could turn my
> stream parallel, but the actual meat of the processing is
sequential when
> it really doesn't need to be.
>
> Let me know if this makes more sense. And sorry, you might
have to scroll
> to read earlier emails in this thread to get the context. I
know it was
> several months back.
>
>
> On Fri, Nov 14, 2025 at 5:42 AM Remi Forax
<[email protected]> wrote:
>
>> Hi David,
>> You can always transform an imperative code to a stream by
pushing the
>> element through a consumer.
>> Internally, a stream uses a push iterator (see
>> Spliterator.tryAdvance(consumer)).
>>
>> As a silly example, this is a way to write fibonacci (the
recursive form)
>> with a stream right in the middle.
>>
>>
>> static void fibo(int n, IntConsumer consumer) {
>> if (n < 2) {
>> consumer.accept(n);
>> return;
>> }
>> var result = Stream.of("")
>> .mapMultiToInt((_, consumer2) -> {
>> fibo(n - 1, consumer2);
>> fibo(n - 2, consumer2);
>> })
>> .sum();
>> consumer.accept(result);
>> }
>>
>> static void main() {
>> fibo(7, IO::println);
>> }
>>
>>
>> Here, I use mapMulti() to convert the imperative code to a
Stream
>> (there is no factory method on Stream that takes a
consumer of consumer).
>>
>> If you also want to short-circuit, you can use a gatherer
instead of
>> mapMulti but short-circuiting the recursive code will
require to use an
>> exception as control flow (it will not be pretty).
>>
>> regards,
>> Rémi
>>
>>
>> ------------------------------
>>
>> *From: *"David Alayachew" <[email protected]>
>> *To: *"core-libs-dev" <[email protected]>
>> *Sent: *Tuesday, November 11, 2025 4:36:29 AM
>> *Subject: *Difficulties of recursion with Streams
>>
>> Hello @core-libs-dev <[email protected]>,
>>
>> When working with streams, I often run into situations
where I have to
>> "demote" back to imperative code because I am trying to
solve a problem
>> best solved by recursion.
>>
>> Consider the common use case of cycling through
permutations to find all
>> permutations that satisfy some condition. With recursion,
the answer is
>> incredibly simple -- just grab an element from the set,
then call the
>> recursive method with a copy of the set minus the grabbed
element. Once you
>> reach the empty set, you've reached your terminal condition.
>>
>> Use cases like that are not only incredibly common, but
usually,
>> embarrassingly parallel. The example above of cycling
through permutations
>> is only a few lines of imperative code, but I struggle to
imagine how I
>> would do this with Streams.
>>
>> I guess let me start by asking -- are there any good ways
currently to
>> accomplish the above permutation example with Streams? And
if not, should
>> there be?
>>
>> Thank you for your time and consideration.
>> David Alayachew
>>
>>
--
Cheers,
√
Viktor Klang
Software Architect, Java Platform Group
Oracle
