Re: A parallel HttpClient sendAync question

[Daniel Fuchs]

Hi Max,

On 13/11/2018 02:35, Weijun Wang wrote:
I'm scanning a file and downloading links inside:
lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
      .map(l -> download(c, l))
      .forEach(f -> f.join());

CompletableFuture<HttpResponse<Path>> download(HttpClient c, URI link) {
     return c.sendAsync(HttpRequest.newBuilder(link).build(),
             HttpResponse.BodyHandlers.ofFile(Path.of(link.getPath())));
}

However, it seems the download is one by one and not parallel. I guess maybe 
map() is lazy and each request is only send when forEach() is called and the 
next one is only sent after join().

I believe this is true.

I can only collect the jobs into a list and then call join() on 
CompletableFuture.allOf(list). Is there a simpler way?

That's what I would use too. Though Chris Y.'s suggestion below
should also work:

On 13/11/2018 03:41, Chris Yin wrote: > lines.flapMap(x -> 
Stream.ofNullable(findURIFrom(x)))
    .map(l -> download(c, l))
    .collect(Collectors.toList())
    .forEach(f -> f.join());

since the collection should exhaust the stream before starting
the forEach loop.

best regards,

-- daniel