> On Nov 13, 2018, at 11:41 AM, Chris Yin <xu.y....@oracle.com> wrote:
>
>>
>> On 13 Nov 2018, at 10:35 AM, Weijun Wang <weijun.w...@oracle.com> wrote:
>>
>> I'm scanning a file and downloading links inside:
>>
>> lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
>> .map(l -> download(c, l))
>> .forEach(f -> f.join());
>>
>> CompletableFuture<HttpResponse<Path>> download(HttpClient c, URI link) {
>> return c.sendAsync(HttpRequest.newBuilder(link).build(),
>> HttpResponse.BodyHandlers.ofFile(Path.of(link.getPath())));
>> }
>>
>> However, it seems the download is one by one and not parallel. I guess maybe
>> map() is lazy and each request is only send when forEach() is called and the
>> next one is only sent after join().
>>
>> I can only collect the jobs into a list and then call join() on
>> CompletableFuture.allOf(list). Is there a simpler way?
>
> Just guess your Stream (lines) is not parallel, so either use parallel stream
> or collect into list before join should work?
>
> lines.parallel().flapMap(x -> Stream.ofNullable(findURIFrom(x)))
> .map(l -> download(c, l))
> .forEach(f -> f.join());
Does not work. Still downloading sequentially.
>
> Or
>
> lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
> .map(l -> download(c, l))
> .collect(Collectors.toList())
> .forEach(f -> f.join());
>
> Regards,
> Chris
>
>>
>> Thanks
>> Max
