[
https://issues.apache.org/jira/browse/MRESOLVER-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624217#comment-17624217
]
ASF GitHub Bot commented on MRESOLVER-7:
----------------------------------------
cstamas commented on code in PR #178:
URL: https://github.com/apache/maven-resolver/pull/178#discussion_r1005272778
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/bf/BfDependencyCollector.java:
##########
@@ -365,6 +466,89 @@ else if ( descriptorResult == DataPool.NO_DESCRIPTOR )
return descriptorResult;
}
+ static class ParallelDescriptorResolver
+ {
+ final ExecutorService executorService;
+
+ /**
+ * Artifact ID -> Future of DescriptorResolutionResult
+ */
+ final Map<String, Future<DescriptorResolutionResult>> results = new
ConcurrentHashMap<>( 256 );
+ final Logger logger = LoggerFactory.getLogger( getClass() );
+
+ ParallelDescriptorResolver( RepositorySystemSession session )
+ {
+ this.executorService = getExecutorService( session );
+ }
+
+ void resolveDescriptors( Artifact artifact,
Callable<DescriptorResolutionResult> callable )
+ {
+ results.computeIfAbsent( ArtifactIdUtils.toId( artifact ),
+ key -> this.executorService.submit( callable ) );
+ }
+
+ void cacheVersionRangeDescriptor( Artifact artifact,
DescriptorResolutionResult resolutionResult )
+ {
+ results.computeIfAbsent( ArtifactIdUtils.toId( artifact ),
+ key -> ConcurrentUtils.constantFuture( resolutionResult )
);
+ }
+
+ Future<DescriptorResolutionResult> find( Artifact artifact )
+ {
+ return results.get( ArtifactIdUtils.toId( artifact ) );
+ }
+
+ void shutdown()
+ {
+ executorService.shutdown();
+ }
+
+ private ExecutorService getExecutorService( RepositorySystemSession
session )
+ {
+ int nThreads = ConfigUtils.getInteger( session, 5,
"maven.artifact.threads" );
Review Comment:
One more thing, but I will do this post merge: if you look at
BasicRepositoryConnector, it uses CONFIG_PROP_THREADS and just as "secondary"
property uses this "maven.artifact.threads" as a fallback. I will update this
post merge, to make them behave siimilarly; have it's own prop but fall back to
"maven.artifact.threas" as well
> Download dependency POMs in parallel in BF collector
> ----------------------------------------------------
>
> Key: MRESOLVER-7
> URL: https://issues.apache.org/jira/browse/MRESOLVER-7
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
> Affects Versions: Aether 1.0.2
> Reporter: Harald Wellmann
> Assignee: Tamas Cservenak
> Priority: Major
> Fix For: 1.9.0
>
> Attachments: resolve_deps.png, resolver.log
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> h3. Background
> When building a project with dependencies not yet available in the local
> repository, I noticed that Maven 3.3.9/Aether 1.0.2 first downloads the
> dependency POMs _sequentially_ and then proceeds downloading the dependency
> JARs with up to 5 threads _in parallel_.
> Due to this, when first building a project with a large number of
> dependencies, downloading a large number of small POMs may take a lot longer
> than downloading the much larger JARs, or even longer than building the
> project itself, especially when a repository manager is used which increases
> the download latency.
> h3. Enhancement
> Download POMs of (transitive) dependencies in parallel to significantly speed
> up initial builds of large projects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
