[
https://issues.apache.org/jira/browse/MRESOLVER-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696495#comment-17696495
]
Robert Muir commented on MRESOLVER-325:
---------------------------------------
[~michael-o] no problem. Sorry I don't understand the bigger picture, but happy
to help more if I can. I read a little on the issue and saw mentions of locking
and became a little afraid :)
As someone who hasn't looked at the code, but just the file structure, I'd
high-level naively try to do something like this for this artifact:
{noformat}
$ ls org/apache/maven/maven-core/3.1.1
maven-core-3.1.1.jar maven-core-3.1.1.jar.sha1 maven-core-3.1.1.pom
maven-core-3.1.1.pom.sha1 _remote.repositories
{noformat}
1. first create org/apache/maven/maven-core file tree. Fsync parent directories
of each new one i make when i do it. I'm assuming if this filetree is empty, it
won't cause harm at read time.
2. next create org/apache/maven/maven-core/.tmpXYZ123 temporary to stage my
files. I'm assuming these tempdirs can be filtered out by reads, so that no
uncommitted stuff is really "visible" to maven.
3. next write each one of my files (jars and poms and sha1s and
_remote.repositories) into org/apache/maven/maven-core/.tmpXYZ123
4. fsync all my files, including .tmpXYZ13 directory.
5. rename org/apache/maven/maven-core/.tmpXYZ123 to
org/apache/maven/maven-core/3.1.1. don't use REPLACE_EXISTING. Use ATOMIC_MOVE
and handle the exception if another process beat you to it.
6. fsync org/apache/maven/maven-core
If the maven code supports deleting artifacts (i dont know if this is allowed),
i'd ATOMIC_MOVE 3.1.1 to .tmpABC234, then fsync org/apache/maven/maven-core/,
then actually delete stuff. This way it is "atomically deleted".
The idea here would be, to avoid relying on filesystem locks at all, instead
only solely "publish" or "unpublish" data via move(ATOMIC_MOVE). And since
files are only written to temp directories, they can all be created with e.g.
{{Files.newOutputStream(path, CREATE_NEW)}} and so on.
> [REGRESSION] Suddenly seeing I/O errors under windows aborting the build
> ------------------------------------------------------------------------
>
> Key: MRESOLVER-325
> URL: https://issues.apache.org/jira/browse/MRESOLVER-325
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
> Affects Versions: 1.9.4
> Reporter: Christoph Läubrich
> Assignee: Michael Osipov
> Priority: Major
> Fix For: 1.9.6
>
>
> If one runs a build that otherwise works fine on 3.8.x with 3.9 we now get
> the following exception (full output can be found here
> https://github.com/eclipse-platform/eclipse.platform/actions/runs/4211467991/jobs/7309831666):
> {code:java}
> Error: 5.889 [ERROR] Internal error: java.io.UncheckedIOException:
> java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> -> [Help 1]
> org.apache.maven.InternalErrorException: Internal error:
> java.io.UncheckedIOException: java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:108)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:77)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:568)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> Caused by: java.io.UncheckedIOException: java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> at org.eclipse.aether.internal.impl.DefaultTrackingFileManager.update
> (DefaultTrackingFileManager.java:121)
> at
> org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addRepo
> (EnhancedLocalRepositoryManager.java:274)
> at
> org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addArtifact
> (EnhancedLocalRepositoryManager.java:252)
> at org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.add
> (EnhancedLocalRepositoryManager.java:225)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.evaluateDownloads
> (DefaultArtifactResolver.java:680)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads
> (DefaultArtifactResolver.java:592)
> at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:478)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts
> (DefaultArtifactResolver.java:278)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact
> (DefaultArtifactResolver.java:255)
> at
> org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact
> (DefaultRepositorySystem.java:296)
> at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:197)
> at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:413)
> at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve
> (LegacyRepositorySystem.java:332)
> at
> org.eclipse.tycho.osgi.configuration.MavenDependenciesResolverConfigurer.resolve
> (MavenDependenciesResolverConfigurer.java:104)
> at org.eclipse.tycho.core.shared.MavenDependenciesResolver.resolve
> (MavenDependenciesResolver.java:60)
> at org.eclipse.tycho.core.resolver.MavenTargetDefinitionContent.<init>
> (MavenTargetDefinitionContent.java:262)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolver.resolveContentWithExceptions
> (TargetDefinitionResolver.java:179)
> at org.eclipse.tycho.p2resolver.TargetDefinitionResolver.resolveContent
> (TargetDefinitionResolver.java:110)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolverService.resolveFromArguments
> (TargetDefinitionResolverService.java:90)
> at java.util.concurrent.ConcurrentHashMap.computeIfAbsent
> (ConcurrentHashMap.java:1708)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolverService.getTargetDefinitionContent
> (TargetDefinitionResolverService.java:65)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.resolveTargetDefinitions
> (TargetPlatformFactoryImpl.java:214)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:164)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:137)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:88)
> at
> org.eclipse.tycho.p2resolver.ReactorRepositoryManagerImpl.computePreliminaryTargetPlatform
> (ReactorRepositoryManagerImpl.java:61)
> at
> org.eclipse.tycho.p2resolver.P2DependencyResolver.computePreliminaryTargetPlatform
> (P2DependencyResolver.java:202)
> at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject
> (DefaultTychoResolver.java:130)
> at
> org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.lambda$resolveProjects$2
> (TychoMavenLifecycleParticipant.java:256)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept
> (ForEachOps.java:183)
> at java.util.stream.SpinedBuffer$1Splitr.forEachRemaining
> (SpinedBuffer.java:364)
> at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:509)
> at java.util.stream.ForEachOps$ForEachTask.compute (ForEachOps.java:290)
> at java.util.concurrent.CountedCompleter.exec (CountedCompleter.java:754)
> at java.util.concurrent.ForkJoinTask.doExec (ForkJoinTask.java:373)
> at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec
> (ForkJoinPool.java:1182)
> at java.util.concurrent.ForkJoinPool.scan (ForkJoinPool.java:1655)
> at java.util.concurrent.ForkJoinPool.runWorker (ForkJoinPool.java:1622)
> at java.util.concurrent.ForkJoinWorkerThread.run
> (ForkJoinWorkerThread.java:165)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
