Sure. This would be my first time contributing to Erlang/OTP, so I would definitely appreciate some guidance.
On Wednesday, 16 February 2022 at 16:42:09 UTC+2 José Valim wrote: > That's perfect. I believe the first step is to get those operations into > Erlang/OTP, likely under the file module. Would you like to send a PR for > that? I can probably provide some guidance around it. :) > > On Wed, Feb 16, 2022 at 11:49 AM Brendan Ball <[email protected]> > wrote: > >> I am looking at the Rust Cargo implementation since I know that works: >> https://github.com/rust-lang/cargo/blob/master/src/cargo/util/flock.rs >> It makes use of flock (https://man7.org/linux/man-pages/man2/flock.2.html) >> on unix and LockFileEx ( >> https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) >> >> on windows. >> I believe these OS APIs take care of removing file locks on process exit. >> Locks can be implemented on a best effort basis for >> platforms/environments that support it. E.g. it seems that Cargo doesn't >> implement locks on NFS filesystems. >> >> >> >> On Wednesday, 16 February 2022 at 12:09:07 UTC+2 José Valim wrote: >> >>> My biggest concern with a file lock is: what happens if the other >>> process terminates unexpectedly and does not remove the file lock? How can >>> we ensure this won't happen? >>> >>> On Thu, Jan 27, 2022 at 9:47 AM Brendan Ball <[email protected]> >>> wrote: >>> >>>> Hi >>>> >>>> We've been getting weird errors when building releases: >>>> ``` >>>> Compiling 42 files (.ex) >>>> ** (File.Error) could not remove files and directories recursively from >>>> "/home/brendan/dev/app/_build/prod/lib/shared_app/priv": file already >>>> exists >>>> (elixir 1.13.2) lib/file.ex:1292: File.rm_rf!/1 >>>> (mix 1.13.2) lib/mix/utils.ex:444: Mix.Utils.symlink_or_copy/3 >>>> (mix 1.13.2) lib/mix/project.ex:738: anonymous fn/5 in >>>> Mix.Project.build_structure/2 >>>> (elixir 1.13.2) lib/enum.ex:2396: >>>> Enum."-reduce/3-lists^foldl/2-0-"/3 >>>> (mix 1.13.2) lib/mix/project.ex:737: Mix.Project.build_structure/2 >>>> (mix 1.13.2) lib/mix/tasks/compile.all.ex:34: >>>> Mix.Tasks.Compile.All.run/1 >>>> (mix 1.13.2) lib/mix/task.ex:397: anonymous fn/3 in >>>> Mix.Task.run_task/3 >>>> (mix 1.13.2) lib/mix/project.ex:396: Mix.Project.in_project/4 >>>> ``` >>>> >>>> We build different releases concurrently after running a single `mix >>>> compile`. >>>> It turns out that the different release builds were not running in >>>> isolated environments, but shared the filesystem. This is a very weird and >>>> confusing error especially since most devs didn't know why it was breaking >>>> and didn't know that releases were being run in a shared environment. >>>> >>>> This seems like something that might happen fairly often in CI >>>> environments, so I really think it's in everyone's interest to improve >>>> this. >>>> Most good build systems/compilers implement some kind of file lock for >>>> operations that can't be done concurrently. I think implementing this >>>> would >>>> be a minimal acceptable solution. Ideally the file lock is used to wait >>>> until the operation can be performed, but even just an error saying you >>>> can't run operations in parallel would be better than what we have now. >>>> >>>> Thoughts? >>>> >>>> Kind regards >>>> Brendan Ball >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elixir-lang-core" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elixir-lang-core/8dfd467c-ec43-40de-bb2d-c9632d3de0can%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/elixir-lang-core/8dfd467c-ec43-40de-bb2d-c9632d3de0can%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/ef89c3ec-d175-4f3d-9638-47f24a4bf587n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elixir-lang-core/ef89c3ec-d175-4f3d-9638-47f24a4bf587n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/a386a8dd-fe9a-4e56-b514-e732befad342n%40googlegroups.com.
