Right. Now try running File.ls in some directories and see if that is enough to reproduce the issue. Then I would implement a C script that calls ls as a NIF, see if it is fast or not. And so on.
*José Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Mon, Oct 14, 2019 at 8:29 PM Michael St Clair <[email protected]> wrote: > What is the best way to experiment with that? As I am showing that using > the ls command is much faster (times below) > > On my Mac: > > iex(3)> fn -> Path.wildcard("lib/**/*.ex") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.095601 > > iex(4)> fn -> Path.wildcard("lib/**/*.ex") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.071896 > > > fn -> "ls lib/**/*.ex" |> String.to_charlist() |> :os.cmd() |> to_string() > |> String.split("\n") end |> :timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.034176 > > iex(6)> fn -> "ls lib/**/*.ex" |> String.to_charlist() |> :os.cmd() |> > to_string() |> String.split("\n") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.028481 > > > On our production instance running docker (elixir:1.9.1-alpine): > > iex(zipbooks@phoenix-360h)1> fn -> Path.wildcard("lib/**/*.ex") end |> : > timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.448345 > > iex(zipbooks@phoenix-360h)2> fn -> Path.wildcard("lib/**/*.ex") end |> : > timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.35429 > > > iex(zipbooks@phoenix-360h)7> fn -> "ls lib/**/*.ex" |> > String.to_charlist() |> :os.cmd() |> to_string() |> String.split("\n") end > |> :timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.00361 > > iex(zipbooks@phoenix-360h)8> fn -> "ls lib/**/*.ex" |> > String.to_charlist() |> :os.cmd() |> to_string() |> String.split("\n") end > |> :timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.003731 > > > Local docker container with mounted directories(elixir:1.9.1-alpine): > > iex(1)> fn -> Path.wildcard("lib/**/*.ex") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.881692 > > iex(2)> fn -> Path.wildcard("lib/**/*.ex") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.897629 > > > fn -> "ls lib/**/*.ex" |> String.to_charlist() |> :os.cmd() |> to_string() > |> String.split("\n") end |> :timer.tc |> elem(0) |> Kernel./(1_000_000) > > 0.031227 > > iex(4)> fn -> "ls lib/**/*.ex" |> String.to_charlist() |> :os.cmd() |> > to_string() |> String.split("\n") end |> :timer.tc |> elem(0) |> > Kernel./(1_000_000) > > 0.031241 > > On Monday, October 14, 2019 at 12:14:57 PM UTC-6, José Valim wrote: >> >> There is no way to patch out Mix.Utils to run your own code. The proper >> fix would be to figure out why the path traversal is slow on Elixir (it is >> implemented on top of filelib:wildcard) so we can fix it rather >> permanently. How long does "Path.wildcard("lib/**/*.ex")" take? Can you >> implement your own path traversal in Elixir? Is it faster than >> Path.wildcard? >> >> >> *José Valim* >> www.plataformatec.com.br >> Skype: jv.ptec >> Founder and Director of R&D >> >> >> On Mon, Oct 14, 2019 at 8:08 PM Michael St Clair <[email protected]> >> wrote: >> >>> I've been doing some digging around to see how to make phoenix run >>> faster in Docker. Without this plug the app runs just as fast as native >>> `plug(Phoenix.CodeReloader`. I think I have traced it down to this >>> https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/utils.ex#L237. >>> Running just that function explains most if not all of the extra load time >>> on my endpoints. However this code `"ls lib/**/*.ex" |> >>> String.to_charlist() |> :os.cmd() |> to_string() |> String.split("\n") |> >>> IO.inspect` appears to do the same thing and much faster. I'm just running >>> into the problem of how to override that Mix.Utils file to test it here >>> https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/compilers/elixir.ex#L36. >>> Any suggestions? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/b811d808-04a2-427e-9281-3d337f05de16%40googlegroups.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/b811d808-04a2-427e-9281-3d337f05de16%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/b46f9a3c-4f0a-479d-98df-a921ffeb65fc%40googlegroups.com > <https://groups.google.com/d/msgid/elixir-lang-core/b46f9a3c-4f0a-479d-98df-a921ffeb65fc%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K8G5py3P0eGxQ83C%2Bk0Ajwyjc4BGFDUf3dPt%3DS2rDGUQ%40mail.gmail.com.
