Given supervisors are also processes, they may also become bottlenecks.
While this is unlikely to happen to a Supervisor, since it is mostly
static, it can happen to a DynamicSupervisor.
We can address this by partitioning the dynamic supervisor. Imagine the
following dynamic supervisor:
defmodule MyApp.DynamicSupervisor do
use DynamicSupervisor
def start_link(opts) do
DynamicSupervisor.start_link(__MODULE__, arg, opts)
end
def init(_arg) do
DynamicSupervisor.init(strategy: :one_for_one)
end
end
In order to partition it, we can start 8 instances of said supervisor
inside a regular Supervisor, and then pick one partition at random when
starting a child. For example:
defmodule MyApp.Supervisor do
use Supervisor
@partitions 8
@name __MODULE__
def start_child(module, arg) do
i = :erlang.phash2(self(), @partitions) + 1
DynamicSupervisor.start_child(:"#{__MODULE__}#{i}", {module, arg})
end
def start_link do
Supervisor.start_link(__MODULE__, arg, name: @name)
end
def init(arg) do
children =
for i <- 1..@partitions do
name = :"#{__MODULE__}#{i}"
Supervisor.child_spec({MyApp.DynamicSupervisor, name: name}, id:
name)
end
Supervisor.init(children, strategy: :one_for_one)
end
end
I would like to make the above more convenient by introducing a :partitions
option to DynamicSupervisor.start_link. When given, the new option will
automatically start N dynamic supervisors under a supervisor, like above:
DynamicSupervisor.start_link(__MODULE__, :ok, partitions: 8, name: @name)
For now, the :name option will be required.
Now, when spawning child processes, you will use via tuples:
DynamicSupervisor.start_child({:via, DynamicSupervisor, {@name, self()}},
{module, arg})
The via tuple has the format {:via, DynamicSupervisor, {supervisor_name,
value_to_partition_on}}. Once invoked, it will take care of partitioning
based on the current process and dispatching it.
Overall, encapsulating the partitioning of DynamicSupervisor (and
consequently of Task.Supervisor) into an easy to use API can help to
vertically scale up applications that use those constructs.
## Open questions
One of the confusing aspects of the above is that the :name option no
longer reflects the name of the DynamicSupervisor but of the parent
Supervisor. One alternative is to *not* accept the :name option when
:partitions is given, instead we could have a :root_name option instead (or
something more appropriately named).
Implementation wise, we will store the available processes in ETS or using
a Registry (to be started alongside the Elixir application).
Feedback?
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JsSj1_qN9DMfHjn9ovta_hYoziRE%3D-E%2B%2BFjQcAm%3DqhZQ%40mail.gmail.com.
