Hi everyone, One of the pieces of the puzzle required for model migrations was to get all the workers in the machine agent running under the new dependency engine framework. It will allow us to cleanly shut down workers so that a model to migrated can settle to a stable, "read-only" mode. This has been a major focus for team Onyx and Will.
If you're not sure what the dependency engine is all about, a good place to start is Will's excellent documentation in the source tree: https://github.com/juju/juju/blob/master/worker/dependency/doc.go The tl;dr is that the dependency engine provides a framework where workers are wrapped in a "manifold" which describes the resources the worker depends on, a function to start the worker and (optionally) a function which makes resources available to other manifolds/workers. In the context of the dependency engine framework, a "resource" is any value that is exported by a manifold for use by other manifolds. All the manifolds for an agent are registered with a dependency engine which then starts, stops and restarts workers based on the availability of their configured inputs (i.e. dependencies) and worker behaviour (i.e. when they exit and the errors they exit with). One nice property of the dependency engine framework is that it makes it very clear what a worker depends on and how it starts. The machine agent has become a complex hierarchy of runners and workers with sometimes non-obvious interactions between them. When the machine agent was younger and simpler the "nested runners" architecture was elegant and appropriate, but as more demands have been put on the machine agent it's become almost unmanageable. Fixing this wasn't the main driver for moving the machine agent to the dependency engine framework but it's been a side benefit. The machine agent dependency engine work has also meant that code to set up and start specific workers has been moved out of the machine agent itself to worker-specific packages, meaning everything to do with a worker is located together, and the machine agent's implementation is becoming clearer and more concise. Testing of workers and the agent has also become more straightforward and more robust. So where are things now? Before Onyx started doing this work on the machine agent, the unit agent was already completely converted to use the dependency engine for its workers - a unit agent runs far fewer workers so it was a good place to start. Much of the conversion work for the machine agent has already landed in master and there's several feature branches at varying levels of completion that cover most of the remaining work. Specifically the following areas have been converted: - a manifold which manages the API connection - most of the (many!) workers which depend on the API connection - the base workers which manage a State instance (state servers only) - some of the workers which depend on State What does this mean for Juju developers? If you're adding a new worker to the machine (or unit) agent you'll need to create a manifold for it and integrate it with the agent's dependency engine. Similarly if you're making changes to an existing worker, it's likely to have already been converted so you'll need to understand how manifolds and the dependency engine work. If you're not sure about something dependency engine related, please feel free to ask Will, Jesse or me. There's an initial learning curve but it's great once you get basic concepts straight. Aside from the docs (linked above), there's also lots of examples to study. Any worker package you find in the source with a manifold.go file has been converted. - Menno
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
