On 13 September 2013 18:52, Henrik Lindberg <[email protected]>wrote:
> Hi, > Ideas regarding a potential performance boost that can be gained by > performing batch processing of package installs/operations has been > floating around in the Puppet echo system for quite some time. > > There is a discussion (and a somewhat dated implementation/proposal) in > http://projects.puppetlabs.**com/issues/2198<http://projects.puppetlabs.com/issues/2198>which > is good background reading for this topic. > > In issue #2198 (if you skipped reading it ;-)), the idea is that Puppet > should have the feature to install a list of packages given by the user. > > It seems doable to generalize this idea and let puppet automatically > optimize package installs under certain conditions. Performing individual > package installs is quite expensive and even if the optimization > opportunities may not be extensive (e.g. say that 20% (number completely > made up) of packages could at least be paired with one other package) this > is still a worth while activity. > > To kick this off, we need to do some research and design. So, here is an > attempt to get this started by asking a bunch of questions. > > Under what conditions can two (or more) packages operations be batched? ------------------------------**------------------------------**----------- > In the relationship graph traversal code (puppet/graph/relationship_graph#traverse) resources that are ready to run are put on the queue. I think that it should be possible to batch every package resource currently on the queue when a package resource is encountered. It shouldn't matter which class it comes from or anything. > > What needs to be done to providers? > ------------------------------**----- > Clearly the capability to handle multiple requests must be implemented for > package managers that support this. What should the API look like? > > What needs to be done to the Package type? > ------------------------------**------------ > Is it all an ordering issue and handing off resources to the provider, or > do we need to do things to the Package type as well? > > Are there situations were it is of value to veto batching per resource? > (depending on how much optimization than can be deduced by looking at > resource-dependencies). > I would hope this is some sort of generic feature other types & providers can opt in to. For example database_user could create several users in a single connection without tearing it down between each one. -- Erik Dalén -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev. For more options, visit https://groups.google.com/groups/opt_out.
