On 1/11/11 10:15 AM, "no-re...@cfengine.com" <no-re...@cfengine.com> wrote: > Forum: Cfengine Help > Subject: Re: Random variable persistence problem -- crontab randomization > Author: jgreer > Link to topic: https://cfengine.com/forum/read.php?3,20077,20080#msg-20080 > > I understand. You're right that a "random time" is not the goal, but > distribution of the job more evenly (and predictably) across our hosts. I do > not want the hosts changing their behavior all the time (i.e., one host runs > this job at 1:13, 2:07, 3:41...) - see requirement #1 in my original post.
I caution you to consider that, even with requirement #1, a "persistent pseudo-random" approach could still, persistently, cause overlap. So what if I stash the number if it is X 60% of the time? :) > What the splaytime does in Cfengine is to "hash" hosts over an interval of > time so that as few hosts collide as possible and times are distributed evenly > over the interval. That is not at all random, but it maps each host to a very > specific time (whose value you don't particularly care about). This is what I think you'd want to focus on for an optimal solution: Generating the cron's minute based upon hashing something consistent yet different on each host (hostname?) like SplayTime does. > I'm aware of splaytime and prefer it for scheduling because of its finer > granularity and persistence. In this specific case, though, it's not a good > solution, because of another business requirement: we schedule cf-execd to run > the agent only at Min00 with a splaytime of 10 minutes, and we do not want > agent runs to occur outside that window at the top of the hour. We want to > distribute the run of this cron job across the hour to prevent overloading a > server to which the cron job syncs files. This means you have N clients "phoning home" with TCP connections to a central resource? Warning: scalability nightmare. I realize we don't live in a perfect world and this may be a stop-gap or response to an immediate business need. Please just bear in mind the sooner you "fix" this the less impact will be caused when it comes tumbling down and has to be fixed later. Yes, I have deployed similar solutions... They worked well for 10's and 100's of hosts. Once we hit 1000's, it got interesting. It was also a lot more work (and stress) to go in and retrofit something people depended on rather than taking time in advance to think of ways to avoid breaking cfengine's pull model that would also have been easier to scale. > Cfengine commands are our preferred method of scheduling. While I'm trying to > solve a scheduling problem, scheduling is not the heart of the problem I'm > trying to solve - I'm interested to hear whether anyone has tried to solve the > problem of generating "disposable," pseudo-random data client-side, and > referring to that data in subsequent agent runs. It's not a problem I've run > into before, and don't know that I will again in practice, but was interested > whether the community had thoughts on the matter. You want to generate a pseudo-random number (CSCE won't let me say "random" in this context), use it to populate a template, then only re-generate it if the template changes: Commit template to source control Deploy on policy server (checksum changes) Clients copy template on changes, need_a_new_number becomes true If need_a_new_number: Generate new random_int and stash in /some/random_int Re-generate crontab using new template and random_int If !need_a_new_number, nothing happens If you don't care that the number changes when the template also changes, this should suffice. As others have pointed out, however, you'd need to be clever to ensure 'random' does what you want. You do not want a handful of values 'randomly' generated again and again. So the cfengine implementation seems relatively straightforward at first glance, with the real work being in generating useful numbers. Perhaps this could help with that? https://cfengine.org/pipermail/help-cfengine/2010-February/007659.html Good luck! -- "America is a melting pot. You know, where those on the bottom get burned, and the scum rises to the top." -- Utah Phillips _______________________________________________ Help-cfengine mailing list Help-cfengine@cfengine.org https://cfengine.org/mailman/listinfo/help-cfengine
