On Wed, 13 Jan 2010, Mark McCullough wrote: > On 2010 Jan 13, at 14:13, Richard Chycoski wrote: > >> Weekly is far too frequent for most environments. Unless a very >> particular threat has been exposed, this is overkill, and even getting >> quarterly patching schedules can be difficult. Rebooting weekly is out >> of the question on many of our servers. In some environments, rebooting >> more than once per year takes very-senior-management approval!
My reaction to this is that you probably need a better HA/clustering solution. In an enterprise where uptime is as critical as it seems to be, you should have failover pairs or clusters. These need regular testing anyway, and if you have them you should be able to make use of them to temporarily take one box offline and reboot it while running on the other box, then failover (in the middle of the night possibly) and repeat the process on the other box. Engineering a system to do a fast failover when the old box fails is tricky, doing a fast failover on command is very easy to do reliably. >> Also, if you require proper testing and burn-in of patches in your >> non-production environment first, (which is a very good idea if you want >> to keep your systems stable), then you will spend all of your time >> testing and installing patches, and always be several weeks behind >> anyway - testing patches includes letting them run for at least a week, >> and preferably a month, before confirming their efficacy and impact. >> >> Even our 'exposed' machines are patched quarterly-or-less - they are >> also configured to minimise external threats such that >> security-bug-of-the-week is less likely to be exploitable on these >> machines in the first place. Good security policies help minimise the >> risk of OS security bugs, but occasionally a patch does have to be >> rushed through to combat some known-deadly vulnerability > > First I want to agree completely with the above. > > The threat triage model is a necessary thing to do when looking at security > issues. > > Security patching isn't just applying every patch that claims to be security > right away. Analyze the threat and determine the risk of the exploit and the > risk of the patching causing a problem. Determine a deployment schedule. As > stated above, that deployment schedule needs to involve installing to each > environment in proper sequence. (We do development first, then test, then > prod). Low risk issues may simply be bundled into the regular, non-security > patching bundle. Remember that if it's not installed on the server it doesn't need to be patched. Make sure that your server builds actually remove anything that you don't need. Just because Redhat (or Debian, or Ubuntu...) install something by default doesn't mean that you actually need it. Along the same lines. for large applications (like Apache), don't enable features that you don't need. If possible compile your own so that the features you don't need aren't even compiled. This makes it likely that security patches for this application are not actually required for your installation. On the other hand, there are some things where there are a lot of changes from version to version, and the security effect of the changes are not always known (the linux kernel is a prime example of this, and it's not a matter of hiding things, frequently bugs are fixed by people just cleaning up code and it's later identified as a security issue). For this small number of things I like to patch as frequently as I can, but as others have noted, install it first on test machines and run it for a while. you may have something odd in your setup that wasn't in the setup that others used to test it before it got to you. > Sometimes (rarely) there will be a security alert that requires out of > cycle patching. That's what the triage model is for, to determine what > issues those are and to have a process for emergency deployment in those > cases. I've seen a couple issues that were so critical that required us > to not even take three days burn-in before deployment to production of > some fix over the past ten years (Solaris and RHEL). Rare, but they did > happen. Plan in advance for those cases. > > When looking at security announcements, I look at factors such as local > vs remote attack. What is the damage? DoS is a nuisance in most cases, > but not as crucial generally as a arbitrary command execution. A > surprisingly large number of attacks require local access, which lowers > the threat rating significantly. They still need to be processed, but I > have mitigating factors to limit exposure and to track who could have > triggered the exploit. > > The number of critical alerts I've seen has gone down over the years, > while the number of relatively low priority issues has gone up. All this is true as well. David Lang _______________________________________________ Discuss mailing list Discuss@lopsa.org http://lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
