On Sep 4, 2013, at 11:37 PM, Jürgen Schmidt wrote: > On 9/4/13 10:17 PM, Rob Weir wrote: >> On Wed, Sep 4, 2013 at 12:36 PM, janI <j...@apache.org> wrote: >>> Hi. >>> >>> We have had some longer discussions on different ML/IRC about how a >>> vm-admin should behave and which level of service we expect for our servers. >>> >>> We need new admins, so this is also a request for anyone interested to chip >>> in. >>> >>> We have had some unfortunate incidents on all 3 vm, of different nature, >>> which made me question if we as a community: >> >> I assume we have vms for Forums and for Wiki. But what is the 3rd one? >> >>> a) want servers, that are cared for professionally or by happening. >>> b) want to (are capable to) maintain the servers ourself. >>> c) are prepared to support a change that make a), b) possible. >>> >> >> I assume we want well-maintained servers that help us get our >> project-related tasks done, and also help serve our users. > > +1 that is the main goal, a working reliable infra structure that helps > to run our project. > > In the >> past, before you got involved, we were not very "proactive". It >> seemed like we just waited for something to break, and then tried to >> fix it. > > Exactly and doing it proactive is much better and in the end probably > less work. > >> >> I assume another goal is that we have several people helping with the >> admin, to share the work, avoid burnouts, cover for vacation, etc. > > And that is a very important goal, we need an environment where we can > at any time step in if somebody is not available. I now very well in the > same way as Jan how it is to be a bottleneck. Well it#s true for many > others of us in different areas. But if the servers are not running or > the services not available more people are affected faster > > >> >>> I have formulated some thoughts on how admins could work, but in general I >>> believe we should convince infra to take over the vm responsibility and >>> keep our well functioning forum/wiki admins. >>> >>> We have a vm-team in place, that was created with the purpose of not having >>> a single person as admin. I my opinion the team have not lived up to that >>> purpose but I am still thankful for the help I have received. >>> >>> Remarks the ideas below are my personal thought, which I have used during >>> the time where I maintained the servers: >>> >>> =========== >>> The server should at all times be maintained with the following priority: >>> 1) security (the backside of being popular is to have the attention of >>> people who want to gain merit by breaking our servers) >>> 2) stability (we have limited cpu/ram/disk so we must optimize) >>> 3) add user wishes (we already have stable systems, 1,2 are far more >>> important that enhancing the systems). >>> >> >> and maybe >> >> 3b) Try to evolve systems so users can implement their own wishes, in >> a way compatible with 1) and 2). For example, if routine logos and >> footers are synched to resources in the project's SVN tree, then any >> committer can update things.
I really like this idea. >> >>> Being an admin on a vm is a job that does not take soo much time, but >>> requires a lot of monitoring and communication (especially with infra). >>> >>> A good setup would be, 3 types of admin: >>> Each server will have an appointed "owner" (anchor-admin) >>> A number of persons have full sudo on a server (admin) >>> A number of persons can reboot/restart/work on po files (help-admin) >>> >>> === Anchor-admin responsibilities === >>> Anchor-admin is the "owner" of the vm and the prime contact to infra. >>> >>> Anchor-admin has the overall responsibility of the vm. >>> 1) help when receiving alerts >>> 2) keep informed on available patches, especial security related patches >>> 3) create/keep a maintenance plan >>> 4) coordinate changes external to vm (like dns) with infra >>> 5) participate in infra discussions relevant for the vm (e.g. certificates) >>> 6) monitor the vm regularly for resource usage >>> 7) secure that appl changes are implemented with relevant consensus >>> 8) discuss work with admin, with the goal that they should be able to take >>> over one day. >>> >>> These activities are expected to take 3-4 hours pr week, more in the >>> beginning and less later. The hour usage highly depend on the number and >>> level of admins. >>> >>> === Admin responsibilities === >>> Admins help the anchor admin with ongoing maintenance and have full sudo. >>> >>> All changes must be discussed and agreed with the anchor admin, no change >>> is so important that it cannot wait until discussed ! >>> >> >> We might also want an admin-...@openoffice.apache.org list and perhaps >> a private one as well to coordinate. Perhaps an admin-dev, but a private one is a not a good idea - the team should be on the infrastructure list and infra should know what is up. >> >>> Admins are expected to: >>> 1) help when receiving alerts >>> 2) stay informed with the vm configuration >>> including but not limited to: >>> - where are which configuration done, and stored (svn/backup) >>> - how are the apps. configured >>> - read and update runbook, if something is unclear >>> 3) participate in the regular maintenance >>> 4) coordinate all non-scheduled work with anchor-admin >>> >>> These activities are expected to take 1-2 hours pr week, more in the >>> beginning and less later. >>> >>> Admin does not need to be specialists, we all learn, but it is important >>> that the admin have motivation and time to learn. >>> >>> >>> === Help-admin responsibilities === >>> Help-admins are located in different timezones, so we have 24/7 coverage >>> and have limited sudo (only restart/reboot/handle po files). >>> >>> When a help-admin receives an alert mail, actions should be taken >>> 1) is the vm reachable via ssh, then login else escalate to admin/infra >>> 2) is the vm overloaded, or is apache/mysql not running >>> 3) restart the needed processes >>> 4) mail at least anchor-admin about with obervations and what was done. >>> >>> >>> === >>> remark the above are just my thoughts, there are a lot of other >>> possibilities. > > It sounds very reasonable to me and worth to work in this direction. I > see myself more in the role of a help-admin but I willing to learn more > to be a better admin. I agree that the plan is reasonable and professional. Regards, Dave > > Juergen > >>> >>> Lets hear your opinion? >>> >> >> It sounds like a good way to think of this. >> >> Regards, >> >> -Rob >> >>> rgds >>> jan I. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org >> For additional commands, e-mail: dev-h...@openoffice.apache.org >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > For additional commands, e-mail: dev-h...@openoffice.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
