I have a question related to a variety of similar common server
configurations. In many cases servers are remote; they have no
console, or the console is almost always unattended.
There are a number of settings that will make a system (even a
PC) much more robust under these circumstances. Currently I have
to hack them into the rc*, /etc, /proc and other places by hand
when I'm setting up such a system.
Here's a sample of some of the settings that I change/use:
Move the /usr/bin/logger to /bin (so it's on the root
fs before /usr is mounted).
Change the ifconfig and networking to enable networking
before fsck, mounting, etc.
Put sshd (and libz and libcrypto) on the root filesystem
and start it from a respawn (-D) option in /etc/inittab.
Likewise for syslogd and crond,
Tune the kernel for panic=60 (don't hang at a crashdump
forever), reboot after a small timeout.
Configure lilo and the kernel for serial console; add
dumb-terminal getty entries (if I have some sort of terminal
server in that rack/cabinet, sometimes I just crosslink the
serial ports from nearby boxes --- so I can get to the serial
console of any one box from its neighbor (a daisy chain)
[I'd like to figure out how to do that so each null modem
can be bidrectionally attached --- each as a monitoring getty
but doesn't emit an issue file until it recieves a break or
<Enter> or something].
Install software watchdog and loaddog utilities. (Sometimes,
I install hardware watchdog timer cards).
I go through rc scripts and change as many of them to use
logger -s for their errors and warnings as I can.
I do various other tweaks as they occur to me and has required by
particular systems.
Obviously (being a lazy bum) I'd like to have some of these merged
into Debian policies, or at least made compatible with policy and
automated into an installable package.
I'd like to see Debian GNU/Linux come to completely dominate the
colocated and telco-grade remote/isolated server markets. These
sorts of unattended systems (sometimes called "embedded" or "turnkey"
systems) are a huge marketshare among full-time professional system
administrators.
It's pretty common to want maximal robustness and secure remote
accessibility for such systems. Driving across town and "getting
into the cage" (dealing with Pinkerton's dweebs and fingerprint
or palm reader machines at the front door, etc) is a hassle. Linux
is far better for this than *cough* any Microsoft OS; and it's
almost as good as any other UNIX (better than any other PC UNIX).
But, it could be better.
What do I want?
I want to see a version of syslog that can be started off the
root fs before any other filesystems are started, even before
the network interfaces are configured. It would buffer its
messages in RAM (malloc'd) until it can open all of its
pipes, files, and sockets. It would have a signal handler that
would tell it when to try opening /var/log, and it's UDP/TCP sockets
tunnels, or whatever, et al (and/or it could try periodically until
those succeed). Maybe another signal (USR2?) could tell it to
give up on any dead sinks that were listed in /etc/syslog.conf
(posting a warning/error to any of the successfully opened sinks).
I want to see logger moved to the rootfs. I'd like to see all
rc scripts use it for all error and warning output (using it with -s
causes those to be tee'd to stderr, too).
I want to see sshd and its requisite libraries moved to the rootfs
(and optionally put in inittab with a respawn and the necessary
--- and quite recent! --- -D (don't fork/daemonize) option).
I'd like to see static network configuration (optionally) resequenced
so that it occurs before fsck/mounting.
I'd love to see a (tricky) option that would run the rc scripts
under a detached screen session (if screen was installed) so that
any sort of rc script "hang at fsck" type conditions could be
handled remotely (assuming we got far enough to get sshd and screen
started --- I'd ssh into the "down" system, re-attach to the screen
session and handle the problem interactively). Of course we'd like
for all scripts to have and ABSOLUTELY UNATTENDED, DON'T STOP FOR
ANYTHING, robustness --- but we might not ever quite get there ---
so a run under screen hack might help for that last 1%.
I'd like to see an installable package that would configure the
kernel, bootloader, and inittab for serial console and login.
I'd like to see a 'task-unattended-server' that would include
software watchdog and panic= settings.
I'd like to see other ideas for the things I've missed.
I'd like a winning lottery ticket in that California lottery,
too. (Think of what SPI and FSF could do with little portions
of that!).
