Executable code and supporting data that doesn't change often, along with software and data that has an operational impact when inadvertently or maliciously altered, should be stored on media that is read-only or volatile (reverts to a known stable image on reboot).
We place a lot of faith in operating systems to enforce user privileges, and it is possible to mount storage partitions and volumes as read-only, but these protections are far too easily overridden. The data on a "secure" server can be altered from a console by booting from a Linux or DOS CD or a USB flash or external hard drive. In clusters and farms with fail-over and load balancing, access to the console of any one server can compromise secure data shared by the entire enterprise.
[ Stay abreast of security issues in InfoWorld's Security Adviser blog. ]
I have always marveled at the elaborate security and RAS mechanisms that IT is compelled to add to servers using inherently vulnerable, mutable software. Readily available hardware technology can make it difficult to impossible to alter such data as OS kernels, drivers, privileged loadable modules, and configuration parameters set at install time that comprises a server's sacred ground.
Why do these most sensitive files need to be mixed in with ordinary data when it's so easy to set them apart on media that can't be altered? And why do they need to be stored in multiple files at all? Once a configuration is solidified, a server can bootstrap from a read-only RAM image, like the ones created by laptops when they go into hibernation, that contains the kernel, drivers, and fixed configuration.
I'll tell you what brought this to mind. On a recent trip to an electronics store, I got sucked into an impulse buy: A 4GB SDHC USB flash card that cost $19.95. That's large enough to hold a 64-bit OS' privileged code, configuration data, and then some. The card is just about as fast as a hard drive, plenty fast enough to boot a server in a reasonable amount of time, and it's many times faster than optical. To make that storage accessible to my MacBook Pro, which doesn't have an SDHC slot, requires installing the card in a reader. The inexpensive flash reader I'm using is a simple, cheap USB device built from a single chip -- a microcontroller -- and a handful of discrete components. That got me thinking.
I'm experimenting now with a microcontroller from U.S. semiconductor manufacturer Microchip called the PIC32. This is a one-chip, 32-bit, MIPS architecture (the CPU used in Silicon Graphics workstations in the 1990s) RISC computer with pipelined execution that's capable of executing over 150 million 32-bit instructions per second when running at just 80MHz. That's paltry compared to even a cell phone, but this is a device that only requires a power supply to operate.
Along with the CPU core, the 100-pin chip, six of which fit on my fingernail, has 512KB of built-in flash memory (a 4GB address space and an external bus leave room for much more) and a pair of USB controllers. Microchip isn't Intel or AMD; there's no bragging about advanced fabrication processes. But the chip sells for about $3 in quantity, $9 if you want to buy just one, and it has two built-in USB controllers.
Using an open source library supplied by the vendor, the PIC32 connects to any computer as a USB storage device, meaning that you can boot a server from a bitstream supplied by a computer. It can perform encryption, log arbitrary information, intelligently alter boot data -- for example, disabling drivers for external ports or supplying a safe or maintenance mode boot image if it suspects the system is compromised or faulty -- all without high-level OS awareness or involvement and without the possibility of external tampering.
I have a pretty high standard for tamper-proof. You can seal a microcontroller, along with a 32GB MicroSDHC flash card, a lithium battery, and a piezo alarm, in a barnacle of plumber's putty (for example) that you can stick to the inside of your server chassis. If anyone snips the wires or tries to drill a hole in the package, or so much as opens the server case without an approved RFID, fingerprint, voice scan, key, password, or remote disarm command issued directly to the controller via Ethernet, the controller can scramble its memory, blow an on-chip fuse that renders the device useless, and blast a 130-decibel alarm in a matter of milliseconds. With very little effort, any attached external flash memory can be fried with a voltage spike or short, or literally fried by heating a length of wire. Pack some gunpowder in there to set off the smoke detector in the server room while you're at it.
An even simpler setup with a 50-cent, 8-bit microcontroller can make a parallel ATA hard drive non-writable, or useless unless conditions that you've defined are met. Imagine equipping the controller with an accelerometer, which is also extremely cheap, so that a hard drive will burn itself out if a pattern of motion is detected indicating that the PC, or just the drive, is being moved.
The gunpowder may be a bit much, and programming your own microcontroller to secure a server isn't a project you're likely to take on. In that same trip to the electronics store, I bought a $50 Wi-Fi router that can be flashed to run open source firmware, making it possible to alter or block LAN traffic on the fly in a manner that's opaque, and the device can "brick" itself, completely cutting off external access in response to an in-house threat. Security shouldn't kick in after the OS boots. That requires assuming that the OS is trustworthy, and practically all exploits are based on compromising the OS. Even just booting the OS from a CD or DVD that's sealed into an optical drive affords more protection than any software you can install on your hard drive or networked storage. If you can't change it, you can't hack it.