Right. Sit down. This is the most demanding bit on the whole ruddy site. No worries .. 🙂 .. smile!
Linux is a flexible friend. We take a kernel, bung on bits and bobs and, tweaked, end up with a souped up server. The problem is, all too often, emphasis is placed on performance while security's left to a firewall, a few closed ports, and a large dose of wishful thinking.
Like a good guard dog, what we need is snarl and teeth. Not just at application level, but at the very heart of the system, its kernel. Let's go nuts.
Growling quietly with greater security
grsecurity from Brad “Spender” Spengler is a resource-light modular suite that patches a rack of Linux indiscretions. It's not exactly straightforward to set up but, once configured, it can be largely ignored, not least of all due to the learning capability of its user controls.
As well as guarding against kernel calamities, its PaX component protects against memory exploits, there are tight constraints for chroot environments and its advanced user access controls greatly assist system integrity. Here's a potted summary:
- Role Based Access Control (RBAC) system that auto-generates policies
- Change root (chroot) hardening
- Curbs to arbitrary code execution (stack smashing, heap corruption and so on)
- Prevention of arbitrary code execution in the kernel
- Randomization of the stack, library, and heap bases
- Kernel stack base randomization
- /tmp race prevention
- Protection against exploitable null-pointer kernel bugs
- Reduction of the risk of sensitive information being leaked by kernel bugs
- A restriction that allows a user to only view his or her processes
- Security alerts and extensive auditing
There's a lot to learn to make the most of grsecurity. Have some starting points:
Controlling user access with RBAC
Linux has the user-group-other permissions system that wpCop summed up in WordPress Files Users & Permissions. In geek-speak, this is called Discretionary Access Control (DAC) with file usage rights afforded on a discretionary basis. Users do something with a file depending on the file's ownership and the user's identity and permissions. The root superuser is exceptional with full rights.
This DAC security model is convenient, for sure, but the worry lingers that if a file is compromised it can be manipulated to do something nasty according to the privileged powers of its user which – yikes! – could be root.
Second-tier access control with RBAC
An extra authentication system can be overlaid to DAC to effectively cordon a process into doing only those things it absolutely needs to, regardless of its user's permissions.
grsecurity uses the Role Based Access Control (RBAC) model to do this. Users are assigned to groups that are granted access only to those files that they absolutely need. Now, if a file owned by some user is prompted to carry out a task, and if DAC says the user has the rights, up chirps RBAC to ensure the user also has group rights to the task. So that's two green lights rather than one.
Training the RBAC system with Gradm
So how do we implement this bells-whistles system without getting a headache? Gradm manages access control and combines a program called grlearn to observe the system, creating least privilege policies on the fly which we can subsequently tweak.
Memory protection with PaX
Excuse me for not detailing every grsecurity feature. There's simply too much to cover.
PaX though, must be excepted. Essentially, PaX thwarts many types of memory attack such as the buffer overflows mentioned in the Cop's Access and Authentication Server Issues. With the prevalence of this kind of assault this patch pack, while no guarantee, is a firm friend of security.
The multi-layered protection model
Extending upon our introduction to PaX and RBAC, we can better understand the value of grsecurity as a whole. If a hardening patch fails to deny an attack there are fallbacks to minimize damage, if not entirely negate the potential impact, to our system.
How to install grsecurity
As with most Linux packages there are various installation methods, depending on your distribution and on whether you are installing from repositories or are compiling using source files.
As with all wpCop's tutorials, here we'll example the installation for Debian-based Linux so, if for instance you're using another distribution such as RedHat-based CentOS, you'll need to swap the package management commands for those appropriate to your server setup. That tweaking should not be too big a deal.
Another quirk to allow for is if you use a Xen-based VPS system, which requires jumping through a few extra hoops and regardless of the Linux distro that's sat on top. wpCop sets out the procedure, either way.
Another option, if you run Debian or its fork Ubuntu, is to install grsecurity not by compiling the source files but instead by using a repository, and we'll look at that first.
Debian grsecurity from repositories
We'll come on to the regular installation method but, if you have Debian-based Linux, you can install grsecurity the easy way thanks to a helpful chap called Julien Tinnes. Assuming root, add the repository to your software sources list.
… For Ubuntu servers we do this:
… Whereas Debian types instead need to pull from a different repository:
Either way, we add the repository signature key like this:
And following the prompts, install the package:
You can find further configuration documentation here. But. Debian or not, if your server is a Xen-based VPS this won't work. Read on…
Compiling grsecurity into a kernel
The more regular option, and in wpCop's estimation the better albeit more convoluted method, is to compile a custom kernel, embedding grsecurity within. Maybe that sounds scary but it really is no big deal and, besides, with this option we can tweak grsecurity options more easily. The only downtime is a single reboot.
If you run a Xen-virtualized VPS there are extra hurdles. Don't sweat it, we cover those.
What we will do is to download a vanilla (essentially that means unmodified) kernel, grsecurity itself and its training tool, gradm. We'll compile the new kernel with the configuration of our pre-existing kernel, wrapping in grsecurity before booting into it. Then we'll install gradm.
Assuming root, we'll change into a suitable location and install some assistive packages:
Matching the kernel and grsecurity packages
The grsecurity patch must match the corresponding kernel version. Also, the gradm version must match the patch. Look for the latest corresponding versions at grsecurity's download page.

Noting grsecurity's Linux kernel reference
As of writing, for example, 2.2.0 refers to the latest patch and gradm. The patch also tells us what kernel it wants, grsecurity-2.2.0-2.6.32.15-201006271253.patch, the text in bold referring to the supported kernel series, such as 2.6, and the version, such as 2.6.32.15. Check that against the stable kernel downloads at kernel.org, as shown here:

Selecting a kernel compatible with the grsecurity version
Exporting the version numbers
We'll use the export command to make life easier. Replace the kernel series and the package versions with the latest matching ones and paste this at the command prompt:
Verifying the package downloads
Paste this, unedited, to download the kernel, its signature and to verify file integrity (wpCop looked at this package security stuff in Check Package Integrity: MD5 Checksums & GnuPG Signatures):
Now a similar palaver for grsecurity and gradm. Copy and paste again:
Patching the kernel
Unzip the kernel package, patch it with grsecurity and change into its folder:
Being rather cunning, we use the existing kernel configuration which your system already likes:
You'll be prompted with quite a few questions. Just hit Return for each.
Xen VPS configuration part 1
Xen won't like the new kernel. Bribe it! Open this:
And append the file with this lot:
Configuring the kernel
We'll open a menu interface to make life pretty:
To set your configuration, navigate the menu from Security options to grsecurity.
Use the space bar to select grsecurity, opening an extended menu. Scroll down and select Security level and make a choice between Low, Medium, High, and Custom.
For more detail refer to menuconfig's extensive Help and to the explanatory page here.
Make a choice by pressing the space bar and you will be returned to the grsecurity screen.
Kernel level chroot hardening
The premise of chroot is to segregate files so they don't interfere with the wider system. Originally intended not as a security technique but as a development tool, it's hardly surprising that the opportunity afforded by chroot for privilege separation has been pursued as a means to jail user accounts and their processes. So how does it work?
At the core of the file system we have /, or root, from which the directory tree stems. chroot works by creating a new folder, adding files, and telling them that's root. Hence, we have changed root for those files and, properly implemented, files in the new folder tree, and their non-superuser users, can't maraud beyond their root.
Properly implemented?
There is a slight security snag. As an example, if a user, like that guy trying to hack you, can chance superuser rights to a program then, oh dear, you're trumped by his get out of jail card. He can Advance To Go, the real root, bankrupting your server's integrity.
The point here is that chroot can provide a highly desirable additional layer of security but, poorly implemented or unmaintained, provides little more than a false sense of security. It's important to grasp this, so have a couple of guides:
grsecurity and chroot
grsecurity addresses issues in the above guides to give us super-tight chroot functionality. That doesn't mean that we need not be literate about and employ chroot best practices. Nonetheless, grsecurity's chroot options do give human error a safety net by toughening, for example, the chrooted SFTP area that we implemented in wpCop's Setup chroot SFTP Jail for WordPress using OpenSSH.
Enable the hardening in the menuconfig‘s grsecurity panel by clicking on Filesystem Protections and check-marking the chroot jail restrictions.
Using Sysctl support to maximize security settings
Also in the grsecurity menuconfig panel you'll see an option for Sysctl support. Enabling this allows you to disable any opted grsecurity features after only a reboot or, in other words, without having to recompile the kernel. This feature eases troubleshooting a clash when, for instance, an application fails due to an over-zealous protective measure.
You'd tend to check the Turn on features by default setting as well. If you don't, you'll have to run sysctl to enable features manually after each reboot. The procedure is outlined at the grsecurity site.
Sysctl support should be enabled only during the configuration phase, and ideally on a development server. To prevent someone tampering with your security settings, when you have a stable system, you must disable Sysctl support with this command:
Options galore
The options afforded by this first rate patchwork can be bewildering at first. Fortunately each feature has a Help function and the online docs are extensive. Have faith :).
With grsecurity tweaked-tastic, don't neglect the other kernel choices. You won't need everything, for sure, and unnecessary items add potential attack routes. Scan the menus, thank Help and clock up those browser search tabs.
When you're finished Exit your way out and save the configuration.
The kernel executable
This creates two Debian packages. It takes a while, so have a cup of tea:
And this installs the custom kernel for use after a reboot:
That also registers the new kernel with your bootloader which will make it the default. That is, unless you are running under Xen …
Xen VPS configuration part 2
VPS servers running on Xen won't recognize the new kernel without additional changes. We'll install a grub bootloader and configure that to work with the new kernel:
Now to make changes to the /boot/grub/menu.lst file to reflect our system. This first line assumes that your root device, where the root of your operating system is mounted, is /dev/xvda. Let's be sure. You can find yours by looking at the file /etc/fstab:
In this example, the root device is indeed /dev/xvda. If yours is different then swap the root= path here and paste the edited command into the terminal:
Next we need to specify the hard drive which, unless you have partitioned your drive, will be hd0. If you have partitioned, ascertain using fdisk -l and specify the boot partition, likely hd0,0. Either way, and maybe changing hd0, paste this:
Another tweak but, this time, only if you are running Ubuntu 9.04 (Jaunty) or greater. In that case, simply paste this into the command line:
Update the bootloader:
One final tweak for Xen-virtualized servers. In your VPS control panel go to the node configuration and ensure your kernel is set to pv-grub-x86_32 or pv-grub-x86_64, the settings allowing Xen guests to fire up a custom 32 or 64bit kernel.
Booting and checking the kernel
Boot into the new kernel:
When the box is back up, check for the new kernel, in our example 2.6.32.15-grsec:
If you want to you can also simulate some kernel hacking using overflow buff Peter Busser's PaXtest. Re-assuming root, download, extract, and install the package:
Run PaXtest in one of two modes, kiddie and blackhat:
And read the readme, why not?
Installing Gradm
Firstly, we need a couple of packages to assist the installation:
Swapping the gradm version number here, for the one you just used, run this lot:
If you need PAM (Pluggable Authentication Modules) to fine-tune user access, do this:
Otherwise, most of us will instead do this:
Either way, you will be prompted to create a unique password.
Now, if you haven't done so – with particular attention to the section Learning Mode – read the gradm manual at grsecurity's wiki. That will help you to set up grlearn, the superb gradm utility that automates the RBAC's least privileges policy setup. Here's a help too:
And that, you might just be pleased to hear, is grsecurity set up and onside.