Wednesday, June 13, 2012

Setting up a network-based, centralized home directory as a service

As I mentioned in my last post, I just built out a dedicated server at home, and I'm migrating a bunch of services to it. One reason I build this server in the first place was so that I could set up roaming home directories for all my Linux PCs in the house.

I tried a few configurations before landing on the one described here. I had problems where Ubuntu 12.04 would fail to boot if the system failed to mount the network drive properly when the mount was in the fstab. Ubuntu 10.04 would error on boot, but give you the opportunity to skip the mount and continue booting. However, if the share was mounted to /home on the client machine, logging into a desktop environment would fail on account of none of the config files being available. Even when the mount worked properly, having it mounted at /home caused performance problems. Relying on the network for processes like booting and logging into a desktop ended up being a deal-breaker for me having automounted network home directories.

But I did settle on a solution that works well. In brief, I servicized the mounting and unmounting of the network drive, and the control script I wrote for it also places netdrive symbolic links in each non-system user's home directory if a directory on the share exists in their name. Here's how it's done.

There are two parts to this configuration - a server and some clients. Let's set up the server.

My server is running Debian Squeeze, but these instructions should translate well to other distributions. First, install the NFS server:

sudo apt-get install nfs-kernel-server

Then set the config to share the /home directory over the network. Add this to the end of /etc/exports:

/home   192.168.0.0/255.255.255.0(rw,sync,fsid=0,no_subtree_check)

You should modify that to suit your specific needs. /home is the directory on the server to share. 192.168.0.0/255.255.255.0 means that the only systems who are allowed to access it will be on the 192.168.0.x subnet, which, for me, means everything behind my router. This share won't be accessible to anyone on the public side of my router. Finally, the (rw,sync,fsid=0,no_subtree_check) part are options that say, respectively, "Mount it read/write so users can save files here," "Write files to disk synchronously so there aren't sudden unmounting problems," "If the NFS server is NFS4, treat this directory as the root of all shared directories," and "Speed things up by allowing all subdirectories of the share to be accessed."

Restart the NFS server:

sudo service nfs-kernel-server restart

If you encounter problems, check /var/log/daemon.log for something to Google with.

I had firewall problems here. Mounting an NFS share on a remote server uses lots of different ports, and I was unable to identify them all. I ended up adding a rule to my firewall to let anything behind my router (that 192.168.0.x subnet) access any ports on the server. For me, this isn't a security issue. You should consider your situation and make that decision for yourself. At any rate, here's my command to make that firewall rule:

sudo ufw allow from 192.168.0.0/24 to any

Now get onto a client machine and try to mount it manually just to be sure it works:

sudo mount -t nfs -o proto=tcp,port=2049 192.168.0.100:/home /mnt

If you get no errors, try...

ls /mnt

...and check if what you see is what you expect. If so, unmount it:

sudo umount /mnt

Assuming that everything works, it's time to set up a control script. The script below is hardcoded to mount the NFS share to /mnt/netdrive. You should probably read through it and understand it before running it to be sure it won't interfere with things on your system. If it's going to work for you, paste it into /etc/init.d/netdrive or wherever else you decide.

safe_start () {

    if [ ! -d /mnt/netdrive ]; then
        mkdir /mnt/netdrive
    fi

    if grep -qs '/mnt/netdrive' /proc/mounts; then
        echo "Something is already mounted at /mnt/netdrive [ FAIL ]"
        exit 1
    else
        mount -t nfs -o proto=tcp,port=2049 192.168.0.100:/home /mnt/netdrive
    fi

    for u in `grep '/home' /etc/passwd | cut -d: -f1`; do
        if [ -d /mnt/netdrive/$u ]; then
            HOME=`grep "$u" /etc/passwd | cut -d: -f6`
            if [ -e $HOME/netdrive ]; then
                rm $HOME/netdrive
            fi
            ln -s /mnt/netdrive/$u $HOME/netdrive
        fi
    done
}

safe_unmount () {
    FILES_IN_USE=`lsof | grep '/mnt/netdrive' | awk '{print $9}'`
    if [ ! "$FILES_IN_USE" = "" ]; then
        echo "The following files are located on the netdrive and are still in use:"
        echo $FILES_IN_USE
        exit 1
    fi

    unmount
}

unmount () {
    umount /mnt/netdrive
    for u in `ls /home`; do
        if [ -d /home/$u/netdrive ]; then
            rm $u/netdrive;
        fi
    done
}

show_status () {
    if grep -qs '/mnt/netdrive' /proc/mounts; then
        echo "Netdrive is mounted."
    else
        echo "Netdrive is not mounted."
    fi
}

show_help () {
    echo "This script accepts the following commands: start, stop, forcestop, status, help"
}

case $1 in
    "start" )
        safe_start ;;
    "stop" )
        safe_unmount ;;
    "forcestop" )
        unmount ;;
    "status" )
        show_status ;;
    * )
        show_help ;;
esac

exit 0

Apply permissions:

sudo chmod 755 /etc/init.d/netdrive

Now you can run...

sudo /etc/init.d/netdrive start

...to mount it, or give it stop instead to stop it safely. The script will refuse to unmount the drive if it has files open still. If you want to override that check, use forcestop. status will tell you if the network drive is mounted or not. help will give you a basic usage message. When mounting, this script will also place the netdrive symlink in users' home directories, so, for example, /home/ryan/netdrive is mapped to /mnt/netdrive/ryan.

Now, I still wanted my drive to mount on boot. So I threw a symlink in the startup:

sudo ln -s /etc/init.d/netdrive /etc/rc2.d/S10netdrive

That's my Ubuntu 12.04 machine. I can't remember at what point in the startup procedure I placed in on my Ubuntu 10.04 box, but it's somewhat discretionary anyway.

So reboot your client and make sure you boot with a working netdrive symlink in your home directory. And done!

Monday, June 11, 2012

Setting up a headless Calibre server

My wife and I like to read books on the go, and we use an Android app called Aldiko (it's great) to read epubs. I have loads of epubs on my computer, and I have traditionally used Calibre's server feature to share them with our phones. In the past, I've kind of clobbered together things on an Ubuntu box that also served as my main desktop computer. That worked well in the past, but I've recently set up a headless Debian Squeeze server, and I decided to migrate my Calibre server there. I had some setbacks, and thought it would be worth it to document the process here, and how I got things to work successfully.

My server runs headless, and that's the first problem with Calibre. The version of it in the Debian Squeeze repositories doesn't have a standalone server mode that is configurable without using the graphical UI. I can't have that, so I decided to get newer binaries installed. Ordinarily, I wouldn't recommend this, especially on a system like Debian whose community prides itself on the stability of their software packages, but this is a functional necessity, so I'll take it. This command, ripped straight from the official Calibre website, worked for me:

sudo python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main()"

When prompted, I told it to install to ''/opt/calibre'' so it wouldn't conflict with any system libs or binaries.

That means that ''/opt/calibre/calibre-server'' is the standalone daemon for serving e-books.

I wanted to servicize it, so I wrote this init script and placed it at ''/etc/init.d/calibre-server'':

#!/bin/bash

CALIBRE_LIBRARY_PATH="/home/shared/Calibre Library"
PIDFILE=/tmp/calibre-server.pid
USER=calibre
PORT=8081

start() {
        echo "Starting Calibre server..."
        su -c "calibre-server --with-library=\"$CALIBRE_LIBRARY_PATH\" -p $PORT --pidfile=$PIDFILE --daemonize" & 
        if [ $? -ne 0 ]; then
                echo "Could not start calibre-server."
        fi
}

stop() {
        echo "Stopping Calibre server..."
        if [ -e $PIDFILE ]; then
                read PID < $PIDFILE
                ps aux | grep "$PID" | grep 'calibre-server' > /dev/null
                RUNNING=$?
                if [ $RUNNING -eq 0 ]; then
                        kill $PID
                        if [ $? -eq 0 ]; then
                                rm $PIDFILE
                        fi
                else
                        echo "Could not find a calibre-server process with PID $PID."
                fi
        else
                echo "Could not find pidfile: $PIDFILE"
        fi
}

restart() {
        stop
        start
}

status() {
        if [ -e $PIDFILE ]; then
                read PID < $PIDFILE
                echo "calibre-server is running with PID $PID."
        else
                echo "calibre-server is not running."
        fi
}

unknown() {
        echo "Unrecognized command: $1"
        echo "Try one of the following: (start|stop|restart|status)"
}

case $1 in
        start ) 
                start
                ;;
        stop )
                stop
                ;;
        restart )
                restart
                ;;
        status )
                status
                ;;
        * )
                unknown
                ;;
esac

You can change the variables at the top to run the server differently. Once this is given execute permissions, you can start the server with:

/etc/init.d/calibre-server start

And stopped with

/etc/init.d/calibre-server stop

It looks like the ''service'' command works, too:

service calibre-server start

I know there are some other problems I need to work out (how to import new books, for example), but this seems like a good start. Anyone have any tweaks or additions to note?

Wednesday, September 21, 2011

Apache segfaults when PHP runs LDAP functions

I've been spending some time at work compiling a web stack to update the old, unsupported, and unmaintained Solaris webstack/coolstack software. It consists of the latest stable releases of Apache's httpd, MySQL, PHP, Python, and phpMyAdmin, compiled from source, to include all of their dependencies. One specific requirement here is that the various LDAP modules work. Many of our customers use this to authenticate users against a master directory on the network. I've encountered a problem.

I have a test script that pulls my own user information from our LDAP server. If I run this through the CLI PHP parser, it successfully and correctly gathers my information. If I hit that page up through Apache, httpd throws a segmentation fault. Even if I set Apache's LogLevel directive to 'debug', I get no information aobut *why* it segfaults. The only relevant output is as follows:

[Wed Sep 21 13:23:06 2011] [notice] child pid 6928 exit signal Segmentation fault (11) [Wed Sep 21 13:23:06 2011] [info] removed PID file /opt/utsawebstack/apache2/logs/httpd.pid (pid=6923) [Wed Sep 21 13:23:06 2011] [notice] caught SIGTERM, shutting down

The segfault *only* happens when LDAP functions are called from PHP. ldap_connect(), for example, fails. Other pages render fine through Apache.

Google hasn't been much help. The closest I've come to a solution is a patch for PHP version 4.4.4 — a version of PHP known to be buggy and unusable — and the patch no longer applies.

For the sake of detail, here are the configure strings for the relevant software (taken from a working build script, hence the variable names):

  • Berkeley DB (a dependency of OpenLDAP):
  • ../dist/configure --prefix=$BUILDBASE/lib
  • OpenLDAP (client only):
  • ./configure --prefix=$BUILDBASE/lib/ --disable-slapd --disable-slurpd
  • httpd:
  • ./configure --prefix=$BUILDBASE/apache2 --with-ldap --enable-authnz-ldap=shared --enable-ldap=shared --enable-headers=shared --enable-mime-magic=shared --enable-proxy=shared --enable-rewrite=shared --enable-mods-shared --enable-ssl=shared --with-z=$BUILDBASE/lib
  • PHP
  • ./configure --prefix=$BUILDBASE/php --with-apxs2=$BUILDBASE/apache2/bin/apxs --with-zlib=$BUILDBASE/lib --with-curl=$BUILDBASE/lib --with-iconv=$BUILDBASE/lib --enable-calendar --with-mysql=mysqlnd --with-mysqli=mysqlnd --enable-sockets --enable-zip --with-pear -with-mcrypt=$BUILDBASE/lib --enable-mbstring --with-ldap=$BUILDBASE/lib

Level of Difficulty: Solaris 10

Other Things I Know:

  • It's probably not a compiler problem. I'm using the latest version of make to do all of the compiling, and it's using GCC 3.4.3. And besides, most of this code requires GCC. Using the Sun/Oracle compiler would likely cause problems and failure.
  • It's not an architecture thing. I have compiled this and reproduced the problem on i386 and sparc hardware.
  • Since PHP and Apache are both compiled against the version of OpenLDAP that gets compiled first (see config string above), both are heavily reliant upon the ldap.conf file associated with it. Perhaps this is obvious to most, but I had wrongly assumed for some reason that Apache was the only thing using that. For what it's worth, that file is properly written.

Who has any idea what's going on here? Anybody?

FOUND A SOLUTION

The problem basically boils down to this:

I wrongly assumed that the --with-ldap configure argument would accept a directory to look in for the OpenLDAP libs (f/ex, --with-ldap=/opt/webstack/lib). It does not. Instead, I needed to throw in the --with-ldap argument with no directory attached to it along with --with-ldap-libs=/my/openldap/lib/dir and --with-ldap-include=/my/openldap/include/dir. Problem solved.

Sunday, January 23, 2011

Setting up public key authentication for SSH

If you're like me, you remote into a handful of servers using SSH all the time. The process is fairly simple:

  1. Get to a terminal
  2. ssh username@hostname
  3. Type password
  4. Get to work

No, it's not terribly difficult, but when you have to type that password fifty times per day, you begin to realize that it's time-consuming and repetitive. And there happens to be a way to eliminate that step from the process.

The SSH protocol supports authentication by public keys, and setting this up is a trivial matter. The configuration process goes something like this:

  1. Generate a key for your client system
  2. Put it on the server

One prerequisite: your SSH server must have public key authentication enabled. This is usually the case by default, but if you want to check, you can look in your /etc/ssh/sshd_config file. Try this:

grep 'PubkeyAuth' /etc/ssh/sshd_config

If the output is

PubkeyAuthentication yes

then you're safe to continue. Otherwise, you'll have to make the change in that file and restart the SSH server. There are a few ways to accomplish this, and I won't go into them here because it's not the point.

The point is that once this is ready, you can create your RSA key on the client machine. This is the first step toward getting this done. On the client machine, run

ssh-keygen -t rsa

You'll be asked for a filename. Just press enter to accept the default, which is probably ~/.ssh/id_rsa

You'll also be asked for a passphrase. You can use this optionally. If the point is to eliminate having to enter a password with every SSH connection, it's best to supply no passphrase. You'll also be asked to confirm it.

When done, you'll get a printout of your fingerprint and you'll return to a prompt.

Now we need to check this file's permissions. We don't want any other users to be able to read this pubkey file lest they compromise your authentication.

chmod 600 ~/.ssh/id_rsa.pub

EDIT: Commentor FKereki rightly points out a simpler and better way to accomplish the publication of your public key to your server. You can (and should) run:

ssh-copy-id username@hostname

This command makes sure that the pubkey is added to the appropriate file, and it makes sure nothing gets lost. This is the best possible way to accomplish this task if the command is available to you. However, should you be working in a system where this isn't available (such as a Solaris environment), you can use the following steps to make things work.

Copy your public key to the server as a specific filename.

scp ~/.ssh/id_rsa.pub username@server:~/.ssh/authorized_keys

Make sure the authorized_keys file has the same permissions so it won't be compromised.

The next time you log in from this client machine, you won't be asked for your password.

Wednesday, August 4, 2010

Techville: Wireless Windows Woes

I've never been a with-the-grain type of tech. This, I suspect, is because I tend to be a hacker in the original sense of the word. I want things to be efficient, even if that makes things a little ugly. Being a hacker, however, I usually manage to make things both functional and attractive. And speedy.

Also, I suspect it is because most people who come into IT jobs come straight out of a college education and into a corporate world, and both college and corporate are ruled by Windows. To many of these people, I suspect that a 20 GB operating system with no default usable software is acceptable, and I guess they think a fifteen minute boot time is worth the wait.

Fifteen minutes? Exaggeration?

Hardly. Not in this case.

See, we're all issued netbooks at my place of employment, and for good reason. We cover a lot of physical ground, and are often away from the office fixing computers for hours at a stretch. We need to update our trouble tickets in the meantime, so we bring along our netbooks with solid state drives (to improve mobility and decrease hardware damage from jostling) and update tickets while connected to the ubiquitous wireless network.

There are rules applied to the use of these netbooks, mostly surrounding security, and there's an image that gets blasted onto these machines that contains the OS (Windows XP SP2), a handful of useless software (does anybody still use iTunes 6, and what purpose does that hold at work if it's too old for iPhones?), and SafeBoot (disk encryption with some remote password recovery software). All of this software presumably meets the requirements of the security department, but it all conspired against me because (and I cannot stress this enough) I hate bloat.

Furthermore, wireless didn't work. This is a Dell Inspiron Mini 10 with a Broadcom wireless chipset in it, and I tried everything. I enabled and disabled services. I switched which applications were in control of the hardware. I fiddled with the wireless switch (which is Fn+F2 on this device). Nothing worked. In the end, every single program (including the Dell WLAN management software) informed me that although the hardware was present, recognized, had the proper drivers, and was working, it was detecting no wireless networks in range.

Every computer around me, including the expensive, oversized paperweight they call an iMac was detecting the aforementioned ubiquitous wireless networks (there are two). So why wouldn't my netbook see them?

I'd been planning on tartsenefeding my computer anyway (which is like defenestrating, but backward; I'm not throwing the device out of a window, I'm throwing Windows out of the device) because of my problem with the bloated OS image, but I decided to use this as a troubleshooting opportunity as well.

I put together a bootable flash drive with Ubuntu 10.04 on it and a 1 GB persistence file, and booted to it. I installed the Broadcom STA Wireless driver and rebooted. Thanks to the persistence file, the driver remained intact upon second boot, and I immediately had access to four wireless networks. For those not counting, that's two more than every other PC in the room saw. And for those wondering, both boots and the driver install took less time than it took to boot the native Windows image once.

Thanks to Ubuntu, my netbook runs faster and more stably than anybody else's in the office, and the hacker in me is satisfied without my ever having to hack anything. I realize, too, that Ubuntu isn't exactly a lightweight distro, and that Puppy (yes, I know it's really Ubuntu) or Knoppix or even a custom-built Slackware would almost certainly run faster and more stably. But considering the convenience factor of getting Ubuntu installed (20-30 minutes), its built-in support for encrypted file systems (a mandate for these netbooks), and its overall great appearance and performance, it's probably the best distro for this netbook at the moment.