Setting up a network-based, centralized home directory as a service

As I mentioned in my last post, I just built out a dedicated server at home, and I'm migrating a bunch of services to it. One reason I build this server in the first place was so that I could set up roaming home directories for all my Linux PCs in the house.

I tried a few configurations before landing on the one described here. I had problems where Ubuntu 12.04 would fail to boot if the system failed to mount the network drive properly when the mount was in the fstab. Ubuntu 10.04 would error on boot, but give you the opportunity to skip the mount and continue booting. However, if the share was mounted to /home on the client machine, logging into a desktop environment would fail on account of none of the config files being available. Even when the mount worked properly, having it mounted at /home caused performance problems. Relying on the network for processes like booting and logging into a desktop ended up being a deal-breaker for me having automounted network home directories.

But I did settle on a solution that works well. In brief, I servicized the mounting and unmounting of the network drive, and the control script I wrote for it also places netdrive symbolic links in each non-system user's home directory if a directory on the share exists in their name. Here's how it's done.

There are two parts to this configuration - a server and some clients. Let's set up the server.

My server is running Debian Squeeze, but these instructions should translate well to other distributions. First, install the NFS server:

sudo apt-get install nfs-kernel-server

Then set the config to share the /home directory over the network. Add this to the end of /etc/exports:

/home   192.168.0.0/255.255.255.0(rw,sync,fsid=0,no_subtree_check)

You should modify that to suit your specific needs. /home is the directory on the server to share. 192.168.0.0/255.255.255.0 means that the only systems who are allowed to access it will be on the 192.168.0.x subnet, which, for me, means everything behind my router. This share won't be accessible to anyone on the public side of my router. Finally, the (rw,sync,fsid=0,no_subtree_check) part are options that say, respectively, "Mount it read/write so users can save files here," "Write files to disk synchronously so there aren't sudden unmounting problems," "If the NFS server is NFS4, treat this directory as the root of all shared directories," and "Speed things up by allowing all subdirectories of the share to be accessed."

Restart the NFS server:

sudo service nfs-kernel-server restart

If you encounter problems, check /var/log/daemon.log for something to Google with.

I had firewall problems here. Mounting an NFS share on a remote server uses lots of different ports, and I was unable to identify them all. I ended up adding a rule to my firewall to let anything behind my router (that 192.168.0.x subnet) access any ports on the server. For me, this isn't a security issue. You should consider your situation and make that decision for yourself. At any rate, here's my command to make that firewall rule:

sudo ufw allow from 192.168.0.0/24 to any

Now get onto a client machine and try to mount it manually just to be sure it works:

sudo mount -t nfs -o proto=tcp,port=2049 192.168.0.100:/home /mnt

If you get no errors, try...

ls /mnt

...and check if what you see is what you expect. If so, unmount it:

sudo umount /mnt

Assuming that everything works, it's time to set up a control script. The script below is hardcoded to mount the NFS share to /mnt/netdrive. You should probably read through it and understand it before running it to be sure it won't interfere with things on your system. If it's going to work for you, paste it into /etc/init.d/netdrive or wherever else you decide.

safe_start () {

    if [ ! -d /mnt/netdrive ]; then
        mkdir /mnt/netdrive
    fi

    if grep -qs '/mnt/netdrive' /proc/mounts; then
        echo "Something is already mounted at /mnt/netdrive [ FAIL ]"
        exit 1
    else
        mount -t nfs -o proto=tcp,port=2049 192.168.0.100:/home /mnt/netdrive
    fi

    for u in `grep '/home' /etc/passwd | cut -d: -f1`; do
        if [ -d /mnt/netdrive/$u ]; then
            HOME=`grep "$u" /etc/passwd | cut -d: -f6`
            if [ -e $HOME/netdrive ]; then
                rm $HOME/netdrive
            fi
            ln -s /mnt/netdrive/$u $HOME/netdrive
        fi
    done
}

safe_unmount () {
    FILES_IN_USE=`lsof | grep '/mnt/netdrive' | awk '{print $9}'`
    if [ ! "$FILES_IN_USE" = "" ]; then
        echo "The following files are located on the netdrive and are still in use:"
        echo $FILES_IN_USE
        exit 1
    fi

    unmount
}

unmount () {
    umount /mnt/netdrive
    for u in `ls /home`; do
        if [ -d /home/$u/netdrive ]; then
            rm $u/netdrive;
        fi
    done
}

show_status () {
    if grep -qs '/mnt/netdrive' /proc/mounts; then
        echo "Netdrive is mounted."
    else
        echo "Netdrive is not mounted."
    fi
}

show_help () {
    echo "This script accepts the following commands: start, stop, forcestop, status, help"
}

case $1 in
    "start" )
        safe_start ;;
    "stop" )
        safe_unmount ;;
    "forcestop" )
        unmount ;;
    "status" )
        show_status ;;
    * )
        show_help ;;
esac

exit 0

Apply permissions:

sudo chmod 755 /etc/init.d/netdrive

Now you can run...

sudo /etc/init.d/netdrive start

...to mount it, or give it stop instead to stop it safely. The script will refuse to unmount the drive if it has files open still. If you want to override that check, use forcestop. status will tell you if the network drive is mounted or not. help will give you a basic usage message. When mounting, this script will also place the netdrive symlink in users' home directories, so, for example, /home/ryan/netdrive is mapped to /mnt/netdrive/ryan.

Now, I still wanted my drive to mount on boot. So I threw a symlink in the startup:

sudo ln -s /etc/init.d/netdrive /etc/rc2.d/S10netdrive

That's my Ubuntu 12.04 machine. I can't remember at what point in the startup procedure I placed in on my Ubuntu 10.04 box, but it's somewhat discretionary anyway.

So reboot your client and make sure you boot with a working netdrive symlink in your home directory. And done!

Setting up a headless Calibre server

My wife and I like to read books on the go, and we use an Android app called Aldiko (it's great) to read epubs. I have loads of epubs on my computer, and I have traditionally used Calibre's server feature to share them with our phones. In the past, I've kind of clobbered together things on an Ubuntu box that also served as my main desktop computer. That worked well in the past, but I've recently set up a headless Debian Squeeze server, and I decided to migrate my Calibre server there. I had some setbacks, and thought it would be worth it to document the process here, and how I got things to work successfully.

My server runs headless, and that's the first problem with Calibre. The version of it in the Debian Squeeze repositories doesn't have a standalone server mode that is configurable without using the graphical UI. I can't have that, so I decided to get newer binaries installed. Ordinarily, I wouldn't recommend this, especially on a system like Debian whose community prides itself on the stability of their software packages, but this is a functional necessity, so I'll take it. This command, ripped straight from the official Calibre website, worked for me:

sudo python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main()"

When prompted, I told it to install to ''/opt/calibre'' so it wouldn't conflict with any system libs or binaries.

That means that ''/opt/calibre/calibre-server'' is the standalone daemon for serving e-books.

I wanted to servicize it, so I wrote this init script and placed it at ''/etc/init.d/calibre-server'':

#!/bin/bash

CALIBRE_LIBRARY_PATH="/home/shared/Calibre Library"
PIDFILE=/tmp/calibre-server.pid
USER=calibre
PORT=8081

start() {
        echo "Starting Calibre server..."
        su -c "calibre-server --with-library=\"$CALIBRE_LIBRARY_PATH\" -p $PORT --pidfile=$PIDFILE --daemonize" & 
        if [ $? -ne 0 ]; then
                echo "Could not start calibre-server."
        fi
}

stop() {
        echo "Stopping Calibre server..."
        if [ -e $PIDFILE ]; then
                read PID < $PIDFILE
                ps aux | grep "$PID" | grep 'calibre-server' > /dev/null
                RUNNING=$?
                if [ $RUNNING -eq 0 ]; then
                        kill $PID
                        if [ $? -eq 0 ]; then
                                rm $PIDFILE
                        fi
                else
                        echo "Could not find a calibre-server process with PID $PID."
                fi
        else
                echo "Could not find pidfile: $PIDFILE"
        fi
}

restart() {
        stop
        start
}

status() {
        if [ -e $PIDFILE ]; then
                read PID < $PIDFILE
                echo "calibre-server is running with PID $PID."
        else
                echo "calibre-server is not running."
        fi
}

unknown() {
        echo "Unrecognized command: $1"
        echo "Try one of the following: (start|stop|restart|status)"
}

case $1 in
        start ) 
                start
                ;;
        stop )
                stop
                ;;
        restart )
                restart
                ;;
        status )
                status
                ;;
        * )
                unknown
                ;;
esac

You can change the variables at the top to run the server differently. Once this is given execute permissions, you can start the server with:

/etc/init.d/calibre-server start

And stopped with

/etc/init.d/calibre-server stop

It looks like the ''service'' command works, too:

service calibre-server start

I know there are some other problems I need to work out (how to import new books, for example), but this seems like a good start. Anyone have any tweaks or additions to note?

Techville: Wireless Windows Woes

I've never been a with-the-grain type of tech. This, I suspect, is because I tend to be a hacker in the original sense of the word. I want things to be efficient, even if that makes things a little ugly. Being a hacker, however, I usually manage to make things both functional and attractive. And speedy.

Also, I suspect it is because most people who come into IT jobs come straight out of a college education and into a corporate world, and both college and corporate are ruled by Windows. To many of these people, I suspect that a 20 GB operating system with no default usable software is acceptable, and I guess they think a fifteen minute boot time is worth the wait.

Fifteen minutes? Exaggeration?

Hardly. Not in this case.

See, we're all issued netbooks at my place of employment, and for good reason. We cover a lot of physical ground, and are often away from the office fixing computers for hours at a stretch. We need to update our trouble tickets in the meantime, so we bring along our netbooks with solid state drives (to improve mobility and decrease hardware damage from jostling) and update tickets while connected to the ubiquitous wireless network.

There are rules applied to the use of these netbooks, mostly surrounding security, and there's an image that gets blasted onto these machines that contains the OS (Windows XP SP2), a handful of useless software (does anybody still use iTunes 6, and what purpose does that hold at work if it's too old for iPhones?), and SafeBoot (disk encryption with some remote password recovery software). All of this software presumably meets the requirements of the security department, but it all conspired against me because (and I cannot stress this enough) I hate bloat.

Furthermore, wireless didn't work. This is a Dell Inspiron Mini 10 with a Broadcom wireless chipset in it, and I tried everything. I enabled and disabled services. I switched which applications were in control of the hardware. I fiddled with the wireless switch (which is Fn+F2 on this device). Nothing worked. In the end, every single program (including the Dell WLAN management software) informed me that although the hardware was present, recognized, had the proper drivers, and was working, it was detecting no wireless networks in range.

Every computer around me, including the expensive, oversized paperweight they call an iMac was detecting the aforementioned ubiquitous wireless networks (there are two). So why wouldn't my netbook see them?

I'd been planning on tartsenefeding my computer anyway (which is like defenestrating, but backward; I'm not throwing the device out of a window, I'm throwing Windows out of the device) because of my problem with the bloated OS image, but I decided to use this as a troubleshooting opportunity as well.

I put together a bootable flash drive with Ubuntu 10.04 on it and a 1 GB persistence file, and booted to it. I installed the Broadcom STA Wireless driver and rebooted. Thanks to the persistence file, the driver remained intact upon second boot, and I immediately had access to four wireless networks. For those not counting, that's two more than every other PC in the room saw. And for those wondering, both boots and the driver install took less time than it took to boot the native Windows image once.

Thanks to Ubuntu, my netbook runs faster and more stably than anybody else's in the office, and the hacker in me is satisfied without my ever having to hack anything. I realize, too, that Ubuntu isn't exactly a lightweight distro, and that Puppy (yes, I know it's really Ubuntu) or Knoppix or even a custom-built Slackware would almost certainly run faster and more stably. But considering the convenience factor of getting Ubuntu installed (20-30 minutes), its built-in support for encrypted file systems (a mandate for these netbooks), and its overall great appearance and performance, it's probably the best distro for this netbook at the moment.

Eight Things Windows Needs Before I'll Contemplate Using it Again

Windows 7 is better than Vista. Great. But saying that is like saying you'd rather catch the common cold instead of swine flu. I've demoed the release candidate for Windows 7, and I can safely say that I still don't like it. Aside from the default options being obnoxious and hard to use (the icons for running applications are identical to the directly-adjacent Quick Launch icons; running programs have no text to show you what they are; unless you have the hardware to back up the Aero interface, you can't get the window previews to help you, either), there are several things I need to see in a Windows operating system before I'll even contemplate switching back.

1. Multiple virtual desktops — Windows is pretty much the only significant operating system that does not support this. Mac OS X's desktops may not be implemented very well, but they're there all the same. My cell phone has multiple desktops. Why can't Windows get with the program on this? It's an invaluable feature which reduces clutter. I think you'll find that clutter reduction is centric to many of my needs.
2. Application organization — When I click on the Start Menu in Windows, I have a list of programs to sort through which aren't even alphabetized until I tell them to be. The list is huge, presenting me with a different "folder" for each program I have installed. When I have to go looking for a program, I want to be able to look in one of these "folders" that tells me what type of program it is. Is it an Internet program? A productivity program? Is it a minor accessory? One of my programming applications? Keeping this kind of organization to programs keeps the list short, which would be a blessing considering the tiny, half-height, scrolling list of applications which contains six times as many programs as will fit in its frame. Microsoft tried implementing something like this with games when they launched Vista, but that doesn't work automatically for everything because it's layered on top of the existing system, not integrated as part of the system. The way they implemented it required you to open a new window just to see your shortcuts. First of all, that's counterintuitive. Secondly, it clutters my desktop.
3. Useful window management — In Linux, I can click and drag windows across my multiple desktops by dragging to the edge of the screen in the appropriate direction. I can move a window by holding the Alt key and clicking and dragging anywhere at all on the window. I can move a window to the best location and resize it so that it's as big as it can get without overlapping any windows it wasn't already overlapping at a single keypress (see video below). In Windows 7, they have added some window management features where the movement of a window to an edge of the screen resizes the window to fill half of the screen along that edge. Whoopee. What if I don't want it at exactly half size? What if I just want my window on the right-hand side of the screen? There's no customization here, only an assumption that I want my windows to be exactly where Microsoft wants them. I sincerely hope this feature can be turned off.
4. Installation across drives — As it stands, I get a tiny speed boost and a major OS installation advantage by being able to install my home directory on a different drive or partition than the rest of my OS. This is great for home users because it means they can reinstall the operating system without damaging any of their personal data or application settings. It's also great for server users because MySQL databases can sit on a RAW partition, which is often faster because they don't have to follow the rules of the filesystem that way. The best I can manage in Windows is to create a separate partition and manually save and copy files to that partition after the fact. Nothing will be automatic, and I will have a large separation in functionality between the two. Unlike Unix OSes, Windows does not mount all filesystems fluidly together.
5. Security built in — With Vista, Microsoft attached "User Account Control" to Windows, and that turned out to be a major annoyance that did little to aid security. It prevented nearly every program from running because Windows required administrative privileges to run nearly every program. When all users have instant administrative control, that's a bad thing, and a security problem. That's why they pushed UAC through. But UAC popped up for everything, and most users just turned it off so they could be allowed to use their computer. Again, this is a bad thing, creating even more of a security problem. With Windows 7, not much has changed. Users can now select how many UAC warnings they receive. What will be the effect of this? Just like last time, users will either be annoyed or turn it off. Still a bad thing. Still a security problem. When Microsoft manages to write an OS that has security layered into its core, when they can sort out what should and should not require administrative privileges, they might have a chance at winning me over.
6. Fragmentation-free file system — I don't want to have to spend hours every month defragging my harddrive and slowing my computer to a crawl because my operating system allows fragmentation to happen. I certainly don't want my computer to do this in the background on a schedule that I'm unaware of, slowing my computer down when I need to use it. Mac and Linux do not allow for this to happen. A defragging program is not the proper solution. The NTFS file system is about a decade old now. It's no longer "New Technology." I never wanted to work for my computer in the first place, and it's time to ditch this abhorrable system.
7. Singular application installer and updater — In Windows, when I want software, I go to the Internet and either search Google or go to a website that I know carries that software. I install it using a six-page installation wizard that probably only needs to be a one-pager. I install software one program at a time. And then a week later, when the fifteen programs I took the time to install last week have been updated, I have to either download the software from the individual websites again and then reinstall them all separately through more wizards, or I must run fifteen separate updater programs in the background constantly, just waiting there for an update to happen. Neither of these are viable options. Linux uses central, customizable repositories to pipe software through a single, centralized installer/updater/uninstaller program that allows me to install, update, and uninstall as many programs as I want simultaneously, and in one fell swoop. Again, even my cell phone does this. And as with the fragmentation problem, Microsoft should fix the core problem instead of adding on layer after layer of faux-solution to bandage it.
8. Let me customize! — I don't want the ugly Aero interface, and I don't want the even uglier atrocity that I get when I turn off Aero. I want something that I like and that I choose. I want my colors in front of me. I want my style, my appearance, my everything. Please, Microsoft, let me do this without paying for third-party software that only adds a separate layer to the problem. The software linked above uses the Windows API to accomplish this, which means that the functionality exists deep within the Windows system files. If Stardock can do it, so can you. You can implement it straight into the software. Do it, already! Let me use my computer the way I want to.