SABnzbd+ and Sick Beard on Ubuntu Linux

zac

There are heaps of guides out there on the Internets for setting up SABnzbd+ and Sick Beard for automated download goodness – but most of them assume you’re setting up SAB and Sick Beard on a computer that’s being used as a regular PC, rather than a media server. As a result, the instructions out there could use some tweaking..

Recipe, pre-requisites and assumptions:

  • You’re using Ubuntu Linux
  • You have a user account to use on the media server (your own account, another account .. just not the root account)
  • You already have details for a news server

Step 1: Install SABnzbd+

Add the SABnzbd+ repository, update apt:

sudo add-apt-repository ppa:jcfp/ppa
sudo aptitude update

Install SAB and all its’ dependencies:

sudo aptitude install sabnzbdplus

Configure SAB as a service – set USER= to the user account you’ve chosen and set HOST= to 0.0.0.0 – everything else is fine to leave as is:

sudo vim /etc/default/sabnzbdplus
USER=username
HOST=0.0.0.0

Start SAB:

sudo service sabnzbdplus start

Run the configuration wizard by going to http://servername:8080/sabnzbd/ and following the prompts. You’ll need to enter the news server details; also make sure that ‘I want SABnzbd to be viewable by any pc on my network’ is selected and set a username and password (optional).

References
How to install SABnzbd on Ubuntu?
SABnzbd Manual – Ubuntu PPA Repository

Step 2: Install Sick Beard

Sick Beard installation needs git – if you don’t have it already:

sudo aptitude install git

Create the directory for installation:

sudo mkdir /var/lib/sickbeard

Pull down a copy of Sick Beard via git, then change ownership of the whole lot to the chosen user account:

sudo git clone git://github.com/midgetspy/Sick-Beard.git /var/lib/sickbeard
sudo chown -R username.groupname /var/lib/sickbeard/

Change to the Sick Beard directory, copy the init file out to where it belongs:

cd /var/lib/sickbeard
sudo cp init.ubuntu /etc/init.d/sickbeard
sudo update-rc.d sickbeard defaults

Edit the defaults file and add the following lines (changing out the username and home directory):

sudo vim /etc/default/sickbeard
SB_USER=username
SB_HOME=/var/lib/sickbeard
SB_DATA=/var/lib/sickbeard/sickbeard_data
SB_PIDFILE=/var/lib/sickbeard/pid

Start the Sick Beard service:

sudo service sickbeard start

Sick Beard should now be available at http://servername:8081/

References
Installing Sick Beard on Ubuntu Server

Step 3: Configure SABnzbd

Wherein the download directories are selected and Sick Beard configuration magic is made:

Load the SAB configuration section: http://servername:8080/sabnzbd/config/

In Folders, configure the Temporary Download Folder and Completed Download Folder items. Don’t choose directories that are the final destination for the TV – and make a note of the Completed Download Folder for later. Next, configure the Post-Processing Scripts Folder item – it should be /var/lib/sickbeard/autoProcessTV, but it depends on where Sick Beard is installed.

Scroll to the bottom, Save Changes and then Restart SABnzbd.

In General, make a note of the API Key for later.

In Categories, the magic to notify Sick Beard that the download has completed is configured – this is where the Post-Processing Scripts Folder comes in to play. Configure a new line on the table as follows:

ItemConfiguration
Categorytv
PriorityDefault
Processing+Delete
ScriptsabToSickBeard.py
Folder/PathThe Completed Download Folder from above
Groups / Indexer tagstv

Save – SABnzbd+ is now configured.

Step 4: Configure Sick Beard

Wherein the newznab services are configured and Sick Beard learns how to play with SAB . A few assumptions:

  • You have an account with a newznab service
  • The directory structure for your TV shows looks a little like this:
    /drive/tv/showname/season/episode

Load the Sick Beard configuration section: http://servername:8081/config/

In Search Settings, check Search NZBs, select SABnzbd from the dropdown and enter details for the SAB installation..:

ItemConfiguration
SABnzbd URLhttp://servername:8080/
SABnzbd UsernameUsername, if any
SABnzbd PasswordPassword, if any
SABnzbd API KeyAPI Key from above
SABnzbd Categorytv

Test SABnzbd – if everything works, Save Changes.

In Search Providers, configure a newznab service according to the documentation from the provider. None of the built-in providers seem to work particularly well; configure a Custom provider for best results.

In Post Processing, the downloaded file gets picked up and put in to the final directory structure. The base directory for the TV show is configured elsewhere; this is just the season and file name configuration. The important things to change are..:

ItemConfiguration
TV Download DirFolder from above (SAB’s Completed Download Folder)
Keep Original FilesUnchecked
Move Associated FilesUnchecked
Rename EpisodesChecked
Scan and ProcessUnchecked
Name PatternSeason 02/Show.Name.S02E03.HD.TV-RLSGROUP (from dropdown)
Multi-Episode StyleRepeat

Metadata is probably important if using XBMC, but it can be configured later.

Step 5: Test!

Wherein a new TV show (that hasn’t previously been downloaded) is added and an episode is downloaded to make sure the automation works:

Load the Sick Beard home page: http://servername:8081/home/

Select Add Shows then Add New Show. Start with a show that hasn’t been downloaded to test; add the others later. In Step 1, search for and select the show to download. In Step 2, select the directory that contains subdirectories with TV shows (/drive/tv/ from the above directory structure). Set this as the default. In Step 3, select the download quality, Save Defaults and Add Show. Wait a minute while Sick Beard gets the show details from the Internets..

The show ought to have been added with all episodes set to ‘Skipped’ – change one episode to ‘Wanted’ and within a minute it should begin downloading in SAB (and be transferred in to the appropriate directory once the download is complete). 🙂

If everything works, go back to Add Shows and then Add Existing Show. Based on the defaults set when adding a new show, Sick Beard should be able to see any other show downloaded; they can all be added to Sick Beard by stepping through the wizard to make sure the show name and directory have been correctly guessed. Enjoy! 😀

Software RAID on Ubuntu Linux

zac

Preamble

I recently purchased myself a HP ProLiant MicroServer with a stack of Western Digital Red hard drives with the intention of building a media server. After stumbling through a few tutorials that didn’t work very well, a colleague took pity on me and gave me some really good advice on how to get things running – so I figured a post on how to build a software-based RAID using Ubuntu Linux might be a reasonably useful thing. 🙂

Ingredients

  • Server hardware and dedicated hard drives for the RAID array
    (shouldn’t really matter what specific hardware gets used)
  • Ubuntu Server 13.04 or later
  • Patience

Recipe

First up, install the server edition of Ubuntu Linux. All the default install options are probably fine; there are enough guides on the Internets to cover setting this up if you get lost.

Once Linux is up and running and all of the hard drives have been detected appropriately (use lsblk to confirm device names), use parted to configure the partition table and create a partition on each disk..:

sudo parted -a optimal
select /dev/sda
mklabel gpt
mkpart
  -> raid_d1
  -> ext4
  -> 1MiB
  -> 3TiB
quit

Rinse and repeat for each drive – you’ll need to change the device being selected each time (use the output from lsblk as a guide).

For the mkpart section, the question ‘Partition name?’ is mostly irrelevant (I went with “raid_d1”, “raid_d2” etc) .. and for the question ‘End?’, use the advertised size of the drive (in my case, 3 Terabytes).

In hindsight, because I’m using the full drive for the array (vs. some for the array and some for other partitions), I’m not even sure that creating partitions using parted > mkpart is entirely necessary. It doesn’t take long though, so I guess it doesn’t hurt. 😉

Time to build the array (this is where having patience comes in)..:

sudo mdadm --create /dev/md0 --chunk=256 --level=5 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd

My colleague suggested a chunk size of 256k (vs. the default of 512k) for better performance. Other than that, the command should make plenty of sense: you’re creating a RAID 5 array at /dev/md0 with 4 devices (as listed at the end of the command).

In case it complains mdadm isn’t installed..:

sudo aptitude install mdadm

This will take ages .. ~18 hours for me. Monitor the progress with..:

watch -n5 cat /proc/mdstat

Once the array has completed building, you need to..:

  1. Update /etc/mdadm/mdadm.conf to list the hard drives that form part of the array (DON’T list the array device itself)
  2. Update initrd
  3. Reboot

I customised the following two lines in mdadm.conf (no need to change anything else from defaults)..:

DEVICE /dev/sda /dev/sdb /dev/sdc /dev/sdd
MAILADDR some@email.address

then

sudo update-initramfs -u
sudo reboot

After the reboot, your RAID array should turn up at /dev/md0. Confirm with the following..:

sudo mdadm --detail /dev/md0

If it’s not there, try..:

sudo mdadm --detail /dev/md127

If the array turns up at /dev/md127, it could mean that you have problems with mdadm.conf – see this Ubuntu Forums thread for further reading.

Now the array is configured, time to format it. Choice of filesystem is up to you, but I chose XFS (it plays nice with RAID and large file sizes). First, install XFS as it’s not a part of the standard Ubuntu Server image, then create the filesystem..:

sudo aptitude install xfsprogs
sudo mkfs.xfs -L data /dev/md0

Finally, get the UUID of the file system and load that in to /etc/fstab. Use blkid to find the UUID that belongs to /dev/md0..:

sudo blkid
/dev/md0: LABEL="data" UUID="3d3cf1c1-6015-4b5d-ac08-e38832fa29d6" TYPE="xfs"

Now, add that to /etc/fstab (I’m using /data as my mount point, but whatever works best for you)..:

# RAID array
UUID=3d3cf1c1-6015-4b5d-ac08-e38832fa29d6 /data xfs defaults 0 0

Reboot once more for good measure. Hopefully you get something like this as your df -h output..:

Filesystem Size Used Avail Use% Mounted on
/dev/sde1 28G 1.4G 25G 6% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 929M 8.0K 929M 1% /dev
tmpfs 188M 284K 188M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 939M 4.0K 939M 1% /run/shm
none 100M 0 100M 0% /run/user
/dev/md0 8.2T 15G 8.2T 1% /data

😀

Add a VPN to a FRITZ!Box router

zac

FRITZ!Box modem routers are able to run IPSec VPN endpoints, and even come with a clever VPN configuration utility .. that’s only available for Windows. If you don’t run Windows, you’re mostly out of luck (at least, as far as the official documentation goes), because the assumption is that everyone is running Windows (!)

If you’re not running Windows, you’ll need to create the VPN configuration file and upload it to the FRITZ!Box router by hand. It’s actually not that tricky, but the instructions are near impossible to find on the Internets – so here’s my own version. I’ve borrowed liberally from Marius van Witzenburg’s blog post on the same topic.

Copy the following in to a plain text file and save it as vpn.cfg:

vpncfg {
        connections {
                enabled = yes;
                conn_type = conntype_user;
                name = "NAME";
                always_renew = no;
                reject_not_encrypted = no;
                dont_filter_netbios = yes;
                localip = 0.0.0.0;
                local_virtualip = 0.0.0.0;
                remoteip = 0.0.0.0;
                remote_virtualip = VPN_CLIENT_IP;
                remoteid {
                        key_id = "NAME";
                }
                mode = phase1_mode_aggressive;
                phase1ss = "all/all/all";
                keytype = connkeytype_pre_shared;
                key = "SHARED_SECRET";
                cert_do_server_auth = no;
                use_nat_t = yes;
                use_xauth = yes;
                use_cfgmode = no;
                xauth {
                        valid = yes;
                        username = "USERNAME";
                        passwd = "PASSWORD";
                }
                phase2localid {
                        ipnet {
                                ipaddr = 0.0.0.0;
                                mask = 0.0.0.0;
                        }
                }
                phase2remoteid {
                        ipaddr = VPN_CLIENT_IP;
                }
                phase2ss = "esp-all-all/ah-none/comp-all/no-pfs";
                accesslist = "permit ip any VPN_CLIENT_IP 255.255.255.255";
        }
        ike_forward_rules = "udp 0.0.0.0:500 0.0.0.0:500", "udp 0.0.0.0:4500 0.0.0.0:4500";
}

A few changes are needed — highlighted above and explained below:

  • NAME – replace these (x2) with an alphanumeric name – if you’re using an Android device, it’s important that you do NOT use special characters, symbols or any punctuation (if you do, the connection will likely fail without any sort of useful error message)
  • VPN_CLIENT_IP – replace these (x3) with an IP address from the internal network range being used by the FRITZ!Box, but outside the DHCP pool range. If you haven’t changed any of the default settings in the Home Network > Network > Network Settings section, then use 192.168.178.201 (if you have made changes, you’ll need to work out what IP address to use for yourself :))
  • SHARED_SECRET – replace this with a long, alphanumeric password – I use a random string about 30 characters long – just bear in mind that you’ll need to enter this string on anything you set the VPN up on 😉
  • USERNAME – replace with a username (only use lowercase letters)
  • PASSWORD – replace with a password (avoid special characters, especially quote marks)

Leave everything else as is and re-save the file. Log in to the FRITZ!Box and go to the VPN configuration section (Internet > Permit Access > VPN), upload the file and you should now see a configured VPN waiting to be used. 🙂

To configure an iDevice, check this Apple Support KB article and use the following settings:

Configuration optionSetting to use
VPN TypeIPSec
DescriptionFritzVPN
ServerThe (static) IP address, hostname or DDNS hostname for the Internet service that your FRITZ!Box is connected to
AccountUSERNAME from above
PasswordLeave set to ‘Ask Every Time’ (or use PASSWORD from above, however consider the security implications of doing so..)
Use CertificateLeave set to ‘Off’
Group NameNAME from above
SecretSHARED_SECRET from above

To configure an Android device, go to Settings, select ‘More…’ under Wireless & Networks, select VPN and then tap the ‘+’ symbol. In the Edit VPN profile pop-up, use the following settings:

Configuration optionSetting to use
NameFritzVPN
TypeIPSec Xauth PSK
Server addressThe (static) IP address, hostname or DDNS hostname for the Internet service that your FRITZ!Box is connected to
IPSec identifierNAME from above
IPSec preshared keySHARED_SECRET from above

The Android instructions were written based on the AOSP release of Android 4.2 Jelly Bean – your mileage may vary with different/customised versions of Android. The earliest version of Android with built-in support for the IPSec VPN that the FRITZ!Box uses is 4.0.4 Ice Cream Sandwich – if you have an earlier version and can’t upgrade, you’ll need to buy VpnCilla from the Google Play store instead.

Setting up VPN access from a computer or other device may be slightly tricker but should follow the outline above – the actual instructions will depend on operating system version etc (Google will be your best friend here).

If multiple concurrent VPN connections are required, simply edit all of the fields per the above instructions to something else (except maybe leave SHARED_SECRET the same), save and upload as a second VPN connection.