Ubuntu 12.04 Ultimate Server Guide

Part 11: File Backups (Bacula)

This section covers the installation and configuration of Bacula, a backup system for your servers. In this scenario, there are two servers Neo and Morpehus. Neo is the main server as configured throughout the rest of this guide and Morpheus is the backup server that has so far been configured to provide backup DHCP, DNS and now file backup services.

Bacula is a very modular system, it can run entirely on one server, or spread across many. The most important Bacula module is “Bacula Director” this is the brains of the entire operation and coordinates everything that Bacula does. For this setup, Neo will be the director and Morpehus will be a designated storage server.

Neo – Bacula Director
Morpheus – Bacula Storage Server

Bacula needs a database to work with, so we’re going to install the MySQL service on Neo.

sudo apt-get install mysql-server

Set a password for the root MySQL user.

Now we’re going to install Bacula Director on Neo as well as the console tool for management purposes and bacula-fd (Bacula File Director). Bacula File director needs to be installed on any machine that you want to be able to backup.

sudo apt-get install bacula-director-mysql bacula-console bacula-fd

Select “Yes” when asked if you’d like db-config to setup the database for you, then enter the password you set for the root MySQL user when prompted, then set a password for the bacula MySQL database user.

sudo nano /etc/bacula/bacula-dir.conf

The first thing to look at is the Director section. You’ll need to set DirAddress to the FQDN for your machine.

Director {                            # define myself
  Name = neo-dir
  DIRport = 9101                # where we listen for UA connections
  QueryFile = "/etc/bacula/scripts/query.sql"
  WorkingDirectory = "/var/lib/bacula"
  PidDirectory = "/var/run/bacula"
  Maximum Concurrent Jobs = 1
  Password = "vOchjaYxGgFnKdbE5me5a3Tp5vs763FyoIhXdqUF733A"         # Console p$
  Messages = Daemon
  DirAddress = neo.danbishop.org #machine's FQDN
}

You’ll also need to modify the storage section to tell bacula that we intend to use morpheus for storing files. Do this by modifying Address to be the FQDN of your storage machine. Also make a note of the password, so that we can set the same one on morpehus.

Storage {
  Name = File
# Do not use "localhost" here
  Address = morpehus.danbishop.org                # N.B. Use a fully qualified $
  SDPort = 9103
  Password = "8JjXg-u9iNBcETFs14Rjv4GJDytGeAkSa"
  Device = FileStorage
  Media Type = File
}

That’s the director configured, but so that we can test and monitor things, we also need to configure the Bacula Console on Neo. This program will provide our interface to Bacula.

sudo nano /etc/bacula/bconsole.conf

The password should already be correct, but you can check this against the console password in /etc/bacula/bacula-dir.conf You will, however, need to change the name from localhost-dir to neo-dir and most importantly, the address from localhost to the FQDN for your director, e.g. neo.danbishop.org

Director {
  Name = neo-dir
  DIRport = 9101
  address = neo.danbishop.org
  Password = "jfdkjsdhfkjashdfksdjh837423424fd"
}

Now on morpheus:

We’re going to install the Bacula Storage Daemon to make Morpheus a storage server.

sudo apt-get install bacula-sd
sudo nano /etc/bacula/bacula-sd.conf

Set the SDAddress under the storage section to morpheus’┬áIP address… I’ve found that otherwise you get connection errors when Neo tries to access Morpheus. Not entirely sure why yet, perhaps a bug? In any case, this way definitely works.

 
Storage {                             # definition of myself
  Name = morpheus-sd
  SDPort = 9103                  # Director's port
  WorkingDirectory = "/var/lib/bacula"
  Pid Directory = "/var/run/bacula"
  Maximum Concurrent Jobs = 20
  SDAddress = 192.168.0.3
}

Now scroll down to the Director section and set the name and password to the same ones used on neo

Director {
  Name = neo-dir
  Password = "8JjXg-u9iNBcETFs14Rjv4GJDytGeAkSa"
}

Finally, the storage media… in this case I’ve created a /backup folder (sudo mkdir /backup) configured like so:

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /backup
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}

Configure the messages section at the end of the file so that neo is informed of any problems.

Messages {
  Name = Standard
  director = neo-dir = all
}

Now restart the service to load the new configuration:

sudo service bacula-sd restart

Switch back to Neo now.

By default, Bacula will backup /usr/sbin only… probably not very useful given everything in there has come from the Ubuntu repos and is available on thousands of mirrors across the globe… so we’re going to change that.

sudo nano /etc/bacula/bacula-dir.conf

Scroll down to the following section:

Job {
  Name = "BackupClient1"
  JobDefs = "DefaultJob"
}

This is one of the default jobs that will be executed daily by Bacula, you might want to rename it to something a little more descriptive, for example:

Job {
  Name = "BackupNeo"
  JobDefs = "DefaultJob"
}

Scroll down a little further and you’ll find the following:

# List of files to be backed up
FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
    }
#
#  Put your list of files here, preceded by 'File =', one per line
#    or include an external list with:
#
#    File = 

This lists all the files to back up, as you can see, this is currently only /usr/sbin, I've modified it below so that the entire / partition is backed up excluding locations recommended by Bacula AND the directory /home/dan/Videos (I just don't have enough space to back that up!).

# List of files to be backed up
FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
    }
#
#  Put your list of files here, preceded by 'File =', one per line
#    or include an external list with:
#
#    File = 

Save the file and restart bacula-dir to load the configuration we set earlier and the new job definition we've just created.

sudo service bacula-director restart

Now we can use the Bacula console to manually start the first run of the backup job.

sudo bacula-console

You'll be met with a * prompt. You can enter "help" to see a list of commands, for now though, type "run". Then select job 1 "BackupNeo". Type yes and hit enter.

You'll be told you have messages. Type "messages" to retrieve them. You shoudl see something like the following:

05-Apr 11:23 neo-dir JobId 2: No prior Full backup Job record found.
05-Apr 11:23 neo-dir JobId 2: No prior or suitable Full backup found in catalogue. Doing FULL backup.
05-Apr 11:23 neo-dir JobId 2: Start Backup JobId 2, Job=BackupNeo.2013-04-05_11.23.30_03
05-Apr 11:23 neo-dir JobId 2: Using Device "FileStorage"
05-Apr 11:23 File JobId 2: Job BackupNeo.2013-04-05_11.23.30_03 is waiting. Cannot find any appendable volumes.
Please use the "label" command to create a new Volume for:
    Storage:      "FileStorage" (/backup/)
    Pool:         File
    Media type:   File

This shows that Bacula has detected that the job has never been run before and consequently a Full backup needs to be done, not just an incremental one. However, it hasn't been able to find any appendable volumes on Morpheus to save the backup to. That's just because we haven't labelled Morpheus yet. Simply enter the command "label" and then a name for Morpheus to use for the backup, e.g. "Neo". You'll be asked to choose a pool for the new volume, select File (3).

Now attempt to run the job again...

run, 1, yes

Again, you'll have messages, this time though, simply telling you a full backup is commencing:

05-Apr 11:27 File JobId 2: Wrote label to prelabelled Volume "Neo" on device "FileStorage" (/backup/)
05-Apr 11:27 neo-dir JobId 3: No prior Full backup Job record found.
05-Apr 11:27 neo-dir JobId 3: No prior or suitable Full backup found in catalogue. Doing FULL backup.

You're done... you can monitor the progress using the status command. Bacula will automatically run this job every night at 23:10 by default.

Coming Soon

Printing

Coming soon, a guide to configuring CUPS print server for your new environment.

Managing Clients - Puppet

Coming soon, a guide on using Puppet to manage Ubuntu, Windows and OS X clients - distributing software, updates, settings etc.

  • Christian Oswald

    Hello,
    it’s a very useful tutorial and I learned a lot from it.
    I had also the problem with “Error adding group domainusers to LDAP” and in my case I solved it with switches TLS off in the LDAP-Server. I made it with webmin because I can’t find the correct place for it in the configuration files. I think it depends from the defaults of the ubuntu installation (in my case 14.04).
    But I have also a problem with the kerberos authentification. It works nice on the server (kadmin.local runs, kinit brings a ticket …) but from a client I get all times the error “kadmin: Cannot contact any KDC for requested realm while initializing kadmin interface”.
    I have reinstalled all, checked the configuration file of dnsmasq, krb5 … nothing helps, no firewall runs …
    I have tested a lot – ping, nslookup works and give the correct server. But nmap said that only port 749 is open on the server but in the kdc.conf is written that port 750 and 88 is used. I don’t if it’s important.
    Has anyone any idea for the reason of this error?

    Thanks,
    Christian

  • Jezzirolk

    hey Dan, i have used your guides a few times and they are great. Still work with 14.04 i dont think there was any tweaking i really had to do. i have a question though, is there a reason you disabled cache_credntials. Not saying there arent possible security reasons but i was more curious if there were other technical reasons becasue when connecting a laptop it is providing to make this a bit harder.

    –jezzirolk

    • danbishop88

      Hi Jezzirolk,

      I believe my reason for this was to do with: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1039151

      Basically, without it SSSD tends to come up before your network connection. This forces it into “offline” mode and it won’t even try to reconnect to your ldap/kerberos server until two minutes have elapsed. This prevents anyone from logging in for the full two minutes.

      A better workaround is listed in that thread, which is forcing the login screen to wait for the network to come up before appearing. I intend to move to that if I ever get round to finishing my 14.04 guide.

      Hope that helps…

      Dan

      • Jezzirolk

        Hey Dan,
        this still doesn’t really solve the issues i think, waiting for the network doesn’t do much for my case of a laptop. if i am off site it still wont connect properly unless you try to use cached credentials. Are we saying use cached credentials and then wait for network as to prevent the false negative of can not connect to ldap server? if that’s the case that might work.

        i guess the better question is if i log in off line. how does reconnecting once we end up back on a network with access to the server?

        Any thoughts on this and how to deal with the NFS mounts with laptop or systems that end up off site.

        –jezzirolk