Ubuntu 14.04 Ultimate Server Guide

Part 11: File Backups (Bacula)

This section covers the installation and configuration of Bacula, a backup system for your servers. In this scenario, there are two servers Neo and Morpehus. Neo is the main server as configured throughout the rest of this guide and Morpheus is the backup server that has so far been configured to provide backup DHCP, DNS and now file backup services.

Bacula is a very modular system, it can run entirely on one server, or spread across many. The most important Bacula module is “Bacula Director” this is the brains of the entire operation and coordinates everything that Bacula does. For this setup, Neo will be the director and Morpehus will be a designated storage server.

Neo – Bacula Director
Morpheus – Bacula Storage Server

Bacula needs a database to work with, so we’re going to install the MySQL service on Neo.

sudo apt-get install mysql-server

Set a password for the root MySQL user.

Now we’re going to install Bacula Director on Neo as well as the console tool for management purposes and bacula-fd (Bacula File Director). Bacula File director needs to be installed on any machine that you want to be able to backup.

sudo apt-get install bacula-director-mysql bacula-console bacula-fd

Select “Yes” when asked if you’d like db-config to setup the database for you, then enter the password you set for the root MySQL user when prompted, then set a password for the bacula MySQL database user.

sudo nano /etc/bacula/bacula-dir.conf

The first thing to look at is the Director section. You’ll need to set DirAddress to the FQDN for your machine.

Director {                            # define myself
  Name = neo-dir
  DIRport = 9101                # where we listen for UA connections
  QueryFile = "/etc/bacula/scripts/query.sql"
  WorkingDirectory = "/var/lib/bacula"
  PidDirectory = "/var/run/bacula"
  Maximum Concurrent Jobs = 1
  Password = "vOchjaYxGgFnKdbE5me5a3Tp5vs763FyoIhXdqUF733A"         # Console p$
  Messages = Daemon
  DirAddress = neo.danbishop.org #machine's FQDN
}

You’ll also need to modify the storage section to tell bacula that we intend to use morpheus for storing files. Do this by modifying Address to be the FQDN of your storage machine. Also make a note of the password, so that we can set the same one on morpehus.

Storage {
  Name = File
# Do not use "localhost" here
  Address = morpehus.danbishop.org                # N.B. Use a fully qualified $
  SDPort = 9103
  Password = "8JjXg-u9iNBcETFs14Rjv4GJDytGeAkSa"
  Device = FileStorage
  Media Type = File
}

That’s the director configured, but so that we can test and monitor things, we also need to configure the Bacula Console on Neo. This program will provide our interface to Bacula.

sudo nano /etc/bacula/bconsole.conf

The password should already be correct, but you can check this against the console password in /etc/bacula/bacula-dir.conf You will, however, need to change the name from localhost-dir to neo-dir and most importantly, the address from localhost to the FQDN for your director, e.g. neo.danbishop.org

Director {
  Name = neo-dir
  DIRport = 9101
  address = neo.danbishop.org
  Password = "jfdkjsdhfkjashdfksdjh837423424fd"
}

Now on morpheus:

We’re going to install the Bacula Storage Daemon to make Morpheus a storage server.

sudo apt-get install bacula-sd
sudo nano /etc/bacula/bacula-sd.conf

Set the SDAddress under the storage section to morpheus’ IP address… I’ve found that otherwise you get connection errors when Neo tries to access Morpheus. Not entirely sure why yet, perhaps a bug? In any case, this way definitely works.

 
Storage {                             # definition of myself
  Name = morpheus-sd
  SDPort = 9103                  # Director's port
  WorkingDirectory = "/var/lib/bacula"
  Pid Directory = "/var/run/bacula"
  Maximum Concurrent Jobs = 20
  SDAddress = 192.168.0.3
}

Now scroll down to the Director section and set the name and password to the same ones used on neo

Director {
  Name = neo-dir
  Password = "8JjXg-u9iNBcETFs14Rjv4GJDytGeAkSa"
}

Finally, the storage media… in this case I’ve created a /backup folder (sudo mkdir /backup) configured like so:

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /backup
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}

Configure the messages section at the end of the file so that neo is informed of any problems.

Messages {
  Name = Standard
  director = neo-dir = all
}

Now restart the service to load the new configuration:

sudo service bacula-sd restart

Switch back to Neo now.

By default, Bacula will backup /usr/sbin only… probably not very useful given everything in there has come from the Ubuntu repos and is available on thousands of mirrors across the globe… so we’re going to change that.

sudo nano /etc/bacula/bacula-dir.conf

Scroll down to the following section:

Job {
  Name = "BackupClient1"
  JobDefs = "DefaultJob"
}

This is one of the default jobs that will be executed daily by Bacula, you might want to rename it to something a little more descriptive, for example:

Job {
  Name = "BackupNeo"
  JobDefs = "DefaultJob"
}

Scroll down a little further and you’ll find the following:

# List of files to be backed up
FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
    }
#
#  Put your list of files here, preceded by 'File =', one per line
#    or include an external list with:
#
#    File = 

This lists all the files to back up, as you can see, this is currently only /usr/sbin, I've modified it below so that the entire / partition is backed up excluding locations recommended by Bacula AND the directory /home/dan/Videos (I just don't have enough space to back that up!).

# List of files to be backed up
FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
    }
#
#  Put your list of files here, preceded by 'File =', one per line
#    or include an external list with:
#
#    File = 

Save the file and restart bacula-dir to load the configuration we set earlier and the new job definition we've just created.

sudo service bacula-director restart

Now we can use the Bacula console to manually start the first run of the backup job.

sudo bacula-console

You'll be met with a * prompt. You can enter "help" to see a list of commands, for now though, type "run". Then select job 1 "BackupNeo". Type yes and hit enter.

You'll be told you have messages. Type "messages" to retrieve them. You shoudl see something like the following:

05-Apr 11:23 neo-dir JobId 2: No prior Full backup Job record found.
05-Apr 11:23 neo-dir JobId 2: No prior or suitable Full backup found in catalogue. Doing FULL backup.
05-Apr 11:23 neo-dir JobId 2: Start Backup JobId 2, Job=BackupNeo.2013-04-05_11.23.30_03
05-Apr 11:23 neo-dir JobId 2: Using Device "FileStorage"
05-Apr 11:23 File JobId 2: Job BackupNeo.2013-04-05_11.23.30_03 is waiting. Cannot find any appendable volumes.
Please use the "label" command to create a new Volume for:
    Storage:      "FileStorage" (/backup/)
    Pool:         File
    Media type:   File

This shows that Bacula has detected that the job has never been run before and consequently a Full backup needs to be done, not just an incremental one. However, it hasn't been able to find any appendable volumes on Morpheus to save the backup to. That's just because we haven't labelled Morpheus yet. Simply enter the command "label" and then a name for Morpheus to use for the backup, e.g. "Neo". You'll be asked to choose a pool for the new volume, select File (3).

Now attempt to run the job again...

run, 1, yes

Again, you'll have messages, this time though, simply telling you a full backup is commencing:

05-Apr 11:27 File JobId 2: Wrote label to prelabelled Volume "Neo" on device "FileStorage" (/backup/)
05-Apr 11:27 neo-dir JobId 3: No prior Full backup Job record found.
05-Apr 11:27 neo-dir JobId 3: No prior or suitable Full backup found in catalogue. Doing FULL backup.

You're done... you can monitor the progress using the status command. Bacula will automatically run this job every night at 23:10 by default.

Coming Soon

Printing

Coming soon, a guide to configuring CUPS print server for your new environment.

Managing Clients - Puppet

Coming soon, a guide on using Puppet to manage Ubuntu, Windows and OS X clients - distributing software, updates, settings etc.

  • Florent

    Hi Dan !
    Thank you for your great work, as usual 🙂

    I have an error on part 7, like another follower of your 12.04 guide (http://www.danbishop.org/2012/06/02/ubuntu-12-04-ultimate-server-guide/#comment-1552860993).
    Command kadmin -p dan/admin -q “addprinc -randkey nfs/$(hostname -f)” failed with :
    Authenticating as principal dan/admin with password.
    kadmin: Cannot resolve network address for admin server in requested realm while initializing kadmin interface

    Moreover, after restart sssd, it is impossible to connect to LDAP user with su, and getent passwd only shows local user.
    I think there is a problem between client and server. Evertyhing works fine on server side, and client can ping server through kerberos.domain.local and ldap.domain.local.

    Thank you for your help

    • danbishop88

      Hi Florent,

      Try appending -s kerberos.domain.local to that command and see what happens. It looks like you’re customising some parts of the guide for your own environment, but not others… it’s possible that your /etc/krb5.conf file on the client is wrong.

      I’m hoping to have a way of doing all of this using puppet going very soon. I’m almost there and will hopefully publish in the next couple of weeks. That should eliminate all the problems people are having by varying parts of the guide for their environment, but not others.

      Dan

      • Florent

        Thank you very much for your answer.
        I reinstall server & client. No customising except domain (domain.local) and kerberos username as florent/admin. On the client side, sudo dpkg-reconfigure krb5-config only ask realm but server and admin server where difined during installation. My /etc/krb5.conf file seems to be good. My domain isn’t defined in [domain_realm] but it is in [libdefaults]. Servers name are in top of [realms] (kdc and admin_server).
        Is there au place where krb logs ?

        Thanks again.
        Florent

    • Hi Florent,

      i run accoss the same problem as you have, after several tryout found out that you need to point you DNS Client into IP of Server.

      ie. DNS Client is : 8.8.8.8 from Network Manager, then you change to 192.168.1.100 (IP of Ubuntu Server)

      and voila :)… Hope this helps !

    • aqw

      I also got this error and solved it by adding 192.168.0.2 kerberos.danbishop.org to /etc/hosts of the client machine

    • OleHoppe

      Solved this issue by editing /etc/nsswitch.conf on the client (kubuntu 16.04.2). Modify “hosts” line to read:
      hosts: files dns
      HTH

  • Marco

    Hi Dan, thanks for this beatifoul guide

    i try many times your guide but have some problem

    server configure as your guide
    when try login from a configured client (using server username) it log succesfull but unable to mount home directory

    “Could not chdir to home directory /home/fabio: No such file or directory”

    check and check again but still this problem….

    can you help me?

    • Marco

      I have only linux (debian) pc, i’m interested central accounting and share users home directory… samba in future (now not important)

      ubuntu 14 server configure as your VERY clear guide (and thanks!!)
      i’m able lo ssh login on server using LDAP user accounts and is fantastic

      then want allow login from clients, pc… configure as guide… login is accepted but NO home directory mounted

      how i can debug where problem is?

      Sorry for my english

      • GuiSenges

        Hello Marco,

        I got the same problem. Any news? I’ve double checked this tutorial multiple times and everthing seems ok…

        • GuiSenges

          Small Update: When running automount in verbose mode I get “mount(nfs): no hosts available”. NFS server is up and running…Very strange…

        • GuiSenges

          Ok, I figured it out: Ports 111 and 2049 both TCP and UDP needs to be openned….Thanks for the excelent tutorial Dan!

        • Marco

          In my case there was no firewall…

        • Marco

          Problem solved, was a problem with reverse resolution… because i have an existing bind dns server and need use it, can’t use dnsmasq. Now NFS/kerberos works perfect!!
          Tanks for this perfect, clean guide DAN!!

          Now going on to configure samba, but i have a problem again (sorry!!): windows PC can see samba share but can join to samba domain. When try to join samba domain from a windows pc, it ask user name and password (i use my user (domainuser) with net rpc rights grant) but give me an error… each username and password i try to use same error…

    • Hi Marco,

      i have the same situation as you are, can you help me out the solutions here ?

  • Marco

    Hi again,
    i have this strange problem:

    if i reinstal ubuntu on server, add again ldap users, kerberos etc etc (as in this guide)

    clients can’t mount their home using nfs

    try to login one from client, perform again:
    kadmin -p dan/admin -q “addprinc -randkey nfs/$(hostname -f)”
    sudo kadmin -p dan/admin -q “ktadd nfs/$(hostname -f)”
    but nothink happens

    if i reinstall ubuntu on client pc, follow client configuration (of this guide) all return perfect working

    I think problem is Kerberos, but how to solve this problmem when i reinstall OS on server?
    try to remove nfs princ from server but not solution

    any advices?
    thanks

    • Hi Marco,

      happens to be the same issue with me as well,
      but instead of reformatting ubuntu client, here are what i did :

      apt-get purge –remove krb5-user -y
      REBOOT NOW !

      apt-get install krb5-user -y
      dpkg-reconfigure krb5-config

      sudo kadmin -p dan/admin -q “addprinc -randkey nfs/$(hostname -f)”
      sudo kadmin -p dan/admin -q “ktadd nfs/$(hostname -f)”

      REBOOT NOW !

      and you can easily login with your domain users credentials 🙂

      (PS : your guess is right it is somehow connected to Kerberos, but no need to remove nfs princ 🙂

      • Marco

        Hi,
        thanks for this solution!

  • Steve Thompson

    Hi,
    Thanks for the great guide. I have a question about this setup – I also had this problem with the 12.04 guide. In both, you set neo to have a static IP and then you configure dnsmasq to use Google’s public servers. At no point, as far as I can see, do you tell neo which DNS servers to use which means it has no DNS. I’ve added dns-nameservers 127.0.0.1 to the static config to get round this but I can’t figure out if I’m missing something in the instructions that would make this unnecessary?

    Thanks

  • Hi Dan, just a quick correction,

    on Page 6
    there is something like

    #BINDPWDFILE=”/etc/ldapscripts/ldapscripts.passwd”

    it should be not comment (remove the hashtag)

    otherwise it will come out results

    “Unable to read password file , exiting…”

    But overall this is Awesome tutorials :))

    • Thanks Adhi… I had the same issue. Great guide by the way, Dan 😀

  • Marco

    Hi, i have new problem

    new workstation wit Windows 10 need to be joined to Samba

    not no succes 🙁

    no problem when add windows7 macchines

  • Ian Carey

    The package ldapscripts didn’t work for me. I used ldap-account-manager instead.

    For Samba, by default you get Samba 4. Your command “adminlocal@neo:$ net rpc rights grant -U dan “danbishop.orgDomain Admins” SeMachineAccountPrivilege SePrintOperatorPrivilege SeAddUsersPrivilege SeDiskOperatorPrivilege SeRemoteShutdownPrivilege” is depreciated and will not work with Samba 4.

    Fix is a new syntax below.

    net sam rights grant “danbishop.orgDomain Admins” SeMachineAccountPrivilege SePrintOperatorPrivilege SeAddUsersPrivilege SeDiskOperatorPrivilege SeRemoteShutdownPrivilege -Udan

    Other than those two things I have it all up and running perfectly. Thank you Dan!

  • Jheguy2

    I’ve been getting the error:
    Error adding group domainadmins to LDAP
    every time I try and create a new group. I’ve checked the logs and they are returning:
    ldap_add: No such object (32)
    If anyone could help me I would be more than grateful. Thank you.

    • Pekka R.

      Had similar problem. Did not check the logs, so not completely sure if same problem.
      I commented line BINDDN=”cn=Manager,dc=example,dc=com” from

      /etc/ldapscripts/ldapscripts.conf
      since similar line was already added for actual DN by Dan’s instructions.

    • Dheeraj Bharat Sethi

      Hi there, In just faced the same issue, can you please let me know the solution for the same?
      Thanku

      • Dheeraj Bharat Sethi

        Got it resolved, adding password by vi was causing the issue.

  • Olivier Dir

    hi all after client install (ubuntu 14.04) without error
    i don’t found domainadmins with command getent group
    how do to debug this point ?

    where is the log file for see if there are a problem ?

    thank for your help

  • Marco

    All works fantastic (10months perfect working)
    but now i have new workstation with windows10 preinstalled, want join them tho ubuntu domain
    follow same guide than win7 but win10 give me an error, not join into domain giving an error (unable to contact domain)

    any help?

  • Kleder Osodarck

    Hello Dan!

    First thanks for the great tutorial!

    I’m using your guide to build an GNS3 network for a college project, the problem is I have to join Windows XP and 2000 machines in the domain, any information or direction on how to do so will be really appreciated! below is the error I’m receiving in a Windows XP machine:

    error 0x000005B4 ERROR_TIMEOUT

  • John Christensen

    I am not having luck getting Windows 7 to connect to the Samba DC. The error I am getting says that windows was trying to contact the SRV record for _ldap._tcp.dc._msdcs.. I added a SVR record to dnsmasq.conf and pointed the requests to ldap. on port 398. Now I am getting an error from Windows that the domain controller at ldap. could not be reached. Any thoughts on why this is not working for me?

  • TC

    On Ubuntu 15.10, I failed at Step 6, “sudo ldapaddgroup domainadmins”. In /var/log/ldapscripts.log, the error message is ldapadd: invalid format (line 1) entry “”. It turns out it is a bug in ldapscripts—it fails in some locales. For more information, read this:

    https://github.com/martymac/ldapscripts/issues/3

    You can fix this bugs by these commands:

    sudo su
    LC_ALL=C ldapaddgroup domainadmins
    LC_ALL=C ldapaddgroup domainusers
    LC_ALL=C ldapadduser csatc2002 domainadmins
    LC_ALL=C ldapaddusertogroup csatc2002 domainusers

    Just my two cents, and hope it helps.

  • Thanks a lot, Dan.
    I migrated the authentication of a cluster of machines with Ubuntu 14.04 from NIS to LDAP+Kerberos and it worked great. I tried to follow the instructions for SingleSignOn from the Ubuntu Help but it was very complicated, didn’t explain some concepts and contained some “magical” instructions. This guide help me figure out what I needed to do.

  • Srinath Bk

    Hi Dan,

    thanks for the excellent tutorial.

    I cannot use DHCP server for my setup since, there is already another DHCP server in the LAN. I would like to use static IP for both server and clients. Can you please suggest what needs to be changed?

    regards,
    Srinath

  • Dheeraj Bharat Sethi

    Hi dan,
    Thanku so much for the guide.
    been stuck on last step. when trying to login via the new ldap user.
    “No passwd entry for user”
    When trying to add passwd, “user does not exist”
    What is going wrong here?
    Thanku

  • Dheeraj Bharat Sethi

    Getting Permission Denied via SSH for the Ldap user added , and Incorrect Login via server login

  • Hello Dan,

    Are there any GUI based apps for managing Sudoer Rules in LDAP? Using a
    LDAP Browser is not a very user-friendly way for managing a large number
    of Sudo Rules. FreeIPA has a UI for managing Sudoer Rules, but it
    requires 389 Directory Server, and we don’t use that in our environment
    (for good reasons)

  • Bineural juls

    Hi, i think all its working ok, but as sson as i login by ssh with user i have this error.. Could not chdir to home directory /home/robert: No such file or directory
    And localy on the machine this one:
    No directory, logging in with HOME=/
    I can see the problem is that my home is not working.. but i dunno how to solve this.
    Any idea please?
    Thanks a lot

  • Pekka R.

    Salut!
    This is still the most clear and complete guide to set up a small company network environment with servers. I can not find better. And I have looked around. Just bloody exellent. “Ultimate” describes this well.
    Did finally have time to look at chapter 10. Just wondering what does mean “In this case a stock install of 12.04 server is being used” in this context.

    Dan? Does it mean a copy of the primary server installation that I could make with dd command and then twist around the dnsmasq configuration file?

    Maybe safest option is to start installing server from the start and follow up the master server setup instructions untill this point. I expect that the /etc/dnsmasq.conf file will be mirror of the master dnsmasq server regarding to ip-adresses and host names. Will start playing with image copied with dd first.

    Many thanks to Dan for giving this to public domain. To boil up this kind of instruction set so, that it is also comprehensible, must have taken a ton of work.

  • Gin

    Great guide and working one 🙂
    Any chance to update same tutorial for Ubuntu 16.04? Or maybe a little help pls. Things seems working till samba setup. First of all in Ubuntu 16.04 libpam-smbpass was removed. So I add user manually for krb and samba using same credentials (even account for machine in some tests). When i try to join an Windows 7 to domain I got follow error:
    “The join operation was not successful. The could be because an existing computer account having the name xxxx was previously created using a different set of credentials. Use a different computer name or contact your administrator to remove any stale conflicting raccount. The error was: Access is denied.”
    Ldap log:
    Jan 31 13:58:59 dcdoxmain slapd[1539]: conn=1052 fd=20 ACCEPT from IP=xxx.xxx.xxx.xxx (IP=0.0.0.0:389)
    Jan 31 13:58:59 dcdoxmain slapd[1539]: conn=1052 op=0 BIND dn=”” method=163
    Jan 31 13:58:59 dcdoxmain slapd[1539]: SASL [conn=1052] Failure: GSSAPI Error: An unsupported mechanism was requested (Unknown error)
    Jan 31 13:58:59 dcdoxmain slapd[1539]: conn=1052 op=0 RESULT tag=97 err=49 text=SASL(-13): authentication failure: GSSAPI Failure: gss_accept_sec_context
    Samba shares could be accessed from Windows 7 using same credentials.

  • Marco

    Hi again DAN, thanks again for this beatifoul guide… i’m using it in my office and works like a charm!!
    I’m really happy to have Win7 and Linux (debian) with distributed logins, home directories!
    Now i really have a bit headache trying join new windows 10 pro macchines to samba domain… i try many many different samba configuration but no solution.
    You know any solution?

    Thanks