Using bscan to Recreate a Catalog from a Volume

If you find yourself using this program, you have probably done something wrong. For example, the best way to recover a lost or damaged Bacula database is to reload the database by using the bootstrap file that was written when you saved it (default bacula-dir.conf file).

The bscan program can be used to re-create a database (catalog) records from the backup information written to one or more Volumes. This is normally needed only if one or more Volumes have been pruned or purged from your catalog so that the records on the Volume are no longer in the catalog, or for Volumes that you have archived. Note, if you scan in Volumes that were previously purged, you will be able to do restores from those Volumes. However, unless you modify the Job and File retention times for the Jobs that were added by scanning, the next time you run any Job with the same name, the records will be pruned again. Since it takes a long time to scan Volumes this can be very frustrating.

With some care, bscan can also be used to synchronize your existing catalog with a Volume. Although we have never seen a case of bscan damaging a catalog, since bscan modifies your catalog, we recommend that you do a simple ASCII backup of your database before running bscan just to be sure.

bscan can also be useful in a disaster recovery situation, after the loss of a hard disk, if you do not have a valid bootstrap file for reloading your system, or if a Volume has been recycled but not overwritten, you can use bscan to re-create your database, which can then be used to restore your system or a file to its previous state.

Note: Let's assume that you already reinstall your bacula server and put the volume in theirs location with the correct permission.

Let's take a look in the options that we can use with bcan

bscan -?
Copyright (C) 2001-2014 Free Software Foundation Europe e.V.

Version: 7.0.5 (28 July 2014)

Usage: bscan [ options ] <bacula-archive>
       -b bootstrap      specify a bootstrap file
       -c <file>         specify configuration file
       -d <nn>           set debug level to <nn>
       -dt               print timestamp in debug output
       -m                update media info in database
       -D <driver name>  specify the driver database name (default NULL)
       -n <name>         specify the database name (default bacula)
       -u <user>         specify database user name (default bacula)
       -P <password>     specify database password (default none)
       -h <host>         specify database host (default NULL)
       -t <port>         specify database port (default 0)
       -p                proceed inspite of I/O errors
       -r                list records
       -s                synchronize or store in database
       -S                show scan progress periodically
       -v                verbose
       -V <Volumes>      specify Volume names (separated by |)
       -w <dir>          specify working directory (default from conf file)
       -?                print this message

As was told in the beginning of this how to we need to have the volume, here I am pretend you are working as me with a storege rather than a tape.

We need to use the bscan to take a look in the volume first to make sure the volume is ok and we can recover some information from it.

Note: Put the volume into the directory that is set up by the device into bacula.

Let's take a look at the device configuration

cat /etc/bacula/devices/default-device.conf 
### Device default that will be used by bacula server
Device {
  Name = Default-Device                   # Device Name
  Media Type = File                       # Midia Type (DVD, CD, HD, FITA)
  Archive Device = /srv/backup/default    # Directory where will be put the volumes
  LabelMedia = yes;                       # Media Label that will be used by Bacula
  Random Access = Yes;                    # The archive device is assumed to be a random access medium which supports the lseek
  AutomaticMount = yes;                   # Mount the Device automatically
  RemovableMedia = no;                    # Removable Media needs to be set up as no
  AlwaysOpen = no;                        # Keep the Device always open
}

We need to define the device that will be use as the last argument in the command line, as I told let's pretend the volume is stored in /srv/backup/default as we can see in the configuration file above or if you are using a tape /dev/nst0

bscan -v -V volume-default-0001 /srv/backup/default
bscan: butil.c:289-0 Using device: "/srv/backup/default" for reading.
23-Dec 14:42 bscan JobId 0: Ready to read from volume "volume-default-0001" on file device "Default-Device" (/srv/backup/default).
23-Dec 14:42 bscan: ERROR TERMINATION at bscan.c:289
postgresql.c:236 Unable to connect to PostgreSQL server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect; max_connections exceeded.

As we can see something went wrong in the possible cause we can see a problem with the password or max_connections. So let's put the password, username, database and hostname in the command line.

The password put in with the parameter -P needs to be the same as in the bacula-dir.conf.

bscan -v -V volume-default-0001 -u bacula -P "4e4fb461c26dd94eb4f4354bfca14222" -h localhost -n bacula  /srv/backup/default
bscan: butil.c:289-0 Using device: "/srv/backup/default" for reading.
23-Dec 14:50 bscan JobId 0: Ready to read from volume "volume-default-0001" on file device "Default-Device" (/srv/backup/default).
bscan: bscan.c:293-0 Using Database: bacula, User: bacula
bscan: bscan.c:440-0 Pool record for Default-Pool found in DB.
bscan: bscan.c:454-0 Pool type "Backup" is OK.
bscan: bscan.c:464-0 Media record for volume-default-0001 found in DB.
bscan: bscan.c:482-0 Media type "File" is OK.
bscan: bscan.c:492-0 VOL_LABEL: OK for Volume: volume-default-0001
bscan: bscan.c:510-0 SOS_LABEL: Found Job record for JobId: 1
bscan: bscan.c:689-0 32,768 file records. At file:blk=0:213,083,178 bytes=212,923,651
bscan: bscan.c:510-0 SOS_LABEL: Found Job record for JobId: 2
23-Dec 14:50 bscan JobId 0: End of Volume at file 0 on device "Default-Device" (/srv/backup/default), Volume "volume-default-0001"
23-Dec 14:50 bscan JobId 0: End of all volumes.
bscan: bscan.c:643-0 End of all Volumes. VolFiles=0 VolBlocks=0 VolBytes=236,303,052
Records would have been added or updated in the catalog:
      1 Media
      1 Pool
      2 Job
  36808 File

As we can see here we have some information got from the volume, so now we need to update the catalog with that information.

bscan -s -m -v -V volume-default-0001 -u bacula -P "4e4fb461c26dd94eb4f4354bfca14222" -h localhost -n bacula  /srv/backup/default
bscan: butil.c:289-0 Using device: "/srv/backup/default" for reading.
23-Dec 15:04 bscan JobId 0: Ready to read from volume "volume-default-0001" on file device "Default-Device" (/srv/backup/default).
bscan: bscan.c:293-0 Using Database: bacula, User: bacula
bscan: bscan.c:440-0 Pool record for Default-Pool found in DB.
bscan: bscan.c:454-0 Pool type "Backup" is OK.
bscan: bscan.c:967-0 Created Media record for Volume: volume-default-0001
bscan: bscan.c:482-0 Media type "File" is OK.
bscan: bscan.c:492-0 VOL_LABEL: OK for Volume: volume-default-0001
bscan: bscan.c:1038-0 Created Client record for Client: bacula-fd
bscan: bscan.c:1121-0 Created new JobId=1 record for original JobId=1
bscan: bscan.c:689-0 32,768 file records. At file:blk=0:213,083,178 bytes=212,923,458
bscan: bscan.c:1064-0 Created FileSet record "Default-Linux"
bscan: bscan.c:1179-0 Updated Job termination record for JobId=1 Level=Full TermStat=T
bscan: bscan.c:1271-0 Created JobMedia record JobId 1, MediaId 1
bscan: bscan.c:1038-0 Created Client record for Client: bacula-fd
bscan: bscan.c:1121-0 Created new JobId=2 record for original JobId=2
bscan: bscan.c:1064-0 Created FileSet record "Catalog"
bscan: bscan.c:1179-0 Updated Job termination record for JobId=2 Level=Full TermStat=T
bscan: bscan.c:1271-0 Created JobMedia record JobId 2, MediaId 1
23-Dec 15:06 bscan JobId 0: End of Volume at file 0 on device "Default-Device" (/srv/backup/default), Volume "volume-default-0001"
bscan: bscan.c:988-0 Updated Media record at end of Volume: volume-default-0001
23-Dec 15:06 bscan JobId 0: End of all volumes.
bscan: bscan.c:988-0 Updated Media record at end of Volume: volume-default-0001
bscan: bscan.c:643-0 End of all Volumes. VolFiles=0 VolBlocks=0 VolBytes=236,302,859
Records added or updated in the catalog:
      1 Media
      1 Pool
      2 Job
  36808 File

Now we can take a look at bconsole and make sure that the volume information as loaded sucessfully.

bconsole 
Connecting to Director 127.0.0.1:9101
1000 OK: 1 bacula-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*list jobs
Automatically selected Catalog: Catalogo
Using Catalog "Catalogo"
+-------+-----------------+---------------------+------+-------+----------+-------------+-----------+
| jobid | name            | starttime           | type | level | jobfiles | jobbytes    | jobstatus |
+-------+-----------------+---------------------+------+-------+----------+-------------+-----------+
|     1 | Director-Backup | 2015-12-23 11:50:16 | B    | F     |   36,807 | 232,525,437 | T         |
|     2 | Catalog-Backup  | 2015-12-23 12:46:26 | B    | F     |        1 |   1,617,878 | T         |
+-------+-----------------+---------------------+------+-------+----------+-------------+-----------+

As we can see we have the information imported from the volume.

Another option to recreate the catalog is import the bacula.sql file that is create everyday.

If you are using PostgreSQL you can use the following command lines to import the file.

Let's export the password, otherwise we will need to put the password many times throughout the process.

export PGPASSWORD="4e4fb461c26dd94eb4f4354bfca14222"

Now we need to import the file.

psql -h localhost -U bacula -d bacula -f bacula.sql

If you are using MySQL you can use the following command lines to import the file.

Now we need to import the file.

mysql -h localhost -U bacula -D bacula -p < bacula.sql

Note: Something that I want to be crystal clear is if you are using cryptografy to sign the backup you will need to have this files to restore the backup otherwise you will get something like this when you are running the restore

*mes
23-Dec 14:23 bacula-dir JobId 4: Start Restore Job Restore-Files.2015-12-23_14.23.43_03
23-Dec 14:23 bacula-dir JobId 4: Using Device "Default-Device" to read.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-sd JobId 4: Ready to read from volume "volume-default-0001" on file device "Default-Device" (/srv/backup/default).
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.
23-Dec 14:23 bacula-fd JobId 4: Error: Missing private key required to decrypt encrypted backup data.

So make sure that you restored the configuration files and the keys.

References