CedarBackup3.actions package — Cedar Backup v3 documentation

# CedarBackup3.actions package

Cedar Backup actions.

This package code related to the offical Cedar Backup actions (collect, stage, store, purge, rebuild, and validate).

The action modules consist of mostly “glue” code that uses other lower-level functionality to actually implement a backup. There is one module for each high-level backup action, plus a module that provides shared constants.

All of the public action function implement the Cedar Backup Extension Architecture Interface, i.e. the same interface that extensions implement.

author: | Kenneth J. Pronovici <pronovic@ieee.org>

## Submodules

## CedarBackup3.actions.collect module

Implements the standard ‘collect’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.collect.` `executeCollect`(configPath, options, config)

  Executes the collect backup action.

  Note: When the collect action is complete, we will write a collect indicator to the collect directory, so it’s obvious that the collect action has completed. The stage process uses this indicator to decide whether a peer is ready to be staged.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – Under many generic error conditions `TarError` – If there is a problem creating a tar file

## CedarBackup3.actions.constants module

Provides common constants used by standard actions. :author: Kenneth J. Pronovici <pronovic@ieee.org>

## CedarBackup3.actions.initialize module

Implements the standard ‘initialize’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.initialize.` `executeInitialize`(configPath, options, config)

  Executes the initialize action.

  The initialize action initializes the media currently in the writer device so that Cedar Backup can recognize it later. This is an optional step; it’s only required if checkMedia is set on the store configuration.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration

## CedarBackup3.actions.purge module

Implements the standard ‘purge’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.purge.` `executePurge`(configPath, options, config)

  Executes the purge backup action.

  For each configured directory, we create a purge item list, remove from the list anything that’s younger than the configured retain days value, and then purge from the filesystem what’s left.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – Under many generic error conditions

## CedarBackup3.actions.rebuild module

Implements the standard ‘rebuild’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.rebuild.` `executeRebuild`(configPath, options, config)

  Executes the rebuild backup action.

  This function exists mainly to recreate a disc that has been “trashed” due to media or hardware problems. Note that the “stage complete” indicator isn’t checked for this action.

  Note that the rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

## CedarBackup3.actions.stage module

Implements the standard ‘stage’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.stage.` `executeStage`(configPath, options, config)

  Executes the stage backup action.

  Note: The daily directory is derived once and then we stick with it, just in case a backup happens to span midnite.

  Note: As portions of the stage action is complete, we will write various indicator files so that it’s obvious what actions have been completed. Each peer gets a stage indicator in its collect directory, and then the master gets a stage indicator in its daily staging directory. The store process uses the master’s stage indicator to decide whether a directory is ready to be stored. Currently, nothing uses the indicator at each peer, and it exists for reference only.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

## CedarBackup3.actions.store module

Implements the standard ‘store’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org> :author: Dmitry Rutsky <rutsky@inbox.ru>

`CedarBackup3.actions.store.` `consistencyCheck`(config, stagingDirs)

  Runs a consistency check against media in the backup device.

  It seems that sometimes, it’s possible to create a corrupted multisession disc (i.e. one that cannot be read) although no errors were encountered while writing the disc. This consistency check makes sure that the data read from disc matches the data that was used to create the disc.

  The function mounts the device at a temporary mount point in the working directory, and then compares the indicated staging directories in the staging directory and on the media. The comparison is done via functionality in `filesystem.py`.

  If no exceptions are thrown, there were no problems with the consistency check. A positive confirmation of “no problems” is also written to the log with `info` priority.

  @warning: The implementation of this function is very UNIX-specific.

  Parameters: | config – Config object stagingDirs – Dictionary mapping directory path to date suffix
  Raises: | `ValueError` – If the two directories are not equivalent `IOError` – If there is a problem working with the media

`CedarBackup3.actions.store.` `executeStore`(configPath, options, config)

  Executes the store backup action.

  Note: The rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.

  Note: When the store action is complete, we will write a store indicator to the daily staging directory we used, so it’s obvious that the store action has completed.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

`CedarBackup3.actions.store.` `writeImage`(config, newDisc, stagingDirs)

  Builds and writes an ISO image containing the indicated stage directories.

  The generated image will contain each of the staging directories listed in `stagingDirs`. The directories will be placed into the image at the root by date, so staging directory `/opt/stage/2005/02/10` will be placed into the disc at `/2005/02/10`.

  Note: This function is implemented in terms of `writeImageBlankSafe`. The `newDisc` flag is passed in for both `rebuildMedia` and `todayIsStart`.

  Parameters: | config – Config object newDisc – Indicates whether the disc should be re-initialized stagingDirs – Dictionary mapping directory path to date suffix
  Raises: | `ValueError` – Under many generic error conditions `IOError` – If there is a problem writing the image to disc

`CedarBackup3.actions.store.` `writeImageBlankSafe`(config, rebuildMedia, todayIsStart, blankBehavior, stagingDirs)

  Builds and writes an ISO image containing the indicated stage directories.

  The generated image will contain each of the staging directories listed in `stagingDirs`. The directories will be placed into the image at the root by date, so staging directory `/opt/stage/2005/02/10` will be placed into the disc at `/2005/02/10`. The media will always be written with a media label specific to Cedar Backup.

  This function is similar to `writeImage`, but tries to implement a smarter blanking strategy.

  First, the media is always blanked if the `rebuildMedia` flag is true. Then, if `rebuildMedia` is false, blanking behavior and `todayIsStart` come into effect:

      If no blanking behavior is specified, and it is the start of the week,
      the disc will be blanked

      If blanking behavior is specified, and either the blank mode is "daily"
      or the blank mode is "weekly" and it is the start of the week, then
      the disc will be blanked if it looks like the weekly backup will not
      fit onto the media.

      Otherwise, the disc will not be blanked

  How do we decide whether the weekly backup will fit onto the media? That is what the blanking factor is used for. The following formula is used:

      will backup fit? = (bytes available / (1 + bytes required) <= blankFactor

  The blanking factor will vary from setup to setup, and will probably require some experimentation to get it right.

  Parameters: | config – Config object rebuildMedia – Indicates whether media should be rebuilt todayIsStart – Indicates whether today is the starting day of the week blankBehavior – Blank behavior from configuration, or `None` to use default behavior stagingDirs – Dictionary mapping directory path to date suffix
  Raises: | `ValueError` – Under many generic error conditions `IOError` – If there is a problem writing the image to disc

`CedarBackup3.actions.store.` `writeStoreIndicator`(config, stagingDirs)

  Writes a store indicator file into staging directories.

  The store indicator is written into each of the staging directories when either a store or rebuild action has written the staging directory to disc.

  Parameters: | config – Config object stagingDirs – Dictionary mapping directory path to date suffix

## CedarBackup3.actions.util module

Implements action-related utilities :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.util.` `buildMediaLabel`()

  Builds a media label to be used on Cedar Backup media. :returns: Media label as a string

`CedarBackup3.actions.util.` `checkMediaState`(storeConfig)

  Checks state of the media in the backup device to confirm whether it has been initialized for use with Cedar Backup.

  We can tell whether the media has been initialized by looking at its media label. If the media label starts with MEDIA_LABEL_PREFIX, then it has been initialized.

  The check varies depending on whether the media is rewritable or not. For non-rewritable media, we also accept a `None` media label, since this kind of media cannot safely be initialized.

  Parameters: | storeConfig – Store configuration
  Raises: | `ValueError` – If media is not initialized

`CedarBackup3.actions.util.` `createWriter`(config)

  Creates a writer object based on current configuration.

  This function creates and returns a writer based on configuration. This is done to abstract action functionality from knowing what kind of writer is in use. Since all writers implement the same interface, there’s no need for actions to care which one they’re working with.

  Currently, the `cdwriter` and `dvdwriter` device types are allowed. An exception will be raised if any other device type is used.

  This function also checks to make sure that the device isn’t mounted before creating a writer object for it. Experience shows that sometimes if the device is mounted, we have problems with the backup. We may as well do the check here first, before instantiating the writer.

  Parameters: | config – Config object
  Returns: | Writer that can be used to write a directory to some media
  Raises: | `ValueError` – If there is a problem getting the writer `IOError` – If there is a problem creating the writer object

`CedarBackup3.actions.util.` `findDailyDirs`(stagingDir, indicatorFile)

  Returns a list of all daily staging directories that do not contain the indicated indicator file.

  Parameters: | stagingDir – Configured staging directory (config.targetDir)
  Returns: | List of absolute paths to daily staging directories

`CedarBackup3.actions.util.` `getBackupFiles`(targetDir)

  Gets a list of backup files in a target directory.

  Files that match INDICATOR_PATTERN (i.e. `"cback.store"`, `"cback.stage"`, etc.) are assumed to be indicator files and are ignored.

  Parameters: | targetDir – Directory to look in
  Returns: | List of backup files in the directory
  Raises: | `ValueError` – If the target directory does not exist

`CedarBackup3.actions.util.` `initializeMediaState`(config)

  Initializes state of the media in the backup device so Cedar Backup can recognize it.

  This is done by writing an mostly-empty image (it contains a “Cedar Backup” directory) to the media with a known media label.

  Note: Only rewritable media (CD-RW, DVD+RW) can be initialized. It doesn’t make any sense to initialize media that cannot be rewritten (CD-R, DVD+R), since Cedar Backup would then not be able to use that media for a backup.

  Parameters: | config – Cedar Backup configuration
  Raises: | `ValueError` – If media could not be initialized `ValueError` – If the configured media type is not rewritable

`CedarBackup3.actions.util.` `writeIndicatorFile`(targetDir, indicatorFile, backupUser, backupGroup)

  Writes an indicator file into a target directory. :param targetDir: Target directory in which to write indicator :param indicatorFile: Name of the indicator file :param backupUser: User that indicator file should be owned by :param backupGroup: Group that indicator file should be owned by

  Raises: | `IOException` – If there is a problem writing the indicator file

## CedarBackup3.actions.validate module

Implements the standard ‘validate’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

`CedarBackup3.actions.validate.` `executeValidate`(configPath, options, config)

  Executes the validate action.

  This action validates each of the individual sections in the config file. This is a “runtime” validation. The config file itself is already valid in a structural sense, so what we check here that is that we can actually use the configuration without any problems.

  There’s a separate validation function for each of the configuration sections. Each validation function returns a true/false indication for whether configuration was valid, and then logs any configuration problems it finds. This way, one pass over configuration indicates most or all of the obvious problems, rather than finding just one problem at a time.

  Any reported problems will be logged at the ERROR level normally, or at the INFO level if the quiet flag is enabled.

  Parameters: | configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
  Raises: | `ValueError` – If some configuration value is invalid