CedarBackup3.tools package — Cedar Backup v3 documentation

# CedarBackup3.tools package

Official Cedar Backup Tools

This package provides official Cedar Backup tools. Tools are things that feel a little like extensions, but don’t fit the normal mold of extensions. For instance, they might not be intended to run from cron, or might need to interact dynamically with the user (i.e. accept user input).

Tools are usually scripts that are run directly from the command line, just like the main `cback3` script. Like the `cback3` script, the majority of a tool is implemented in a .py module, and then the script just invokes the module’s `cli()` function. The actual scripts for tools are distributed in the util/ directory.

author: | Kenneth J. Pronovici <pronovic@ieee.org>

## Submodules

## CedarBackup3.tools.amazons3 module

Synchonizes a local directory with an Amazon S3 bucket.

No configuration is required; all necessary information is taken from the command-line. The only thing configuration would help with is the path resolver interface, and it doesn’t seem worth it to require configuration just to get that.

author: | Kenneth J. Pronovici <pronovic@ieee.org>

class `CedarBackup3.tools.amazons3.` `Options`(argumentList=None, argumentString=None, validate=True)

  Bases: `object`

  Class representing command-line options for the cback3-amazons3-sync script.

  The `Options` class is a Python object representation of the command-line options of the cback3-amazons3-sync script.

  The object representation is two-way: a command line string or a list of command line arguments can be used to create an `Options` object, and then changes to the object can be propogated back to a list of command-line arguments or to a command-line string. An `Options` object can even be created from scratch programmatically (if you have a need for that).

  There are two main levels of validation in the `Options` class. The first is field-level validation. Field-level validation comes into play when a given field in an object is assigned to or updated. We use Python’s `property` functionality to enforce specific validations on field values, and in some places we even use customized list classes to enforce validations on list members. You should expect to catch a `ValueError` exception when making assignments to fields if you are programmatically filling an object.

  The second level of validation is post-completion validation. Certain validations don’t make sense until an object representation of options is fully “complete”. We don’t want these validations to apply all of the time, because it would make building up a valid object from scratch a real pain. For instance, we might have to do things in the right order to keep from throwing exceptions, etc.

  All of these post-completion validations are encapsulated in the `Options.validate` method. This method can be called at any time by a client, and will always be called immediately after creating a `Options` object from a command line and before exporting a `Options` object back to a command line. This way, we get acceptable ease-of-use but we also don’t accept or emit invalid command lines.

  Note: Lists within this class are “unordered” for equality comparisons.

  `__init__`(argumentList=None, argumentString=None, validate=True)

    Initializes an options object.

    If you initialize the object without passing either `argumentList` or `argumentString`, the object will be empty and will be invalid until it is filled in properly.

    No reference to the original arguments is saved off by this class. Once the data has been parsed (successfully or not) this original information is discarded.

    The argument list is assumed to be a list of arguments, not including the name of the command, something like `sys.argv[1:]`. If you pass `sys.argv` instead, things are not going to work.

    The argument string will be parsed into an argument list by the `util.splitCommandLine` function (see the documentation for that function for some important notes about its limitations). There is an assumption that the resulting list will be equivalent to `sys.argv[1:]`, just like `argumentList`.

    Unless the `validate` argument is `False`, the `Options.validate` method will be called (with its default arguments) after successfully parsing any passed-in command line. This validation ensures that appropriate actions, etc. have been specified. Keep in mind that even if `validate` is `False`, it might not be possible to parse the passed-in command line, so an exception might still be raised.

    Note: The command line format is specified by the `_usage` function. Call `_usage` to see a usage statement for the cback3-amazons3-sync script.

    Note: It is strongly suggested that the `validate` option always be set to `True` (the default) unless there is a specific need to read in invalid command line arguments.

    Parameters: | argumentList (List of arguments, i.e. `sys.argv`) – Command line for a program argumentString (String, i.e. "cback3-amazons3-sync --verbose stage store") – Command line for a program validate (Boolean true/false) – Validate the command line after parsing it
    Raises: | `getopt.GetoptError` – If the command-line arguments could not be parsed `ValueError` – If the command-line arguments are invalid

  `buildArgumentList`(validate=True)

    Extracts options into a list of command line arguments.

    The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument list. Besides that, the argument list is normalized to use the long option names (i.e. –version rather than -V). The resulting list will be suitable for passing back to the constructor in the `argumentList` parameter. Unlike `buildArgumentString`, string arguments are not quoted here, because there is no need for it.

    Unless the `validate` parameter is `False`, the `Options.validate` method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument list will not be extracted.

    Note: It is strongly suggested that the `validate` option always be set to `True` (the default) unless there is a specific need to extract an invalid command line.

    Parameters: | validate (Boolean true/false) – Validate the options before extracting the command line
    Returns: | List representation of command-line arguments
    Raises: | `ValueError` – If options within the object are invalid

  `buildArgumentString`(validate=True)

    Extracts options into a string of command-line arguments.

    The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument string. Besides that, the argument string is normalized to use the long option names (i.e. –version rather than -V) and to quote all string arguments with double quotes (`"`). The resulting string will be suitable for passing back to the constructor in the `argumentString` parameter.

    Unless the `validate` parameter is `False`, the `Options.validate` method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument string will not be extracted.

    Note: It is strongly suggested that the `validate` option always be set to `True` (the default) unless there is a specific need to extract an invalid command line.

    Parameters: | validate (Boolean true/false) – Validate the options before extracting the command line
    Returns: | String representation of command-line arguments
    Raises: | `ValueError` – If options within the object are invalid

  `debug`

    Command-line debug (`-d,--debug`) flag.

  `diagnostics`

    Command-line diagnostics (`-D,--diagnostics`) flag.

  `help`

    Command-line help (`-h,--help`) flag.

  `ignoreWarnings`

    Command-line ignoreWarnings (`-w,--ignoreWarnings`) flag

  `logfile`

    Command-line logfile (`-l,--logfile`) parameter.

  `mode`

    Command-line mode (`-m,--mode`) parameter.

  `output`

    Command-line output (`-O,--output`) flag.

  `owner`

    Command-line owner (`-o,--owner`) parameter, as tuple `(user,group)`.

  `quiet`

    Command-line quiet (`-q,--quiet`) flag.

  `s3BucketUrl`

    Command-line s3BucketUrl, target of sync.

  `sourceDir`

    Command-line sourceDir, source of sync.

  `stacktrace`

    Command-line stacktrace (`-s,--stack`) flag.

  `validate`()

    Validates command-line options represented by the object.

    Unless `--help` or `--version` are supplied, at least one action must be specified. Other validations (as for allowed values for particular options) will be taken care of at assignment time by the properties functionality.

    Note: The command line format is specified by the `_usage` function. Call `_usage` to see a usage statement for the cback3-amazons3-sync script.

    Raises: | `ValueError` – If one of the validations fails

  `verbose`

    Command-line verbose (`-b,--verbose`) flag.

  `verifyOnly`

    Command-line verifyOnly (`-v,--verifyOnly`) flag.

  `version`

    Command-line version (`-V,--version`) flag.

`CedarBackup3.tools.amazons3.` `cli`()

  Implements the command-line interface for the `cback3-amazons3-sync` script.

  Essentially, this is the “main routine” for the cback3-amazons3-sync script. It does all of the argument processing for the script, and then also implements the tool functionality.

  This function looks pretty similiar to `CedarBackup3.cli.cli()`. It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.

  A different error code is returned for each type of failure:

  > - `1`: The Python interpreter version is < 3.4
  > - `2`: Error processing command-line arguments
  > - `3`: Error configuring logging
  > - `5`: Backup was interrupted with a CTRL-C or similar
  > - `6`: Error executing other parts of the script

  Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.

  Returns: | Error code as described above

## CedarBackup3.tools.span module

Spans staged data among multiple discs

This is the Cedar Backup span tool. It is intended for use by people who stage more data than can fit on a single disc. It allows a user to split staged data among more than one disc. It can’t be an extension because it requires user input when switching media.

Most configuration is taken from the Cedar Backup configuration file, specifically the store section. A few pieces of configuration are taken directly from the user.

author: | Kenneth J. Pronovici <pronovic@ieee.org>

class `CedarBackup3.tools.span.` `SpanOptions`(argumentList=None, argumentString=None, validate=True)

  Bases: `CedarBackup3.cli.Options`

  Tool-specific command-line options.

  Most of the cback3 command-line options are exactly what we need here – logfile path, permissions, verbosity, etc. However, we need to make a few tweaks since we don’t accept any actions.

  Also, a few extra command line options that we accept are really ignored underneath. I just don’t care about that for a tool like this.

  `validate`()

    Validates command-line options represented by the object. There are no validations here, because we don’t use any actions. :raises: `ValueError` – If one of the validations fails

`CedarBackup3.tools.span.` `cli`()

  Implements the command-line interface for the `cback3-span` script.

  Essentially, this is the “main routine” for the cback3-span script. It does all of the argument processing for the script, and then also implements the tool functionality.

  This function looks pretty similiar to `CedarBackup3.cli.cli()`. It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.

  A different error code is returned for each type of failure:

  > - `1`: The Python interpreter version is < 3.4
  > - `2`: Error processing command-line arguments
  > - `3`: Error configuring logging
  > - `4`: Error parsing indicated configuration file
  > - `5`: Backup was interrupted with a CTRL-C or similar
  > - `6`: Error executing other parts of the script

  Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.

  Returns: | Error code as described above