# Debian doc-base Manual

## About `doc-base`

Some time ago, there was a big discussion on the Debian mailing lists about the preferred documentation format in Debian. The discussion showed clearly that people have very different opinions on that topic and thus, we'll have to implement a flexible solution.

The `doc-base` package tries to implement such a flexible solution: Every Debian package that provides online documentation (other than manual pages) will register these documents to `doc-base` via the `install-docs` script (see registering-documents) at installation time and de-register the manuals again when the package is removed.

Since all manuals will eventually be registered, `doc-base` can also be used to solve another outstanding problem: Debian currently has three different online documentation systems, `doc-central`, `dwww` and `dhelp`. Each system has advantages and disadvantages, thus `doc-base` supports all of them. The system administrator can choose which implementation he/she prefers.

Additionally `doc-base` registers the documentation with scrollkeper, thus making it possible to view the manuals using programs like `yelp` or `khelpcenter`.

## The packages interface

### Introduction

Each Debian package that installs online manuals (in any format) should register its manuals to `doc-base`. This is done by installing a `doc-base` control file (see control-files) and calling `install-docs` from the `postinst` script (see registering-documents).

### Document IDs

Each document that is registered to `doc-base` must have a unique document ID.

The document ID is usually taken from the document's title or from the package name. Here are a few examples:

    DOCID                  Title
    ---------------------- ----------------------------
    debian-policy          Debian Policy Manual
    developers-reference   Debian Developers Reference
    doc-base               Debian doc-base Manual
    emacs-manual           GNU Emacs Manual

Legal characters for the document ID are lower case letters (a-z), digits (0-9), plus (+) or minus (-) signs, and dots (.) (the same characters allowed in package names).

### Control Files

For each piece online documentation, `doc-base` needs a control file that describes the documentation and the documentation file formats that are provided initially.

#### Example

Here is an example of a control file:

    Document: doc-base
    Title: Debian doc-base Manual
    Author: Christian Schwarz
    Abstract: This manual describes what doc-base is
     and how it can be used to
     manage online manuals on Debian systems.
    Section: Debian

    Format: DebianDoc-SGML
    Files: /usr/share/doc/doc-base/doc-base.sgml.gz

    Format: Text
    Files: /usr/share/doc/doc-base/doc-base.txt.gz

    Format: HTML
    Index: /usr/share/doc/doc-base/doc-base.html/index.html
    Files: /usr/share/doc/doc-base/doc-base.html/*.html

If the `doc-base` package provided necessary files in other formats, it would be possible to add more sections at the end of the control file:

    Format: PDF
    Files: /usr/share/doc-base/doc-base.pdf

    Format: PostScript
    Files: /usr/share/doc-base/doc-base.ps.gz

    Format: DVI
    Files: /usr/share/doc-base/doc-base.dvi.gz

    Format: Info
    Index: /usr/share/info/doc-base.info.gz
    Files: /usr/share/info/doc-base.info*.gz

#### Syntax of the control file

As you can see from the above example, the syntax -- as is the whole design of `doc-base` -- is heavily influenced by dpkg. This is important since every maintainer will have to work with `doc-base` and thus, it should be simple to remember the basic ideas.

The syntax of the control file is simple:

- The file consist of

  - exactly one main section providing base information about the registered manual (see main-section below);

  - one or more format sections (see format-sections) containing pointers to the registered documentation files.

- Successive sections must be separated with empty lines.

- Non-empty lines use a ``field-name: value`' syntax.

- The field names are case-insensitive.

- The field values are case-sensitive (except for the Format field).

- Field values may be wrapped over several lines by making the first character of subsequent lines a space.

  - If a multi-line value should contain an empty line, a single dot (.) must be placed in the second column.

  - If the Abstract field value should contain lines displayed verbatim, the lines must begin with two spaces.

- The file should be encoded in UTF-8.

##### The main section

The first section of the control file describes the document. The following fields are available:

Document

- Document ID, required field; should be the first field.

Title

- Title of the document; required field.

Author

- Author(s) of the document; optional field.

Abstract

- Short paragraph giving an overview of the document; optional but recommended field.

Section

- Section where the document belongs; see section-field. Required field.

##### Format sections

The next sections describe the different formats for the provided document which is described in the first section. The following fields are available:

Format

- Format for the document. Required field. The following formats are recognised:

  - `HTML`,

  - `Text`,

  - `PDF`,

  - `PostScript`,

  - `Info`,

  - `DVI`,

  - and `DebianDoc-SGML`.

  The values of this field are case-insensitive (e.g. both `Text` and `text` are valid).

Index

- Index or top-level file for this document format. Only applies to document formats `HTML` and `Info`, and required if the format is `HTML` or `Info`.

  This field has to contain the absolute file name of the main page of the document. This file will be specified as the front page link when the document is registered.

Files

- Space separated list of filenames or POSIX shell globs (i.e. `*`, `?`, and `[]` meta-characters) representing the files which constitute the documentation in this format. Required field.

There must be at least one such a section. If there are more, each of them must register files in different formats (e.g. having two `Format: HTML` sections in one control file is not allowed).

Except for the `Info` format the files referred in both Index and Files fields should be placed somewhere under the `/usr/share/doc` hierarchy. If for some reason it's not possible, then the registering package should provide a symbolic link pointing from the above hierarchy to the real files and register its documentation through the link, allowing `doc-base`, `dhelp`, or `dwww` packages to actually handle the documentation. Of course, files in the `Info` format should be located in the `/usr/share/info` directory.

#### The `section` field

The `section` field holds a slash-separated list of hierarchical sections components. The hierarchy is mostly based on the sections outlined in chapter 2.1 of the Debian Menu Policy, however the top-level `Applications` component was removed and a few `doc-base`-specific sections were added.

The full section list is presented below.

`Accessibility`

- Documentation of tools to aid people with disabilities or for machines lacking usual input devices.

`Amateur Radio`

- Anything relating to HAM radio.

`Data Management`

- Interactive database programs, collection managers, address books, bibliography tools, etc.

`Debian`

- Documentation of Debian specific tools, policies, etc.

`Editors`

- Documentation of editors, other than office word processors, for text-based information.

`Education`

- Educational and training softwares.

`Emulators`

- Software that allows you to run non-native software or more than one OS at a time.

`File Management`

- Tools for file management, archiving, searching, CD/DVD burning, backup, etc.

`Games`

- Games and recreations. Entries should be placed in appropriate subsection.

  `Games/Action`

- Games that involve a lot of action and require fast reflexes.

`Games/Adventure`

- Role playing and adventure games, interactive movies and stories, etc.

`Games/Blocks`

- Tetris-like games involving falling blocks.

`Games/Board`

- Games played on a board.

`Games/Card`

- Games involving a deck of cards.

`Games/Puzzles`

- Tests of ingenuity and logic.

`Games/Simulation`

- Simulations of the real world in all detail and complexity.

`Games/Strategy`

- Games involving long-term strategic thinking.

`Games/Tools`

- Server browsers, configurators, editors, and other game-related tools that are not games themselves.

`Games/Toys`

- Amusements, eye-candy, entertaining demos, screen hacks (screen-savers), etc.

`Graphics`

- 2D and 3D graphics manipulation software.

`Help`

- Documentation of programs that provide user documentation.

  `Help/Books`

- Books.

`Help/FAQ`

- Frequently Asked Questions.

`Help/HOWTO`

- Various HOWTOs.

`Help/RFC`

- RFCs

`Help/Standards`

- Standards

`Mobile Devices`

- Software that allows you to interface with mobile devices (phones, PDAs, etc.).

`Network`

- Network related software. This is a two-level section, do not put entries directly here.

  `Network/Communication`

- Mail, USENET news, chat, instant messaging, IP telephony, video conferencing software, etc.

`Network/File Transfer`

- File transfer software such as download managers, FTP clients, P2P clients, etc.

`Network/Monitoring`

- Network monitoring software.

`Network/Remote Access`

- Tools for remotely managing of computer.

`Network/Web Browsing`

- Web browsers, tools for offline browsing, etc.

`Network/Web News`

- Web feed (RSS, Atom, etc.) and podcast aggregators.

`Office`

- Office suites, word processors, spreadsheets, CRM, ERP, financial software, etc.

`Programming`

- IDEs, debuggers, compilers, APIs, libraries, programming languages. Documentation related to only one specific language should be put in subsection named like the language, for example:

  - `Programming/C`

  - `Programming/C++`

  - `Programming/Java`

  - `Programming/OCaml`

  - `Programming/Perl`

  - `Programming/Python`

  - `Programming/Ruby`

`Project Management`

- Timetable managers, group task trackers, bug tracking software, etc.

`Science`

- Documentation of scientific and engineering-related software. Please use appropriate subsection.

  `Science/Astronomy`

- Astronomy-related software.

`Science/Biology`

- Biology-related software.

`Science/Chemistry`

- Chemistry-related software.

`Science/Data Analysis`

- Software designed for processing, extracting, and presenting generic scientific data.

`Science/Electronics`

- Circuit design tools, simulators and assemblers for microprocessors, etc.

`Science/Engineering`

- CAD, UML tools, diagram-drawing and other engineering-related software.

`Science/Geoscience`

- Geoscience-related software.

`Science/Mathematics`

- Mathematics-related software.

`Science/Medicine`

- Medicine-related software.

`Science/Physics`

- Physics-related software.

`Science/Social`

- Social sciences-related software.

`Screen`

- Programs that affect the whole screen.

  `Screen/Saving`

- Tools for blanking the screen. Entries of screen hacks and configuration GUIs should go to other appropriate sections.

`Screen/Locking`

- Tools for locking the screen.

`Shells`

- Various shells to be used inside a terminal emulator.

`Sound`

- Sound players, editors, and rippers/recorders.

`System`

- System related software. Place entries in one of she subsections.

  `System/Administration`

- Administrative and system configuration utilities, also tools for personal user settings.

`System/Hardware`

- Tools for manipulating specific hardware, especially non-standard laptop hardware.

`System/Language Environment`

- This section is reserved for language-env as a special case.

`System/Monitoring`

- System information and monitoring tools, log viewers, etc.

`System/Package Management`

- Package managers and related tools.

`System/Security`

- Security, cryptography and privacy related software, antiviruses, tools to track and report bugs, etc.

`Terminal Emulators`

- Graphical terminal emulators.

`Text`

- Text oriented tools like dictionaries, OCR, translation, text analysis software, etc.

`TV and Radio`

- TV-in, TV-out, FM radio, teletext browsers, etc.

`Typesetting`

- Software for typesetting text and graphics from structured input files like LaTeX or docbook sources, database exports etc.

`Viewers`

- Software for viewing images, documents and other (non-video) media.

`Video`

- Video players, editors, and rippers/recorders.

`Web Development`

- Software for web site editing, web programming, and site administration.

`Window Managers`

- X window managers.

### Registering Documents With `doc-base`

In order to register a piece of online documentation to `doc-base`, all the package needs to do is installing the control file (see control-files) as file `/usr/share/doc-base/`document-id``.

Further processing of the control file is handled by a `dpkg` trigger (cf. `/usr/share/doc/dpkg/triggers.txt.gz`) provided by `doc-base`. The trigger will call `install-docs` to generate `/var/lib/doc-base/documents/`<document-id>`` file and register the online manuals to `dwww`, `dhelp`, and `scrollkeeper` when the package is installed, and de-register the manuals when the package is removed.

### doc-base 0.8.x features and incompatibilities

#### Splitting control files over multiple binary packages

Since version 0.8.7 is it possible to provide documents with the same document-id by more than one binary package. All such documents with be merged together and the merged document will be generated in `/var/lib/doc-base/documents/document-id` file. This feature can be useful in cases when the same documentation, but in different formats, is provided by two binary packages.

For example the `foo-text` package could install the `/usr/share/doc-base/foo-text` file with the following contents:

    Document: foo
    Title: This is foo
    Author: John Foo <foo@foo.net>
    Abstract: Description of foo
    Section: Text

    Format: text
    Files: /usr/share/foo-text/foo.txt.gz

and the `foo-html` package could install the following `/usr/share/doc-base/foo-html` file:

    Document: foo
    Title: This is foo
    Author: John Foo <foo@foo.net>
    Abstract: Description of foo
    Section: Text

    Format: HTML
    Index: /usr/share/foo-html/index.html
    Files: /usr/share/foo-html/*.html

When both packages are installed, `install-docs` will merge contents of the two files into `/var/lib/doc-base/documents/foo`:

    Document: foo
    Title: This is foo
    Author: John Foo <foo@foo.net>
    Abstract: Description of foo
    Section: Text

    Format: HTML
    Index: /usr/share/foo-html/index.html
    Files: /usr/share/foo-html/*.html

    Format: text
    Files: /usr/share/foo-text/foo.txt.gz

#### Registering local documents

Version 0.8.7 and furthers allow user to register local documentations. In order to do this local administrators need to create their own control file (see control-files, place it in the `/etc/doc-base/documents` directory, and then register it with

    install-docs -i /etc/doc-base/documents/<document-id>

Before removing the file, it should be de-registered with

    install-docs -r /etc/doc-base/documents/<document-id>

Since version 0.8.12

    install-docs --install-changed

may be used instead of the two above commands.

#### dpkg triggers

`doc-base` 0.8.11 and greater uses the `dpkg` triggers feature to register and de-register the documentation. There is no longer need to call `install-docs` from maintainer scripts.

### Checking Syntax of Control Files

With the new `--check` (`-c`) option of `install-docs` it is possible to check the control file:

    $ install-docs --check /usr/share/doc-base/doc-base
    /usr/share/doc-base/doc-base: No problems found

    $ install-docs -c /usr/share/doc-base/xlogmaster
    Error in `/usr/share/doc-base/xlogmaster', line 15: `Index' value missing for format info
    /usr/share/doc-base/xlogmaster: Fatal error found, the file won't be registered

    $ install-docs -c /usr/share/doc-base/MC-FAQ /usr/share/doc-base/gnu-privacy-handbook
    /usr/share/doc-base/MC-FAQ: 1 warning(s) or non-fatal error(s) found
    /usr/share/doc-base/gnu-privacy-handbook: 1 warning(s) or non-fatal error(s) found

More details about the warnings and non-fatal errors can be found using the `--verbose` (`-v`) option:

    $ install-docs -v -c /usr/share/doc-base/MC-FAQ /usr/share/doc-base/gnu-privacy-handbook
    Warning in `/usr/share/doc-base/MC-FAQ', line 1: invalid value of `Document' field
    /usr/share/doc-base/MC-FAQ: 1 warning(s) or non-fatal error(s) found
    Warning in `/usr/share/doc-base/gnu-privacy-handbook', line 12: file `/usr/share/doc/gnupg-doc/GNU_Privacy_Handbook/html/book1.html' does not exist
    /usr/share/doc-base/gnu-privacy-handbook: 1 warning(s) or non-fatal error(s) found

With the `--rootdir` option is possible to check non-installed packages:

    $ dpkg-deb -x autoclass_3.3.4-6_i386.deb AUTOCLASS_UNPACKED
    $ install-docs --rootdir AUTOCLASS_UNPACKED -vc AUTOCLASS_UNPACKED/usr/share/doc-base/*
    AUTOCLASS_UNPACKED/usr/share/doc-base/autoclass-results: No problems found
    AUTOCLASS_UNPACKED/usr/share/doc-base/autoclass-theory: No problems found

If the `--rootdir` option was omitted, `install-docs` would complain:

    Warning in `AUTOCLASS_UNPACKED/usr/share/doc-base/autoclass-results', line 20: file mask `/usr/share/doc/autoclass/kdd-95.pdf' does not match any files
    Warning in `AUTOCLASS_UNPACKED/usr/share/doc-base/autoclass-theory', line 20: file mask `/usr/share/doc/autoclass/tr-fia-90-12-7-01.pdf' does not match any files

## Getting information about installed documents

If you want to get information about the status of an installed manual, you can use the ``-s`' or ``--status`' option of `install-docs` followed by the document id:

    $ install-docs -s foo
    ---document-information---
    Document: foo
    Abstract: This manual is about foos, bars, and Debian.
    Author: Wile E. Coyote
    Section: Debian
    Title: Debian Foo's Manual

    ---format-description---
    Format: debiandoc-sgml
    Files: /usr/share/doc/foo/sgml/foo.sgml.gz

    ---format-description---
    Format: html
    Files: /usr/share/doc/foo/html-sgml/*.html
    Index: /usr/share/doc/foo/html-sgml/index.html

    ---status-information---
    Control-Files: /usr/share/doc-base/foo (changed: Tue Apr  8 23:09:24 2008)
    Scrollkeeper-omf-file: /var/lib/doc-base/omf/foo/foo-C.omf
    Scrollkeeper-sid: 99999999-0000-8888-0000-1234567890ab

`doc-base` always creates a `/var/lib/doc-base/documents/<document-id>` file when installing a document.

When a document is registered to `scrollkeeper`, `doc-base` will create ``*-C.omf`' file under the `/var/lib/doc-base/omf` directory. Name of the generated file is recorded in the Scrollkeeper-omf-file field. Lack of the field means the documents was not registered to the package.

## TODO List

### Roadmap for 0.9.* releases

- Internationalisation and po-debconf support. Unfortunately this would require merging Abstract and Title fields into one Description field. See Bug#171363, Bug#171375, and Bug#171378.

- Introduce some new fields, like Package (Bug#71955), or SortSkip (Bug#187590).

- Possibly allow documents to appear in multiple sections, see Steve M. Robbins' mail.

- Possibly index documentation with swish++ that could be used by frontends like `dhelp` or `dwww`.

### Old TODO entries

- Policy: document the `doc-base` document registration file format separately (or SUBDOC it) as a proposed Debian documentation system policy.

- Policy: define a first-cut standard as the document hierarchy.

- Documentation update: show clean and minimal use of `install-docs` from maintainer script.

- It is extremely difficult to deal coherently with a misnamed control file, or a mismatch between a control file and the document field. This hit me in the transition between `doc-base` 0.4 to 0.5 (in 0.4 I had added, in a file install-docs-man, a document ID named install-doc-man). Something needs to be done about that.

- Automated format conversion, including user preferences. (For example, convert texinfo source to GNU info or PostScript, optionally compress or remove HTML manuals which are also available in GNU info format, etc. etc.)