SimGrid: Logging support

SimGrid 3.10 Versatile Simulation of Distributed Systems

- Home page
- Online documentation
- Dev's Corner

All Data Structures Functions Variables Typedefs Enumerations Enumerator Groups Pages
Modules | Macros | Enumerations | Functions
Logging support
Grounding features

A generic logging facility in the spirit of log4j (grounding feature) More...

## Modules
| Existing log categories
| (automatically extracted)

## Macros
#define | XBT_LOG_STATIC_THRESHOLD xbt_log_priority_none
#define | XBT_LOG_NEW_SUBCATEGORY(catName, parent, desc)
#define | XBT_LOG_NEW_CATEGORY(catName, desc)
#define | XBT_LOG_DEFAULT_CATEGORY(cname)
#define | XBT_LOG_NEW_DEFAULT_CATEGORY(cname, desc)
#define | XBT_LOG_NEW_DEFAULT_SUBCATEGORY(cname, parent, desc)
#define | XBT_LOG_EXTERNAL_CATEGORY(cname)
#define | XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(cname)
#define | XBT_LOG_ISENABLED(catName, priority)
#define | XBT_CDEBUG(c,...)
| Log an event at the DEBUG priority on the specified category with these args.
#define | XBT_CVERB(c,...)
| Log an event at the VERB priority on the specified category with these args.
#define | XBT_CINFO(c,...)
| Log an event at the INFO priority on the specified category with these args.
#define | XBT_CWARN(c,...)
| Log an event at the WARN priority on the specified category with these args.
#define | XBT_CERROR(c,...)
| Log an event at the ERROR priority on the specified category with these args.
#define | XBT_CCRITICAL(c,...)
| Log an event at the CRITICAL priority on the specified category with these args (CCRITICALn exists for any n<10).
#define | XBT_DEBUG(...)
| Log an event at the DEBUG priority on the default category with these args.
#define | XBT_VERB(...)
| Log an event at the VERB priority on the default category with these args.
#define | XBT_INFO(...)
| Log an event at the INFO priority on the default category with these args.
#define | XBT_WARN(...)
| Log an event at the WARN priority on the default category with these args.
#define | XBT_ERROR(...)
| Log an event at the ERROR priority on the default category with these args.
#define | XBT_CRITICAL(...)
| Log an event at the CRITICAL priority on the default category with these args.
#define | XBT_IN(...)
| Log at TRACE priority that we entered in current function, appending a user specified format.
#define | XBT_OUT(...)
| Log at TRACE priority that we exited the current function, appending a user specified format.
#define | XBT_HERE(...)
| Log at TRACE priority a message indicating that we reached that point, appending a user specified format.

## Enumerations
enum | e_xbt_log_priority_t { , xbt_log_priority_trace = 1, xbt_log_priority_debug = 2, xbt_log_priority_verbose = 3, xbt_log_priority_info = 4, xbt_log_priority_warning = 5, xbt_log_priority_error = 6, xbt_log_priority_critical = 7, xbt_log_priority_infinite = 8 }
| Log prioritiesThe different existing priorities. More...

## Functions
void | xbt_log_control_set (const char *control_string)

## Detailed Description

A generic logging facility in the spirit of log4j (grounding feature)

    This section describes the API to the log functions used
    everywhere in this project.

# Table of contents

- 1. Introduction
  - 1.1 Category hierarchy
  - 1.2 Logging priorities
  - 1.3 Message appenders
  - 1.5 History of this module
- 2. Programmer interface
  - 2.1 Constructing the category hierarchy
  - 2.2 Declaring message priority
  - 2.3 Checking if a particular category/priority is enabled
  - 2.4 Using a default category (the easy interface)
  - 2.5 Putting all together: the easy interface
  - 2.6 Example of use
- 3. User interface
  - 3.1 Configuration
    - 3.1.1 Threshold configuration
    - 3.1.2 Passing several settings
    - 3.1.3 Format configuration
    - 3.1.4 Category appender
    - 3.1.5 Category additivity
  - 3.2 Misc and Caveats
- 4. Internal considerations
  - 4.1 Performance
  - 4.2 Appenders
- Existing log categories

# 1. Introduction

This module is in charge of handling the log messages of every SimGrid program. The main design goal are:

- configurability: the user can choose at runtime what messages to show and what to hide, as well as how messages get displayed.
- ease of use: both to the programmer (using preprocessor macros black magic) and to the user (with command line options)
- performances: logging shouldn't slow down the program when turned off, for example
- deal with distributed settings: SimGrid programs are [often] distributed ones, and the logging mechanism allows to syndicate each and every log source into the same place. At least, its design would allow to, once we write the last missing pieces

There is three main concepts in SimGrid's logging mechanism: category, priority and appender. These three concepts work together to enable developers to log messages according to message type and priority, and to control at runtime how these messages are formatted and where they are reported.

## 1.1 Category hierarchy

The first and foremost advantage of any logging API over plain printf() resides in its ability to disable certain log statements while allowing others to print unhindered. This capability assumes that the logging space, that is, the space of all possible logging statements, is categorized according to some developer-chosen criteria.

This observation led to choosing category as the central concept of the system. In a certain sense, they can be considered as logging topics or channels.

## 1.2 Logging priorities

The user can naturally declare interest into this or that logging category, but he also can specify the desired level of details for each of them. This is controlled by the priority concept (which should maybe be renamed to severity).

Empirically, the user can specify that he wants to see every debugging message of MSG while only being interested into the messages at level "error" or higher about the XBT internals.

## 1.3 Message appenders

The message appenders are the elements in charge of actually displaying the message to the user. For now, only two appenders exist: the default one prints stuff on stderr while it is possible to create appenders printing to a specific file.

Other are planed (such as the one sending everything to a remote server, or the one using only a fixed amount of lines in a file, and rotating content on need). One day, for sure ;)

## 1.4 Message layouts

The message layouts are the elements in charge of choosing how each message will look like. Their result is a string which is then passed to the appender attached to the category to be displayed.

For now, there is two layouts: The simple one, which is good for most cases, and another one allowing users to specify the format they want. 3.1 Configuration provides more info on this.

## 1.5 History of this module

Historically, this module is an adaptation of the log4c project, which is dead upstream, and which I was given the permission to fork under the LGPL licence by the log4c's authors. The log4c project itself was loosely based on the Apache project's Log4J, which also inspired Log4CC, Log4py and so on. Our work differs somehow from these projects anyway, because the C programming language is not object oriented.

# 2. Programmer interface

## 2.1 Constructing the category hierarchy

Every category is declared by providing a name and an optional parent. If no parent is explicitly named, the root category, LOG_ROOT_CAT is the category's parent.

A category is created by a macro call at the top level of a file. A category can be created with any one of the following macros:

- XBT_LOG_NEW_CATEGORY(MyCat,desc); Create a new root
- XBT_LOG_NEW_SUBCATEGORY(MyCat, ParentCat,desc); Create a new category being child of the category ParentCat
- XBT_LOG_NEW_DEFAULT_CATEGORY(MyCat,desc); Like XBT_LOG_NEW_CATEGORY, but the new category is the default one in this file
- XBT_LOG_NEW_DEFAULT_SUBCATEGORY(MyCat, ParentCat,desc); Like XBT_LOG_NEW_SUBCATEGORY, but the new category is the default one in this file

The parent cat can be defined in the same file or in another file (in which case you want to use the XBT_LOG_EXTERNAL_CATEGORY macro to make it visible in the current file), but each category may have only one definition. Likewise, you can use a category defined in another file as default one using XBT_LOG_EXTERNAL_DEFAULT_CATEGORY

Typically, there will be a Category for each module and sub-module, so you can independently control logging for each module.

For a list of all existing categories, please refer to the Existing log categories section. This file is generated automatically from the SimGrid source code, so it should be complete and accurate.

# 2.2 Declaring message priority

A category may be assigned a threshold priority. The set of priorities are defined by the e_xbt_log_priority_t enum. All logging request under this priority will be discarded.

If a given category is not assigned a threshold priority, then it inherits one from its closest ancestor with an assigned threshold. To ensure that all categories can eventually inherit a threshold, the root category always has an assigned threshold priority.

Logging requests are made by invoking a logging macro on a category. All of the macros have a printf-style format string followed by arguments. If you compile with the -Wall option, gcc will warn you for unmatched arguments, ie when you pass a pointer to a string where an integer was specified by the format. This is usually a good idea.

Here is an example of the most basic type of macro. This is a logging request with priority warning.

`XBT_CLOG(MyCat, xbt_log_priority_warning, "Values are: %d and '%s'", 5, "oops");`

A logging request is said to be enabled if its priority is higher than or equal to the threshold priority of its category. Otherwise, the request is said to be disabled. A category without an assigned priority will inherit one from the hierarchy.

It is possible to use any non-negative integer as a priority. If, as in the example, one of the standard priorities is used, then there is a convenience macro that is typically used instead. For example, the above example is equivalent to the shorter:

`XBT_CWARN(MyCat, "Values are: %d and '%s'", 5, "oops");`

# 2.3 Checking if a particular category/priority is enabled

It is sometimes useful to check whether a particular category is enabled at a particular priority. One example is when you want to do some extra computation to prepare a nice debugging message. There is no use of doing so if the message won't be used afterward because debugging is turned off.

Doing so is extremely easy, thanks to the XBT_LOG_ISENABLED(category, priority).

# 2.4 Using a default category (the easy interface)

If XBT_LOG_NEW_DEFAULT_SUBCATEGORY(MyCat, Parent) or XBT_LOG_NEW_DEFAULT_CATEGORY(MyCat) is used to create the category, then the even shorter form can be used:

`XBT_WARN("Values are: %s and '%d'", 5, "oops");`

Only one default category can be created per file, though multiple non-defaults can be created and used.

# 2.5 Putting all together: the easy interface

First of all, each module should register its own category into the categories tree using XBT_LOG_NEW_DEFAULT_SUBCATEGORY.

Then, logging should be done with the XBT_DEBUG, XBT_VERB, XBT_INFO, XBT_WARN, XBT_ERROR and XBT_CRITICAL macros.

Under GCC, these macro check there arguments the same way than printf does. So, if you compile with -Wall, the following code will issue a warning: `XBT_DEBUG("Found %s (id %d)", some_string, a_double)`

If you want to specify the category to log onto (for example because you have more than one category per file, add a C before the name of the log producing macro (ie, use XBT_CDEBUG, XBT_CVERB, XBT_CINFO, XBT_CWARN, XBT_CERROR and XBT_CCRITICAL and friends), and pass the category name as first argument.

The TRACE priority is not used the same way than the other. You should use the XBT_IN, XBT_OUT and XBT_HERE macros instead.

# 2.6 Example of use

Here is a more complete example:

    #include "xbt/log.h"

    / * create a category and a default subcategory * /
    XBT_LOG_NEW_CATEGORY(VSS);
    XBT_LOG_NEW_DEFAULT_SUBCATEGORY(SA, VSS);

    int main() {
           / * Now set the parent's priority.  (the string would typically be a runtime option) * /
           xbt_log_control_set("SA.thresh:3");

           / * This request is enabled, because WARNING >= INFO. * /
           XBT_CWARN(VSS, "Low fuel level.");

           / * This request is disabled, because DEBUG < INFO. * /
           XBT_CDEBUG(VSS, "Starting search for nearest gas station.");

           / * The default category SA inherits its priority from VSS. Thus,
              the following request is enabled because INFO >= INFO.  * /
           XBT_INFO("Located nearest gas station.");

           / * This request is disabled, because DEBUG < INFO. * /
           XBT_DEBUG("Exiting gas station search");
    }

# 3. User interface

# 3.1 Configuration

Although rarely done, it is possible to configure the logs during program initialization by invoking the xbt_log_control_set() method manually. A more conventional way is to use the –log command line argument. xbt_init() (called by MSG_init() and friends) checks and deals properly with such arguments.

## 3.1.1 Threshold configuration

The most common setting is to control which logging event will get displayed by setting a threshold to each category through the `threshold` keyword.

For example,

    --log=root.threshold:debug

will make SimGrid extremely verbose while

    --log=root.thres:critical

should shut it almost completely off.

Note that the `threshold` keyword can be abbreviated here. For example, all the following notations have the same result.

    --log=root.threshold:debug
    --log=root.threshol:debug
    --log=root.thresho:debug
    --log=root.thresh:debug
    --log=root.thres:debug
    --log=root.thre:debug
    --log=root.thr:debug
    --log=root.th:debug
    --log=root.t:debug
    --log=root.:debug     <--- That's obviously really ugly, but it actually works.

The full list of recognized thresholds is the following:

- trace: enter and return of some functions
- debug: crufty output
- verbose: verbose output for the user wanting more
- info: output about the regular functionning
- warning: minor issue encountered
- error: issue encountered
- critical: major issue encountered

## 3.1.2 Passing several settings

You can provide several of those arguments to change the setting of several categories, they will be applied from left to right. So,

    --log="root.thres:debug root.thres:critical"

should disable almost any logging.

Note that the quotes on above line are mandatory because there is a space in the argument, so we are protecting ourselves from the shell, not from SimGrid. We could also reach the same effect with this:

    --log=root.thres:debug --log=root.thres:critical

## 3.1.3 Format configuration

As with SimGrid 3.3, it is possible to control the format of log messages. This is done through the `fmt` keyword. For example,

    --log=root.fmt:%m

reduces the output to the user-message only, removing any decoration such as the date, or the process ID, everything.

Here are the existing format directives:

- %%: the % char
- %n: platform-dependent line separator (LOG4J compatible)
- %e: plain old space (SimGrid extension)

- %m: user-provided message

- %c: Category name (LOG4J compatible)
- %p: Priority name (LOG4J compatible)

- %h: Hostname (SimGrid extension)
- %P: Process name (SimGrid extension – note that with SMPI this is the integer value of the process rank)
- %t: Thread "name" (LOG4J compatible – actually the address of the thread in memory)
- %i: Process PID (SimGrid extension – this is a 'i' as in 'i'dea)

- %F: file name where the log event was raised (LOG4J compatible)
- %l: location where the log event was raised (LOG4J compatible, like '%F:%L' – this is a l as in 'l'etter)
- %L: line number where the log event was raised (LOG4J compatible)
- %M: function name (LOG4J compatible – called method name here of course). Defined only when using gcc because there is no FUNCTION elsewhere.

- %b: full backtrace (Called %throwable in LOG4J). Defined only under windows or when using the GNU libc because backtrace() is not defined elsewhere, and we only have a fallback for windows boxes, not mac ones for example.
- %B: short backtrace (only the first line of the %b). Called %throwable{short} in LOG4J; defined where %b is.

- %d: date (UNIX-like epoch)
- %r: application age (time elapsed since the beginning of the application)

If you want to mimic the simple layout with the format one, you would use this format: '[%h:%i:(%i) %r] %l: [%c/%p] %m%n'. This is not completely correct because the simple layout do not display the message location for messages at priority INFO (thus, the fmt is '[%h:%i:(%i) %r] [%c/%p] %m%n' in this case). Moreover, if there is no process name (ie, messages coming from the library itself, or test programs doing strange things) do not display the process identity (thus, fmt is '[%r] %l: [%c/%p] %m%n' in that case, and '[%r] [%c/%p] %m%n' if they are at priority INFO).

For now, there is only two format modifiers: the precision and the width fields. You can for example specify %.4r to get the application age with 4 numbers after the radix, or %15p to get the process name on 15 columns. Finally, you can specify %10.6r to get the time on at most 10 columns, with 6 numbers after the radix.

Note that when specifying the width, it is filled with spaces. That is to say that for example %5r in your format is converted to "% 5f" for printf (note the extra space); there is no way to fill the empty columns with 0 (ie, pass "%05f" to printf). Another limitation is that you cannot set specific layouts to the several priorities.

## 3.1.4 Category appender

As with SimGrid 3.3, it is possible to control the appender of log messages. This is done through the `app` keyword. For example,

    --log=root.app:file:mylogfile

redirects the output to the file mylogfile.

Any appender setup this way have its own layout format (simple one by default), so you may have to change it too afterward. Moreover, the additivity of the log category is also set to false to prevent log event displayed by this appender to "leak" to any other appender higher in the hierarchy. If it is not what you wanted, you can naturally change it manually.

## 3.1.5 Category additivity

The `add` keyword allows to specify the additivity of a category (see 4.2 Appenders). '0', '1', 'no', 'yes', 'on' and 'off' are all valid values, with 'yes' as default.

The following example resets the additivity of the xbt category to true (which is its default value).

    --log=xbt.add:yes

# 3.2 Misc and Caveats

- Do not use any of the macros that start with '_'.
- Log4J has a 'rolling file appender' which you can select with a run-time option and specify the max file size. This would be a nice default for non-kernel applications.
- Careful, category names are global variables.

# 4. Internal considerations

This module is a mess of macro black magic, and when it goes wrong, SimGrid studently loose its ability to explain its problems. When messing around this module, I often find useful to define XBT_LOG_MAYDAY (which turns it back to good old printf) for the time of finding what's going wrong. But things are quite verbose when everything is enabled...

# 4.1 Performance

Except for the first invocation of a given category, a disabled logging request requires an a single comparison of a static variable to a constant.

There is also compile time constant, XBT_LOG_STATIC_THRESHOLD, which causes all logging requests with a lower priority to be optimized to 0 cost by the compiler. By setting it to xbt_log_priority_infinite, all logging requests are statically disabled at compile time and cost nothing. Released executables might be compiled with (note that it will prevent users to debug their problems)

    -DXBT_LOG_STATIC_THRESHOLD=xbt_log_priority_infinite

Compiling with the

    -DNLOG

option disables all logging requests at compilation time while the

    -DNDEBUG

disables the requests of priority below INFO.

# 4.2 Appenders

Each category has an optional appender. An appender is a pointer to a structure which starts with a pointer to a do_append() function. do_append() prints a message to a log.

When a category is passed a message by one of the logging macros, the category performs the following actions:

- if the category has an appender, the message is passed to the appender's do_append() function,
- if additivity is true for the category, the message is passed to the category's parent. Additivity is true by default, and can be controlled by xbt_log_additivity_set() or something like –log=root.add:1 (see 3.1.5 Category additivity). Also, when you add an appender to a category, its additivity is automatically turned to off. Turn it back on afterward if it is not what you wanted.

By default, only the root category have an appender, and any other category has its additivity set to true. This causes all messages to be logged by the root category's appender.

The default appender function currently prints to stderr, and the only other existing one writes to the specified file. More would be needed, like the one able to send the logs to a remote dedicated server. This is on our TODO list for quite a while now, but your help would be welcome here, too.

## Macro Definition Documentation

#define XBT_LOG_STATIC_THRESHOLD xbt_log_priority_none

All logging requests with priority < XBT_LOG_STATIC_THRESHOLD are disabled at compile time, i.e., compiled out.

#define XBT_LOG_NEW_SUBCATEGORY | ( |  | catName,
|  |  | parent,
|  |  | desc
| ) |  |

Parameters

  catName | name of new category
  parent | father of the new category in the tree
  desc | string describing the purpose of this category

Defines a new subcategory of the parent.

#define XBT_LOG_NEW_CATEGORY | ( |  | catName,
|  |  | desc
| ) |  |

Parameters

  catName | name of new category
  desc | string describing the purpose of this category

Creates a new subcategory of the root category.

#define XBT_LOG_DEFAULT_CATEGORY | ( |  | cname | ) |

Parameters

  cname | name of the cat

Indicates which category is the default one.

#define XBT_LOG_NEW_DEFAULT_CATEGORY | ( |  | cname,
|  |  | desc
| ) |  |

Parameters

  cname | name of the cat
  desc | string describing the purpose of this category

Creates a new subcategory of the root category and makes it the default (used by macros that don't explicitly specify a category).

#define XBT_LOG_NEW_DEFAULT_SUBCATEGORY | ( |  | cname,
|  |  | parent,
|  |  | desc
| ) |  |

Parameters

  cname | name of the cat
  parent | name of the parent
  desc | string describing the purpose of this category

Creates a new subcategory of the parent category and makes it the default (used by macros that don't explicitly specify a category).

#define XBT_LOG_EXTERNAL_CATEGORY | ( |  | cname | ) |

Parameters

  cname | name of the cat

Indicates that a category you'll use in this file (to get subcategories of it, for example) really lives in another file.

#define XBT_LOG_EXTERNAL_DEFAULT_CATEGORY | ( |  | cname | ) |

Parameters

  cname | name of the cat

Indicates that the default category of this file was declared in another file.

#define XBT_LOG_ISENABLED | ( |  | catName,
|  |  | priority
| ) |  |

Parameters

  catName | name of the category
  priority | minimal priority to be enabled to return true (must be e_xbt_log_priority_t)

Returns true if the given priority is enabled for the category. If you have expensive expressions that are computed outside of the log command and used only within it, you should make its evaluation conditional using this macro.

#define XBT_CDEBUG | ( |  | c,
|  |  | ...
| ) |  |

Log an event at the DEBUG priority on the specified category with these args.

Parameters

  c | the category on which to log
  ... | the format string and its arguments

#define XBT_DEBUG | ( |  | ... | ) |

Log an event at the DEBUG priority on the default category with these args.

Parameters

  ... | the format string and its arguments

## Enumeration Type Documentation

enum e_xbt_log_priority_t

Log prioritiesThe different existing priorities.

Enumerator:

  xbt_log_priority_trace | enter and return of some functions
  xbt_log_priority_debug | crufty output
  xbt_log_priority_verbose | verbose output for the user wanting more
  xbt_log_priority_info | output about the regular functionning
  xbt_log_priority_warning | minor issue encountered
  xbt_log_priority_error | issue encountered
  xbt_log_priority_critical | major issue encountered
  xbt_log_priority_infinite | value for XBT_LOG_STATIC_THRESHOLD to not log

## Function Documentation

void xbt_log_control_set | ( | const char * | control_string | ) |

Parameters

  control_string | What to parse

Typically passed a command-line argument. The string has the syntax:

     ( [category] "." [keyword] ":" value (" ")... )...

where [category] is one the category names (see Existing log categories for a complete list of the ones defined in the SimGrid library) and keyword is one of the following:

- thres: category's threshold priority. Possible values: TRACE,DEBUG,VERBOSE,INFO,WARNING,ERROR,CRITICAL
- add or additivity: whether the logging actions must be passed to the parent category. Possible values: 0, 1, no, yes, on, off. Default value: yes.
- fmt: the format to use. See 3.1.3 Format configuration for more information.
- app or appender: the appender to use. See 3.1.4 Category appender for more information.