libcwd: The C++ Debugging Support Library - Tutorial
Author: Carlo Wood

| The C++ Debugging Support Library By Carlo Wood, ©1999 - 2003.

## Tutorial 8: Debugging Threaded Applications

When debugging a threaded application, you must link with -lcwd_r instead of -lcwd. For example:

    g++ -pthread -DLIBCWD_THREAD_SAFE -DCWDEBUG program.cc -lpthread -lcwd_r

Best practise is to use the tool `pkg-config` to retrieve the flags that one needs to pass to the compiler and linker. That is, the output of `pkg-config --cflags libcwd_r` and `pkg-config --libs libcwd_r`.

Libcwd_r should be completely thread-safe, with the following restrictions:

- All debug objects and debug channels must be global (as they should be in non-threaded applications for that matter).
- You are not allowed to create threads before all static and global objects are initialized; in practise this means that you are not allowed to create threads until `main()` is reached.
- You cannot use `dlopen()` to load libcwd when threads have already been created. Likewise you shouldn't `dlopen()` other libraries that use libcwd when there are already running threads, especially when those libraries define new debug objects and/or channels.
- You need to provide one and only one locking mechanism per ostream device that is also used to write debug output. It is preferable not to use the same ostream with two or more different debug objects.

Essentially the debug objects and channels react towards each thread as if that thread is the only thread. The only (visible) shared variable is the final `ostream` that a given debug object writes to. This means that if one thread changes the `ostream` then all other threads also suddenly start to write to that `ostream`. Basically, it is simply not supported: don't change the output stream on the fly.

All other characteristics like the on/off state and the margin and marker strings as well as the indentation are Thread Specific: Every thread may change those without locking or worrying about the effect on other threads.

Every time a new thread is created, it will start with all debug objects and channels turned off, just as at the start of `main()`.

In all likelihood, you'd want to set the margin string such that it reflects which thread is printing the output. For example:

    #include "sys.h"	// See documentation/reference-manual/preparation.html
    #include "debug.h"
    #include <iostream>
    #include <cstdio>
    #include <pthread.h>

    void* thread_function(void* arguments)
    {
      // Set Thread Specific on/off flags of the debug channels.
      ForAllDebugChannels( if (!debugChannel.is_on()) debugChannel.on(); );
      // And for the debug object.
      Debug( libcw_do.on() );
      // Set a margin.
    #ifdef CWDEBUG
      char margin[16];
      sprintf(margin, "%-10lu ", pthread_self());
    #endif
      Debug( libcw_do.margin().assign(margin, 11) );

      Dout(dc::notice, "Entering thread " << pthread_self());
      // ... do stuff
      Dout(dc::notice, "Leaving thread " << pthread_self());
      return (void*)true;
    }

    #ifdef CWDEBUG
    pthread_mutex_t cout_mutex = PTHREAD_MUTEX_INITIALIZER;
    #endif

    int main(void)
    {
      // Don't output a single character at a time (yuk)
      // (Read http://gcc.gnu.org/onlinedocs/libstdc++/27_io/howto.html#8 for an explanation.)
      Debug(set_invisible_on());
      std::ios::sync_with_stdio(false);	// Cause "memory leaks" ([w]cin, [w]cout, [w]cerr filebuf allocations).
      Debug(set_invisible_off());
      // Do header files and library match?
      Debug( check_configuration() );
      // Send debug output to std::cout.
      Debug( libcw_do.set_ostream(&std::cout, &cout_mutex) );
      // Turn debug object on.
      Debug( libcw_do.on() );
      // Set a margin.
    #ifdef CWDEBUG
      char margin[16];
      sprintf(margin, "%-10lu ", pthread_self());
    #endif
      Debug( libcw_do.margin().assign(margin, 11) );
      // Turn all debug channels on.
      ForAllDebugChannels( if (!debugChannel.is_on()) debugChannel.on(); );
      // List all channels.
      Debug( list_channels_on(libcw_do) );

      // Create and join a few threads...
      int const number_of_threads = 4;
      pthread_t thread_id[number_of_threads];
      for (int i = 0; i < number_of_threads; ++i)
      {
        Dout(dc::notice|continued_cf, "main: creating thread " << i << ", ");
        pthread_create(&thread_id[i], NULL, thread_function, NULL);
        Dout(dc::finish, "id " << thread_id[i] << '.');
      }

      for (int i = 0; i < number_of_threads; ++i)
      {
        void* status;
        pthread_join(thread_id[i], &status);
        Dout(dc::notice, "main loop: thread " << i << ", id " << thread_id[i] <<
             ", returned with status " << ((bool)status ? "OK" : "ERROR") << '.');
      }

      Dout(dc::notice, "Exiting from main()");
      return 0;
    }

Which outputs something like:

    1024       BFD     : Enabled
    1024       DEBUG   : Enabled
    1024       MALLOC  : Enabled
    1024       NOTICE  : Enabled
    1024       SYSTEM  : Enabled
    1024       WARNING : Enabled
    1024       NOTICE  : main: creating thread 0, <unfinished>
    1024       MALLOC  :     malloc(8160) = <unfinished>
    1024       BFD     :         address 0x401fbbd8 corresponds to pthread.c:533
    1024       MALLOC  :     <continued> 0x8386890
    1024       NOTICE  : <continued> id 1026.
    1026       NOTICE  : Entering thread 1026
    1026       NOTICE  : Leaving thread 1026
    1024       NOTICE  : main: creating thread 1, id 2051.
    2051       NOTICE  : Entering thread 2051
    2051       NOTICE  : Leaving thread 2051
    1024       NOTICE  : main: creating thread 2, id 3076.
    3076       NOTICE  : Entering thread 3076
    3076       NOTICE  : Leaving thread 3076
    1024       NOTICE  : main: creating thread 3, id 4101.
    1024       NOTICE  : main loop: thread 0, id 1026, returned with status OK.
    1024       NOTICE  : main loop: thread 1, id 2051, returned with status OK.
    1024       NOTICE  : main loop: thread 2, id 3076, returned with status OK.
    4101       NOTICE  : Entering thread 4101
    4101       NOTICE  : Leaving thread 4101
    1024       NOTICE  : main loop: thread 3, id 4101, returned with status OK.
    1024       NOTICE  : Exiting from main()
    1024       MALLOC  : free(0x8386890)            pthread.c:533  <unknown type>; (sz = 8160)

Congratulations, you are now a libcwd expert. If you still have any questions that you can't find answers to here, feel free to mail me.