SimGrid: Automatic Testing Infrastructure

SimGrid 3.10 Versatile Simulation of Distributed Systems

- Home page
- Online documentation
- Dev's Corner

All Data Structures Functions Variables Typedefs Enumerations Enumerator Groups Pages
Automatic Testing Infrastructure

# How to run test in pipol?

In order to use PIPOL you have to create your account here: http://pipol.inria.fr/

## What is PIPOL?

The PIPOL platform is a porting, testing and compilation cluster. It hosts several hardware and software architectures to perform its work. An user can reserve a resource (a specific hardware, an operating system...) to be used interactively or automatically (with a "Nightly Builds" system) for software testing.

## From PIPOL frontend

You first need to be connected into pipol ssh.

    user@caraja:~$ ssh pipol.inria.fr
    Linux pipol-server 2.6.26-2-amd64 #1 SMP Thu Nov 5 02:23:12 UTC 2009 x86_64
    ***************************************************************************
    *           You are just logged-on the PIPOL platform server              *
    *   You can find some documentation about porting works on the platform   *
    *         on the PIPOL's web site: http://pipol.inria.fr/docs/            *
    *   To report bugs you can write to: pipol-sysadm@lists.gforge.inria.fr   *
    ***************************************************************************

    user@pipol:~$

Then you can see pipol images availables for deployment

    user@pipol:~$ pipol-sub si
        amd64_2010-linux-centos-5.dd.gz
        amd64_2010-linux-debian-squeeze.dd.gz
        amd64_2010-linux-debian-testing.dd.gz
        amd64_2010-linux-fedora-core13.dd.gz
        amd64_2010-linux-fedora-core14.dd.gz
        amd64_2010-linux-fedora-core16.dd.gz
        amd64_2010-linux-ubuntu-lucid.dd.gz
        amd64_2010-linux-ubuntu-maverick.dd.gz
        amd64_2010-linux-ubuntu-natty.dd.gz
        amd64_kvm-linux-debian-lenny
        amd64_kvm-linux-debian-testing
        amd64_kvm-windows-7
        amd64-linux-centos-5.dd.gz
        amd64-linux-debian-etch.dd.gz
        amd64-linux-debian-lenny.dd.gz
    ....
        i386-linux-ubuntu-lucid.dd.gz
        i386-linux-ubuntu-maverick.dd.gz
        i386-linux-ubuntu-natty.dd.gz
        i386-linux-ubuntu-precise.dd.gz
        i386_mac-mac-osx-server-leopard.dd.gz
        i386-unix-freebsd-7.dd.gz
        i386-unix-opensolaris-10.dd.gz
        i386-unix-opensolaris-11.dd.gz
        i386-unix-solaris-10.dd.gz
        ia64-linux-debian-lenny.dd
        ia64-linux-debian-squeeze.dd
        ia64-linux-fedora-core9.dd
        ia64-linux-redhatEL-5.0.dd
        x86_64_mac-mac-osx-server-snow-leopard.dd.gz
        x86_mac-mac-osx-server-snow-leopard.dd.gz

You can also see available architectures on host name:

    navarro@pipol:~$ pipol-sub sa
    =================================================================
                        Availables architectures:
    =================================================================

    pipol18
    :i386_2010:amd64_2010:
    pipol19
    :i386_2010:amd64_2010:
    pipol20
    :i386_2010:amd64_2010:
    pipol1
    :i386:amd64:
    pipol2
    :i386:amd64:
    pipol3
    :i386:amd64:
    pipol4
    :i386:amd64:
    pipol5
    :i386:amd64:
    pipol6
    :i386:amd64:
    pipol7
    :i386:amd64:
    pipol8
    :i386:amd64:
    pipol14
    :i386_kvm:amd64_kvm:
    pipol15
    :i386_kvm:amd64_kvm:
    pipol16
    :i386_kvm:amd64_kvm:
    pipol17
    :i386_kvm:amd64_kvm:
    pipol11
    :i386_mac:x86_mac:
    pipol10
    :ia64:
    pipol9
    :ia64:
    pipol12
    :x86_64_mac:

When you have choose your image and host (not necessary) you deploy with command line:

pipol-sub esn <image name> <host-name> <deployment-time>

    user@pipol:~$pipol-sub esn amd64_2010-linux-ubuntu-maverick.dd.gz pipol20 02:00
    user@pipol:~$ssh pipol20

You can now make all your tests.

## From a computer

You have to renseign to simgrid configuration your pipol login.

    $ cmake -Dpipol_user=user .

Then you have two kind of command:

- make <image-name>

      $ make amd64_2010-linux-ubuntu-maverick

  This command copy your local simgrid directory to pipol and execute a configure, make and ctest.

- make <image_name>_experimental

      $ make amd64_2010-linux-ubuntu-maverick_experimental

  Same as previous but report into cdash

You can also see all available images from pipol

    $ make pipol_test_list_images

# How to use nightly builds?

All the pipol scripts are located in "<project/directory>/buildtools/pipol/". If you modify them you have to update the pipol home directory by using this command: make sync-pipol.

It will copy those files:

    simgrid@caraja:~/workspace/simgrid/build$ make sync-pipol
    Update pipol script for user: simgrid
    rc.debian                                       100% 1338     1.3KB/s   00:00
    rc.fedora                                       100% 1433     1.4KB/s   00:00
    rc.mac                                          100% 1129     1.1KB/s   00:00
    rc.ubuntu                                       100% 1875     1.8KB/s   00:00
    rc.windows                                      100%  494     0.5KB/s   00:00
    Nightly_simgrid.sh                              100% 4004     3.9KB/s   00:00
    Experimental_bindings.sh                        100% 1794     1.8KB/s   00:00

# How to report tests in cdash?

Reporting experiment in cdash is very easy because it is done by ctest.

The easier way is to execute command line "ctest -D Experiemntal" in build directory. More option is available by ctest:

      ctest -D Continuous
      ctest -D Continuous(Start|Update|Configure|Build)
      ctest -D Continuous(Test|Coverage|MemCheck|Submit)
      ctest -D Experimental
      ctest -D Experimental(Start|Update|Configure|Build)
      ctest -D Experimental(Test|Coverage|MemCheck|Submit)
      ctest -D Nightly
      ctest -D Nightly(Start|Update|Configure|Build)
      ctest -D Nightly(Test|Coverage|MemCheck|Submit)
      ctest -D NightlyMemoryCheck

If you want to have a code coverage, please add option on simgrid.

    $ cmake -Denable_coverage=ON .
    $ ctest -D ExperimentalStart
    $ ctest -D ExperimentalConfigure
    $ ctest -D ExperimentalBuild
    $ ctest -D ExperimentalTest
    $ ctest -D ExperimentalCoverage
    $ ctest -D ExperimentalSubmit

# Infrastructure of the SimGrid Performance Regresion tests

The goal of these tests is to detect the impact on simulation performance of the commited changes. It is not quite effective right now but at the end, it should be similar to Phoromatic.

## How to execute g5k campaign?

Quick steps deployment for rennes:

- 1/ Create a G5K account

- 2/ SSH to a frontend (must be rennes, nancy or toulouse for git protocol)

- 3/ Install g5k-campaign

      $ gem install g5k-campaign --source http://g5k-campaign.gforge.inria.fr/pkg -p http://proxy:3128 --no-ri --no-rdoc --user-install

- 4/ Configure the API

      $  mkdir ~/.restfully
      $  echo 'base_uri: https://api.grid5000.fr/stable/grid5000' > ~/.restfully/api.grid5000.fr.yml
      $  chmod 0600 ~/.restfully/api.grid5000.fr.yml

- 5/ Git clone the SimGrid Scalability project

      $  git clone git@github.com:mquinson/simgrid-scalability-XPs.git

  An alternative URL is git://scm.gforge.inria.fr/simgrid/simgrid-scalability-XPs.git, but the repository on github is believed to be more uptodate. You may want to double-check by pulling both:

      $ git remote add g5k git://scm.gforge.inria.fr/simgrid/simgrid-scalability-XPs.git
      $ git pull g5k master

- 6/ Copy the run script into your home

      $  cp simgrid-scalability-XPs/script-sh/run-g5k-scalab.sh ~/

- 7/ Create the result log directory (must be ~/log/)

      $  mkdir ~/log

- 8/ Execute the g5k campaign on a revision "rev"

      $  sh run-g5k-scalab.sh "rev"

You can also have more parameters

- 1/ -> 5/ Same as before

- 6/ Open simgrid-scalability-XPs

- 7/ Execute SGXP.pl to see parameters

      $  perl SGXP.pl --help

- 8/ Execute SGXP.pl with your parameters like

      $  ./SGXP.pl --site=nancy --cluster=graphene --test=chord,goal --rev="09bbc8de,3ca7b9a13"

## How to analyze logs?

To analyze log from g5k-campaign you must install R tool.

- 0/ You can copy logs from g5k to your computer (recommanded)

- 1/ Open ~/log/

- 2/ Execute the perl analyzer for goal log

      $  ~/simgrid-scalability-XPs/libperl/analyzer.pl goal.log.* > goal.csv

- 3/ Execute the R analyer for goal log

      $  ~/simgrid-scalability-XPs/script-R/goal.R goal.csv output.goal.pdf

Example of generated pdf here.