---
title: Tests and Benchmark
---
# Tests and Benchmark

## Tests

There exist several test suites for testing the correctness of a Markdown implementation. The
original [Markdown Test Suite] is the standard which one needs to test against. The [PHP Markdown
suite][MDTest] contains the original test suite and several more tests (some specifically geared
towards the extension of the PHP Markdown Extra package). I have used the latter test tool to
roughly verify that kramdown is able to parse standard Markdown. However, since the syntax used by
kramdown varies slightly from standard Markdown most of the tests fail - which is fine. When looking
at the differences one can see that the failures result from these differences.

Besides using the above mentioned test suite kramdown comes with its own set of tests which is used
to verify that the implementation matches the kramdown specification.

If you believe you have found a bug in the implementation, please follow these steps:

* Check the [syntax page](syntax.html) and see if the behaviour is not intended.

* If the behaviour is not intended and it seems that kramdown should parse some text in another
  fashion, please open a [bug report] and attach two files: one with the text and one with the HTML
  conversion you think is correct.

[bug report]: http://rubyforge.org/tracker/?atid=28673&group_id=7403&func=browse


## Benchmark

kramdown comes with a small benchmark to test how fast it is in regard to four other Ruby Markdown
implementations: Maruku, BlueFeather, BlueCloth, RDiscount and Redcarpet. The first two are written
using only Ruby, the latter three are written in C and need to be compiled.

As one can see below, kramdown is currently (May 2013) ~3-4x faster than Maruku, ~4-5x faster
than BlueFeather but ~30x slower than BlueCloth and RDiscount and ~100x slower than Redcarpet:

<pre><code>
Running tests on 2013-05-08 under ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-linux]

Test using file mdsyntax.text and 20 runs
Rehearsal ----------------------------------------------------
kramdown 1.0.1     1.010000   0.000000   1.010000 (  1.014665)
Maruku 0.6.0       2.910000   0.020000   2.930000 (  2.939359)
BlueFeather 0.40   3.890000   0.000000   3.890000 (  3.903594)
BlueCloth 2.2.0    0.050000   0.000000   0.050000 (  0.048246)
RDiscount 1.6.8    0.020000   0.000000   0.020000 (  0.020237)
redcarpet 2.2.2    0.000000   0.000000   0.000000 (  0.005038)
------------------------------------------- total: 7.900000sec

                       user     system      total        real
kramdown 1.0.1     0.850000   0.000000   0.850000 (  0.856454)
Maruku 0.6.0       2.820000   0.030000   2.850000 (  2.856139)
BlueFeather 0.40   3.870000   0.000000   3.870000 (  3.884309)
BlueCloth 2.2.0    0.040000   0.000000   0.040000 (  0.044865)
RDiscount 1.6.8    0.020000   0.000000   0.020000 (  0.022889)
redcarpet 2.2.2    0.010000   0.000000   0.010000 (  0.005234)

Real time of X divided by real time of kramdown
Maruku             3.3348
BlueFeather        4.5353
BlueCloth          0.0524
RDiscount          0.0267
redcarpet          0.0061

Test using file mdbasics.text and 20 runs
Rehearsal ----------------------------------------------------
kramdown 1.0.1     0.150000   0.000000   0.150000 (  0.154616)
Maruku 0.6.0       0.690000   0.000000   0.690000 (  0.689874)
BlueFeather 0.40   1.000000   0.000000   1.000000 (  1.003185)
BlueCloth 2.2.0    0.010000   0.000000   0.010000 (  0.012767)
RDiscount 1.6.8    0.010000   0.000000   0.010000 (  0.004747)
redcarpet 2.2.2    0.000000   0.000000   0.000000 (  0.001594)
------------------------------------------- total: 1.860000sec

                       user     system      total        real
kramdown 1.0.1     0.160000   0.000000   0.160000 (  0.155369)
Maruku 0.6.0       0.670000   0.000000   0.670000 (  0.673886)
BlueFeather 0.40   1.000000   0.000000   1.000000 (  0.995874)
BlueCloth 2.2.0    0.020000   0.000000   0.020000 (  0.013389)
RDiscount 1.6.8    0.010000   0.000000   0.010000 (  0.006698)
redcarpet 2.2.2    0.000000   0.000000   0.000000 (  0.001691)

Real time of X divided by real time of kramdown
Maruku             4.3373
BlueFeather        6.4097
BlueCloth          0.0862
RDiscount          0.0431
redcarpet          0.0109
</code></pre>

And here are some graphs which show the execution times of the various kramdown releases on
different Ruby interpreters:

![ruby 1.8.5p231]({relocatable: img/graph-ruby-1.8.5-231.png})
![ruby 1.8.6p399]({relocatable: img/graph-ruby-1.8.6-399.png})
![ruby 1.8.7p249]({relocatable: img/graph-ruby-1.8.7-249.png})
![ruby 1.8.7p302]({relocatable: img/graph-ruby-1.8.7-302.png})
![ruby 1.9.2p320]({relocatable: img/graph-ruby-1.9.2p320-320.png})
![ruby 1.9.3p392]({relocatable: img/graph-ruby-1.9.3p392-392.png})
![ruby 2.0.0]({relocatable: img/graph-ruby-2.0.0p0-0.png})

[Markdown Test Suite]: http://daringfireball.net/projects/downloads/MarkdownTest_1.0.zip
[MDTest]: http://www.michelf.com/docs/projets/mdtest-1.0.zip
