Arithmetic Coding and PPM in Java

# Arithmetic Coding and PPM Compression in Java

## Overview

The arithmetic coding package is an open-soruce implementation of a generic arithemtic coder and decoder, along with byte stream models that are subclasses of Java's I/O streams. The implementation of arithmetic coding is based on:

- Witten, I. H., R. Neal, and J. G. Cleary. 1987. Arithmetic coding for data compression. Communications of the ACM 30(6): 520-540.

Example statistical models include a uniform distribution, simple unigram model, and a parametric prediction by partial matching (PPM) model. The PPM model is based on:

- Cleary, J.G. and I.H. Witten. 1984. Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications. 32(4):396-402.

Models other than PPM may be integrated for compression.

## Download

License: The arithcode package is licensed under the standard Apache 2.0 license.

You only need the jar to run the arithcode package, but you need the source distribution to build and test it.

- Java Archive (Jar): arithcode-1.2.jar
- Tar/Gzipped Source: arithcode-1.2.tgz

The source distribution includes JUnit, which is required for testing, but not for building the jar or using the jar.

The distribution also includes Witten and Bell's Calgary corpus, which includes a range of different kinds of byte data for testing compression algorithms.

## Getting Started

The package has an Apache Ant build file which has targets to build the jar, test, and document the package. So just download the source, unpack it and run any of the following ant targets:

- clean: cleans the distribution
- compile: compile the java source
- jar: build the jar file in /arithcode-1.2.jar
- javadoc: generate the javadoc in /build/doc/api/
- test-compile: compiles the tests (requires JUnit)
- test: run the unit tests (requires JUnit)
- calgary: run various PPM compressors on the Calgary corpus
- ppm: compress a file using PPM
- unppm: uncompress a file using PPM
- web: build the web site

## Understanding Arithmetic Coding and PPM

The original papers cited in the introduction are quite readable. I've also provided a tutorial with further references.

- Quick Intro to Arithmetic Coding and PPM

## PPM Compression Rates and Speed

Here are results for the Calgary corpus for speed and compression for various lengths of PPM.

- Version 1.2 Results: 9 February 2011

And just for laughs, some historical results illustrating the progress of compilers and processors and memory.

- Version 1.1 Results: 26 January 2003
- Version 1.0 Results: 6 October 2002

## Acknowledgements

Thanks to the original authors for making their source and corpora available.

Thanks also to Garrick Toubassi for patching a bug in version 1.1; it only arose in fairly extreme boundary conditions, making its diagnosis a very nice piece of debugging indeed.