gfperf.rb

## Name

gfperf.rb — Gfarm measure performance

## Synopsis

`gfperf.rb` [-c, --check] `config file`

## DESCRIPTION

This command measure peformance of gfarm file system according to `config file`. `config file` is written in YAML.

Example config files are (gfarm install directory)/share/gfarm/config/gfperf-[config|simple].yml .

## OPTIONS

`-c, --check`

  Check `config file` syntax.

## EXAMPLES

Config file example is following.

database: {filename: "/var/www/database.db", check span: "10days",
backup: "/var/www/databse.db.bak" }
errlog: "/tmp/gfperf.err.log"
authentication: ["sharedsecret", "gsi_auth", "gsi"]
gfarm2fs_mountpoint: ["/mnt/gfarm2fs"]
metadata: [ {testdir: "gfarm:///tmp", number: "250"},
{testdir: "file:///mnt/gfarm2fs/tmp", number: "250"} ]
tree: [ {testdir: "gfarm:///tmp", width: "5", depth: "3"},
{testdir: "file:///mnt/gfarm2fs/tmp", width: "5", depth: "3"} ]
copy: [
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"}
]
read: [
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
]
write: [
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", overwrite: "true", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", overwrite: "true", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"}
]
replica: [
{testdir: "gfarm:///tmp", src: "gfsd0.local", dst: "gfsd1.local",
filesize: "1M"},
{testdir: "gfarm:///tmp", src: "gfsd1.local", dst: "gfsd0.local",
filesize: "1M"}
]
autoreplica: [
{testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "1"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
number: "3", replica: "1"},
{testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "2"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
number: "3", replica: "2"}
]
parallel: {
group1: [
{rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
],
group2: [
{rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
],
group3: [
{type: "replica", testdir: "gfarm:///tmp", src: "gfsd0.local",
dst: "gfsd1.local", filesize: "1M"},
{type: "replica", testdir: "gfarm:///tmp", src: "gfsd1.local",
dst: "gfsd0.local", filesize: "1M"}
]
}
parallel-autoreplica: {
group4: [
{testdir: "gfarm:///tmp1", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp2", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp3", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp4", filesize: "1M", number: "3",
replica: "2"}
],
group5: [
{testdir: "gfarm:///tmp1", gfarm2fs: "/mnt/gfarm2fs",
filesize: "1M", number: "3", replica: "2"},
{testdir: "gfarm:///tmp2", gfarm2fs: "/mnt/gfarm2fs",
filesize: "1M", number: "3", replica: "2"}
]
}

Following sentences explain keywords of the YAML.

database

  Specify databse filename in filename: . Specify backup of database file in backup: . check span: specifies time span of calculating average and standard deviation.

errlog

  This commands prints error messsages in the file specified by errlog: . All error messages are also written in the database. If you don't need to error log file, this parameter can be omitted.

authentication

  Specify authentication types of performance measuring. Authentication types are sharedsecret, gsi_auth and gsi. Performance test is executed on all specified types.

gfarm2fs_mountpoint

  Specify gfarm2fs mount point. When start testing, this command mounts all gfarm2fs mount point automatically. At the end of testing, unmounted automatically. When changing authenticaion type, remounted automatically.

metadata

  Perfomance measuring using gfperf-metadata. Parameters are gfperf-metadata long options.

tree

  Perfomance measuring using gfperf-tree. Parameters are gfperf-tree long options.

copy

  Perfomance measuring using gfperf-copy. Parameters are gfperf-copy long options.

read

  Perfomance measuring using gfperf-read. Parameters are gfperf-read long options. If you need to test sequentially read, remove random option.

write

  Perfomance measuring using gfperf-write. Parameters are gfperf-write long options. If you need to test sequentially write, remove random option. If you need to test append write, remove overwrite option.

replica

  Perfomance measuring using gfperf-replica. Parameters are gfperf-replica long options.

autoreplica

  Perfomance measuring using gfperf-autoreplica. Parameters are gfperf-autoreplica long options.

parallel

  Specify parallel test patterns. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel-autoreplica). Specify remote host in rhost: . This command uses ssh for executing command. type: specifies one of read, write, replica. These types corresponds to gfperf-parallel-read, gfperf-parallel-write, gfperf-parallel-replica.

parallel-autoreplica

  Specify auto replication test in parallel. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel). Test parameters are gfperf-parallel-autoreplica long options. The result is average of gfperf-parallel-autoreplica outputs.

## SEE ALSO

gfperf-metadata(1), gfperf-tree(1), gfperf-copy(1), gfperf-read(1), gfperf-write(1), gfperf-replica(1), gfperf-autoreplica(1), gfperf-parallel-read(1), gfperf-parallel-write(1), gfperf-parallel-replica(1), gfperf-parallel-autoreplica(1),

## NOTES

Using ssh command execution, no password is recommended.

Bash doesn't read ~/.bashrc in some envronment. Please check before running gfperf.rb.