MULTIPATH.CONF(5)

NAME
multipath.conf - multipath daemon configuration file

DESCRIPTION
multipath.conf is the configuration file for the multipath daemon. It is used to overwrite the built-in configuration table of multipathd. Any line whose first non-white-space character is a '#' is considered a comment line. Empty lines are ignored.

SYNTAX
The configuration file contains entries of the form:

    <section> {
        <attribute> <value>
        ...
        <subsection> {
            <attribute> <value>
            ...
        }
    }

Each section contains one or more attributes or subsections. The recognized keywords for attributes or subsections depend on the section in which they occur.

The following section keywords are recognized:

defaults
    This section defines default values for attributes which are used whenever no values are given in the appropriate device or multipath sections.

blacklist
    This section defines which devices should be excluded from the multipath topology discovery.

blacklist_exceptions
    This section defines which devices should be included in the multipath topology discovery, despite being listed in the blacklist section.

multipaths
    This section defines the multipath topologies. They are indexed by a World Wide Identifier(wwid). For details on the wwid generation see section WWID generation below.

devices
    This section defines the device-specific settings.

overrides
    This section defines values for attributes that should override the device-specific settings for all devices.

defaults section
The defaults section recognizes the following keywords:

polling_interval
    interval between two path checks in seconds. For properly functioning paths, the interval between checks will gradually increase to max_polling_interval. This value will be overridden by the WatchdogSec setting in the multipathd.service definition if systemd is used. Default is 5

max_polling_interval
    maximal interval between two path checks in seconds; default is 4 * polling_interval

multipath_dir
    directory where the dynamic shared objects are stored; default is system dependent, commonly /lib/multipath

find_multipaths
    If set to yes , instead of trying to create a multipath device for every non-blacklisted path, multipath will only create a device if one of three condidions are met. 1 There are at least two non-blacklisted paths with the same wwid, 2 the user manually forces the creation, by specifying a device with the multipath command, or 3 a path has the same WWID as a multipath device that was previously created while find_multipaths was set (even if that multipath device doesn't currently exist). Whenever a multipath device is created with find_multipaths set, multipath will remeber the WWID of the device, so that it will automatically create the device again, as soon as it sees a path with that WWID. This should allow most users to have multipath automatically choose the correct paths to make into multipath devices, without having to edit the blacklist; Default is no

verbosity
    default verbosity. Higher values increase the verbosity level. Valid levels are between 0 and 6; default is 2

reassign_maps
    enable reassigning of device-mapper maps. With this option multipathd will remap existing device-mapper maps to always point to multipath device, not the underlying block devices. Possible values are yes and no. Default is yes

path_selector
    The default path selector algorithm to use; they are offered by the kernel multipath target. There are three selector algorithms.

    round-robin 0
        Loop through every path in the path group, sending the same amount of IO to each.

    queue-length 0
        Send the next bunch of IO down the path with the least amount of outstanding IO.

    service-time 0
        Choose the path for the next bunch of IO based on the amount of outstanding IO to the path and its relative throughput.

path_grouping_policy
    The default path grouping policy to apply to unspecified multipaths. Possible values are

    failover
        1 path per priority group

    multibus
        all paths in 1 priority group

    group_by_serial
        1 priority group per serial number

    group_by_prio
        1 priority group per priority value. Priorities are determined by callout programs specified as a global, per-controller or per-multipath option in the configuration file.

    group_by_node_name
        1 priority group per target node name. Target node names are fetched in /sys/class/fc_transport/target*/node_name.

    Default value is failover.

uid_attribute
    The udev attribute providing a unique path identifier. Default value is ID_SERIAL

getuid_callout
    The default program and args to callout to obtain a unique path identifier. Should be specified with an absolute path. This parameter is deprecated.

prio
    The name of the path priority routine. The specified routine should return a numeric value specifying the relative priority of this path. Higher number have a higher priority. none is a valid value. Currently the following path priority routines are implemented:

    const
        Return a constant priority of 1.

    emc Generate the path priority for EMC arrays.

    alua
        Generate the path priority based on the SCSI-3 ALUA settings.

    ontap
        Generate the path priority for NetApp arrays.

    rdac
        Generate the path priority for LSI/Engenio/NetApp E-Series RDAC controller.

    hp_sw
        Generate the path priority for Compaq/HP controller in active/standby mode.

    hds Generate the path priority for Hitachi HDS Modular storage arrays.

    random
        Generate a random priority between 1 and 10.

    weightedpath
        Generate the path priority based on the regular expression and the priority provided as argument. requires prio_args keyword.

    Default value is none.

prio_args
    Arguments to pass to to the prio function. Currently only used with weighted, which needs a value of the form <hbtl|devname> <regex1> <prio1> <regex2> <prio2> ... hbtl regex can be of SCSI H:B:T:L format Ex: 1:0:.:. , *:0:0:. devname regex can be of device name format Ex: sda , sd.e

features
    Specify any device-mapper features to be used. Syntax is num list where num is the number of features in list. Possible values for the feature list are

    queue_if_no_path
        Queue IO if no path is active; identical to the no_path_retry keyword.

    no_partitions
        Disable automatic partitions generation via kpartx.

path_checker
    The default method used to determine the paths state. Possible values are

    readsector0
        (Deprecated) Read the first sector of the device. This checker is being deprecated, please use directio instead

    tur Issue a TEST UNIT READY command to the device.

    emc_clariion
        Query the EMC Clariion specific EVPD page 0xC0 to determine the path state.

    hp_sw
        Check the path state for HP storage arrays with Active/Standby firmware.

    rdac
        Check the path state for LSI/Engenio/NetApp E-Series RDAC storage controller.

    directio
        Read the first sector with direct I/O.

    Default value is directio.

failback
    Tell multipathd how to manage path group failback.

    immediate
        Immediately failback to the highest priority pathgroup that contains active paths.

    manual
        Do not perform automatic failback.

    followover
        Only perform automatic failback when the first path of a pathgroup becomes active. This keeps a node from automatically failing back when another node requested the failover.

    values > 0
        deferred failback (time to defer in seconds)

    Default value is manual.

rr_min_io
    The number of IO to route to a path before switching to the next in the same path group. This is only for BIO based multipath. Default is 1000

rr_min_io_rq
    The number of IO requests to route to a path before switching to the next in the same path group. This is only for request based multipath. Default is 1

rr_weight
    If set to priorities the multipath configurator will assign path weights as "path prio * rr_min_io". Possible values are priorities or uniform. Default is uniform.

no_path_retry
    Specify the number of retries until disable queueing, or fail for immediate failure (no queueing), queue for never stop queueing. If unset no queueing is attempted. Default is unset.

user_friendly_names
    If set to yes , using the bindings file /etc/multipath/bindings to assign a persistent and unique alias to the multipath, in the form of mpath<n>. If set to no use the WWID as the alias. In either case this be will be overridden by any specific aliases in the multipaths section. Default is no

flush_on_last_del
    If set to yes , multipathd will disable queueing when the last path to a device has been deleted. Default is no

max_fds
    Specify the maximum number of file descriptors that can be opened by multipath and multipathd. This is equivalent to ulimit -n. A value of max will set this to the system limit from /proc/sys/fs/nr_open. If this is not set, the maximum number of open fds is taken from the calling process. It is usually 1024. To be safe, this should be set to the maximum number of paths plus 32, if that number is greated than 1024.

checker_timeout
    Specify the timeout to use for path checkers and prioritizers that issue scsi commands with an explicit timeout, in seconds; default taken from /sys/block/sd<x>/device/timeout

fast_io_fail_tmo
    Specify the number of seconds the scsi layer will wait after a problem has been detected on a FC remote port before failing IO to devices on that remote port. This should be smaller than dev_loss_tmo. Setting this to off will disable the timeout.

dev_loss_tmo
    Specify the number of seconds the scsi layer will wait after a problem has been detected on a FC remote port before removing it from the system. This can be set to "infinity" which sets it to the max value of 2147483647 seconds, or 68 years. It will be automatically adjusted to the overall retry interval no_path_retry * polling_interval if a number of retries is given with no_path_retry and the overall retry interval is longer than the specified dev_loss_tmo value. The linux kernel will cap this value to 300 if fast_io_fail_tmo is not set.

queue_without_daemon
    If set to no , when multipathd stops, queueing will be turned off for all devices. This is useful for devices that set no_path_retry. If a machine is shut down while all paths to a device are down, it is possible to hang waiting for IO to return from the device after multipathd has been stopped. Without multipathd running, access to the paths cannot be restored, and the kernel cannot be told to stop queueing IO. Setting queue_without_daemon to no , avoids this problem. Default is yes

bindings_file
    The full pathname of the binding file to be used when the user_friendly_names option is set. Defaults to /etc/multipath/bindings

wwids_file
    The full pathname of the wwids file, which is used by multipath to keep track of the wwids for LUNs it has created multipath devices on in the past. Defaults to /etc/multipath/wwids

log_checker_err
    If set to once , multipathd logs the first path checker error at logging level 2. Any later errors are logged at level 3 until the device is restored. If set to always , multipathd always logs the path checker error at logging level 2. Default is always

reservation_key
    This is the service action reservation key used by mpathpersist. It must be set for all multipath devices using persistent reservations, and it must be the same as the RESERVATION KEY field of the PERSISTENT RESERVE OUT parameter list which contains an 8-byte value provided by the application client to the device server to identify the I_T nexus. It is unset by default.

retain_attached_hw_handler
    If set to yes and the scsi layer has already attached a hardware_handler to the device, multipath will not force the device to use the hardware_handler specified by mutipath.conf. If the scsi layer has not attached a hardware handler, multipath will continue to use its configured hardware handler. Default is no

detect_prio
    If set to yes , multipath will try to detect if the device supports ALUA. If so, the device will automatically use the alua prioritizer. If not, the prioritizer will be selected as usual. Default is no

force_sync
    If set to yes , multipathd will call the path checkers in sync mode only. This means that only one checker will run at a time. This is useful in the case where many multipathd checkers running in parallel causes significant CPU pressure. The Default is no

deferred_remove
    If set to yes , multipathd will do a deferred remove instead of a regular remove when the last path device has been deleted. This means that if the multipath device is still in use, it will be freed when the last user closes it. If path is added to the multipath device before the last user closes it, the deferred remove will be canceled. Default is no

config_dir
    If set to anything other than "", multipath will search this directory alphabetically for file ending in ".conf" and it will read configuration information from them, just as if it was in /etc/multipath.conf. config_dir must either be "" or a fully qualified directory name. Default is /etc/multipath/conf.d

delay_watch_checks
    If set to a value greater than 0, multipathd will watch paths that have recently become valid for this many checks. If they fail again while they are being watched, when they next become valid, they will not be used until they have stayed up for delay_wait_checks checks. Default is no

delay_wait_checks
    If set to a value greater than 0, when a device that has recently come back online fails again within delay_watch_checks checks, the next time it comes back online, it will marked and delayed, and not used until it has passed delay_wait_checks checks. Default is no

uxsock_timeout
    CLI receive timeout in milliseconds. For larger systems CLI commands might timeout before the multipathd lock is released and the CLI command can be processed. This will result in errors like In these cases it is recommended to increase the CLI timeout to avoid those issues. The default is 1000

blacklist section
The blacklist section is used to exclude specific device from inclusion in the multipath topology. It is most commonly used to exclude local disks or LUNs for the array controller.

The following keywords are recognized:

wwid
    The World Wide Identification of a device.

devnode
    Regular expression of the device nodes to be excluded.

property
    Regular expression of the udev property to be excluded.

device
    Subsection for the device description. This subsection recognizes the vendor and product keywords. For a full description of these keywords please see the devices section description.

blacklist_exceptions section
The blacklist_exceptions section is used to revert the actions of the blacklist section, ie to include specific device in the multipath topology. This allows one to selectively include devices which would normally be excluded via the blacklist section.

The following keywords are recognized:

wwid
    The World Wide Identification of a device.

property
    Regular expression of the udev property to be whitelisted. Defaults to (ID_WWN|SCSI_IDENT_.*)

devnode
    Regular expression of the device nodes to be whitelisted.

device
    Subsection for the device description. This subsection recognizes the vendor and product keywords. For a full description of these keywords please see the devices section description.

The property blacklist and whitelist handling is different from the usual handling in the sense that the whitelist has to be set, otherwise the device will be blacklisted. In these cases the message blacklisted, udev property missing will be displayed.

multipaths section
The only recognized attribute for the multipaths section is the multipath subsection.

The multipath subsection recognizes the following attributes:

wwid
    Index of the container. Mandatory for this subsection.

alias
    (Optional) symbolic name for the multipath map.

The following attributes are optional; if not set the default values are taken from the defaults or devices section:

    path_grouping_policy

    path_selector

    prio

    prio_args

    failback

    rr_weight

    flush_on_last_del

    no_path_retry

    rr_min_io

    rr_min_io_rq

    features

    reservation_key

    deferred_remove

    delay_watch_checks

    delay_wait_checks

devices section
The only recognized attribute for the devices section is the device subsection.

The device subsection recognizes the following attributes:

vendor
    (Mandatory) Vendor identifier

product
    (Mandatory) Product identifier

revision
    (Optional) Revision identfier

product_blacklist
    (Optional) Product strings to blacklist for this vendor

alias_prefix
    (Optional) The user_friendly_names prefix to use for this device type, instead of the default "mpath"

hardware_handler
    (Optional) The hardware handler to use for this device type. The following hardware handler are implemented:

    1 emc
        Hardware handler for EMC storage arrays.

    1 rdac
        Hardware handler for LSI/Engenio/NetApp E-Series RDAC storage controller.

    1 hp_sw
        Hardware handler for Compaq/HP storage arrays in active/standby mode.

    1 alua
        Hardware handler for SCSI-3 ALUA compatible arrays.

The following attributes are optional; if not set the default values are taken from the defaults section:

    path_grouping_policy

    uid_attribute

    path_selector

    path_checker

    prio

    prio_args

    features

    failback

    rr_weight

    no_path_retry

    rr_min_io

    rr_min_io_rq

    fast_io_fail_tmo

    dev_loss_tmo

    flush_on_last_del

    retain_attached_hw_handler

    detect_prio

    deferred_remove

    delay_watch_checks

    delay_wait_checks

overrides section
The overrides section recognizes the following optional attributes; if not set the values are taken from the devices or defaults sections:

    path_grouping_policy

    uid_attribute

    getuid_callout

    path_selector

    path_checker

    alias_prefix

    features

    prio

    prio_args

    failback

    rr_weight

    no_path_retry

    rr_min_io

    rr_min_io_rq

    flush_on_last_del

    fast_io_fail_tmo

    dev_loss_tmo

    user_friendly_names

    retain_attached_hw_handler

    detect_prio

    deferred_remove

    delay_watch_checks

    delay_wait_checks

WWID generation
Multipath uses a World Wide Identification (wwid) to determine which paths belong to the same device. Each path presenting the same wwid is assumed to point to the same device.

The wwid is generated by three methods (in the order of preference):

getuid_callout
    Use the specified external program; cf getuid_callout above. Care should be taken when using this method; the external program needs to be loaded from disk for execution, which might lead to deadlock situations in an all-paths-down scenario.

uid_attribute
    Use the value of the specified udev attribute; cf uid_attribute above. This method is preferred to getuid_callout as multipath does not need to call any external programs here. However, under certain circumstances udev might not be able to generate the requested variable.

vpd_pg83
    If none of the getuid_callout or uid_attribute parameters are present multipath will try to use the sysfs attribute vpd_pg83 to generate the wwid.

KNOWN ISSUES
The usage of queue_if_no_path option can lead to D state processes being hung and not killable in situations where all the paths to the LUN go offline. It is advisable to use the no_path_retry option instead.

The use of queue_if_no_path or no_path_retry might lead to a deadlock if the dev_loss_tmo setting results in a device being removed while I/O is still queued. The multipath daemon will update the dev_loss_tmo setting accordingly to avoid this deadlock. Hence if both values are specified the order of precedence is no_path_retry, queue_if_no_path, dev_loss_tmo

SEE ALSO
udev(8), dmsetup(8) multipath(8) multipathd(8)

AUTHORS
multipath was developed by Christophe Varoqui, <christophe.varoqui@opensvc.com> and others.