Parameter Substitution

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting
Prev | Chapter 10. Manipulating Variables | Next

# 10.2. Parameter Substitution

Manipulating and/or expanding variables

`${parameter}`

  Same as `$parameter`, i.e., value of the variable `parameter`. In certain contexts, only the less ambiguous `${parameter}` form works.

  May be used for concatenating variables with strings.

         1 your_id=${USER}-on-${HOSTNAME}
         2 echo "$your_id"
         3 #
         4 echo "Old \$PATH = $PATH"
         5 PATH=${PATH}:/opt/bin  # Add /opt/bin to $PATH for duration of script.
         6 echo "New \$PATH = $PATH"

`${parameter-default}`, `${parameter:-default}`

  If parameter not set, use default.

         1 var1=1
         2 var2=2
         3 # var3 is unset.
         4
         5 echo ${var1-$var2}   # 1
         6 echo ${var3-$var2}   # 2
         7 #           ^          Note the $ prefix.
         8
         9
        10
        11 echo ${username-`whoami`}
        12 # Echoes the result of `whoami`, if variable $username is still unset.

  | `${parameter-default}` and `${parameter:-default}` are almost equivalent. The extra : makes a difference only when `parameter` has been declared, but is null.

         1 #!/bin/bash
         2 # param-sub.sh
         3
         4 #  Whether a variable has been declared
         5 #+ affects triggering of the default option
         6 #+ even if the variable is null.
         7
         8 username0=
         9 echo "username0 has been declared, but is set to null."
        10 echo "username0 = ${username0-`whoami`}"
        11 # Will not echo.
        12
        13 echo
        14
        15 echo username1 has not been declared.
        16 echo "username1 = ${username1-`whoami`}"
        17 # Will echo.
        18
        19 username2=
        20 echo "username2 has been declared, but is set to null."
        21 echo "username2 = ${username2:-`whoami`}"
        22 #                            ^
        23 # Will echo because of :- rather than just - in condition test.
        24 # Compare to first instance, above.
        25
        26
        27 #
        28
        29 # Once again:
        30
        31 variable=
        32 # variable has been declared, but is set to null.
        33
        34 echo "${variable-0}"    # (no output)
        35 echo "${variable:-1}"   # 1
        36 #               ^
        37
        38 unset variable
        39
        40 echo "${variable-2}"    # 2
        41 echo "${variable:-3}"   # 3
        42
        43 exit 0

  The default parameter construct finds use in providing "missing" command-line arguments in scripts.

         1 DEFAULT_FILENAME=generic.data
         2 filename=${1:-$DEFAULT_FILENAME}
         3 #  If not otherwise specified, the following command block operates
         4 #+ on the file "generic.data".
         5 #  Begin-Command-Block
         6 #  ...
         7 #  ...
         8 #  ...
         9 #  End-Command-Block
        10
        11
        12
        13 #  From "hanoi2.bash" example:
        14 DISKS=${1:-E_NOPARAM}   # Must specify how many disks.
        15 #  Set $DISKS to $1 command-line-parameter,
        16 #+ or to $E_NOPARAM if that is unset.

  See also Example 3-4, Example 31-2, and Example A-6.

  Compare this method with using an and list to supply a default command-line argument.

`${parameter=default}`, `${parameter:=default}`

  If parameter not set, set it to default.

  Both forms nearly equivalent. The : makes a difference only when `$parameter` has been declared and is null, [1] as above.

         1 echo ${var=abc}   # abc
         2 echo ${var=xyz}   # abc
         3 # $var had already been set to abc, so it did not change.

`${parameter+alt_value}`, `${parameter:+alt_value}`

  If parameter set, use `alt_value`, else use null string.

  Both forms nearly equivalent. The : makes a difference only when `parameter` has been declared and is null, see below.

         1 echo "###### \${parameter+alt_value} ########"
         2 echo
         3
         4 a=${param1+xyz}
         5 echo "a = $a"      # a =
         6
         7 param2=
         8 a=${param2+xyz}
         9 echo "a = $a"      # a = xyz
        10
        11 param3=123
        12 a=${param3+xyz}
        13 echo "a = $a"      # a = xyz
        14
        15 echo
        16 echo "###### \${parameter:+alt_value} ########"
        17 echo
        18
        19 a=${param4:+xyz}
        20 echo "a = $a"      # a =
        21
        22 param5=
        23 a=${param5:+xyz}
        24 echo "a = $a"      # a =
        25 # Different result from   a=${param5+xyz}
        26
        27 param6=123
        28 a=${param6:+xyz}
        29 echo "a = $a"      # a = xyz

`${parameter?err_msg}`, `${parameter:?err_msg}`

  If parameter set, use it, else print err_msg and abort the script with an exit status of 1.

  Both forms nearly equivalent. The : makes a difference only when `parameter` has been declared and is null, as above.

Example 10-7. Using parameter substitution and error messages

       1 #!/bin/bash
       2
       3 #  Check some of the system's environmental variables.
       4 #  This is good preventative maintenance.
       5 #  If, for example, $USER, the name of the person at the console, is not set,
       6 #+ the machine will not recognize you.
       7
       8 : ${HOSTNAME?} ${USER?} ${HOME?} ${MAIL?}
       9   echo
      10   echo "Name of the machine is $HOSTNAME."
      11   echo "You are $USER."
      12   echo "Your home directory is $HOME."
      13   echo "Your mail INBOX is located in $MAIL."
      14   echo
      15   echo "If you are reading this message,"
      16   echo "critical environmental variables have been set."
      17   echo
      18   echo
      19
      20 # ------------------------------------------------------
      21
      22 #  The ${variablename?} construction can also check
      23 #+ for variables set within the script.
      24
      25 ThisVariable=Value-of-ThisVariable
      26 #  Note, by the way, that string variables may be set
      27 #+ to characters disallowed in their names.
      28 : ${ThisVariable?}
      29 echo "Value of ThisVariable is $ThisVariable".
      30
      31 echo; echo
      32
      33
      34 : ${ZZXy23AB?"ZZXy23AB has not been set."}
      35 #  Since ZZXy23AB has not been set,
      36 #+ then the script terminates with an error message.
      37
      38 # You can specify the error message.
      39 # : ${variablename?"ERROR MESSAGE"}
      40
      41
      42 # Same result with:   dummy_variable=${ZZXy23AB?}
      43 #                     dummy_variable=${ZZXy23AB?"ZXy23AB has not been set."}
      44 #
      45 #                     echo ${ZZXy23AB?} >/dev/null
      46
      47 #  Compare these methods of checking whether a variable has been set
      48 #+ with "set -u" . . .
      49
      50
      51
      52 echo "You will not see this message, because script already terminated."
      53
      54 HERE=0
      55 exit $HERE   # Will NOT exit here.
      56
      57 # In fact, this script will return an exit status (echo $?) of 1.

Example 10-8. Parameter substitution and "usage" messages

       1 #!/bin/bash
       2 # usage-message.sh
       3
       4 : ${1?"Usage: $0 ARGUMENT"}
       5 #  Script exits here if command-line parameter absent,
       6 #+ with following error message.
       7 #    usage-message.sh: 1: Usage: usage-message.sh ARGUMENT
       8
       9 echo "These two lines echo only if command-line parameter given."
      10 echo "command-line parameter = \"$1\""
      11
      12 exit 0  # Will exit here only if command-line parameter present.
      13
      14 # Check the exit status, both with and without command-line parameter.
      15 # If command-line parameter present, then "$?" is 0.
      16 # If not, then "$?" is 1.

Parameter substitution and/or expansion. The following expressions are the complement to the match `in` expr string operations (see Example 16-9). These particular ones are used mostly in parsing file path names.

Variable length / Substring removal

`${#var}`

  `String length` (number of characters in `$var`). For an array, ${#array} is the length of the first element in the array.

  | Exceptions: ${#*} and ${#@} give the number of positional parameters. For an array, ${#array[*]} and ${#array[@]} give the number of elements in the array.

  Example 10-9. Length of a variable

         1 #!/bin/bash
         2 # length.sh
         3
         4 E_NO_ARGS=65
         5
         6 if [ $# -eq 0 ]  # Must have command-line args to demo script.
         7 then
         8   echo "Please invoke this script with one or more command-line arguments."
         9   exit $E_NO_ARGS
        10 fi
        11
        12 var01=abcdEFGH28ij
        13 echo "var01 = ${var01}"
        14 echo "Length of var01 = ${#var01}"
        15 # Now, let's try embedding a space.
        16 var02="abcd EFGH28ij"
        17 echo "var02 = ${var02}"
        18 echo "Length of var02 = ${#var02}"
        19
        20 echo "Number of command-line arguments passed to script = ${#@}"
        21 echo "Number of command-line arguments passed to script = ${#*}"
        22
        23 exit 0

`${var#Pattern}`, `${var##Pattern}`

  ${var#Pattern} Remove from `$var` the shortest part of `$Pattern` that matches the `front end` of `$var`.

  ${var##Pattern} Remove from `$var` the longest part of `$Pattern` that matches the `front end` of `$var`.

  A usage illustration from Example A-7:

         1 # Function from "days-between.sh" example.
         2 # Strips leading zero(s) from argument passed.
         3
         4 strip_leading_zero () #  Strip possible leading zero(s)
         5 {                     #+ from argument passed.
         6   return=${1#0}       #  The "1" refers to "$1" -- passed arg.
         7 }                     #  The "0" is what to remove from "$1" -- strips zeros.

  Manfred Schwarb's more elaborate variation of the above:

         1 strip_leading_zero2 () # Strip possible leading zero(s), since otherwise
         2 {                      # Bash will interpret such numbers as octal values.
         3   shopt -s extglob     # Turn on extended globbing.
         4   local val=${1##+(0)} # Use local variable, longest matching series of 0's.
         5   shopt -u extglob     # Turn off extended globbing.
         6   _strip_leading_zero2=${val:-0}
         7                        # If input was 0, return 0 instead of "".
         8 }

  Another usage illustration:

         1 echo `basename $PWD`        # Basename of current working directory.
         2 echo "${PWD##*/}"           # Basename of current working directory.
         3 echo
         4 echo `basename $0`          # Name of script.
         5 echo $0                     # Name of script.
         6 echo "${0##*/}"             # Name of script.
         7 echo
         8 filename=test.data
         9 echo "${filename##*.}"      # data
        10                             # Extension of filename.

`${var%Pattern}`, `${var%%Pattern}`

  ${var%Pattern} Remove from `$var` the shortest part of `$Pattern` that matches the `back end` of `$var`.

  ${var%%Pattern} Remove from `$var` the longest part of `$Pattern` that matches the `back end` of `$var`.

Version 2 of Bash added additional options.

Example 10-10. Pattern matching in parameter substitution

       1 #!/bin/bash
       2 # patt-matching.sh
       3
       4 # Pattern matching  using the # ## % %% parameter substitution operators.
       5
       6 var1=abcd12345abc6789
       7 pattern1=a*c  # * (wild card) matches everything between a - c.
       8
       9 echo
      10 echo "var1 = $var1"           # abcd12345abc6789
      11 echo "var1 = ${var1}"         # abcd12345abc6789
      12                               # (alternate form)
      13 echo "Number of characters in ${var1} = ${#var1}"
      14 echo
      15
      16 echo "pattern1 = $pattern1"   # a*c  (everything between 'a' and 'c')
      17 echo "--------------"
      18 echo '${var1#$pattern1}  =' "${var1#$pattern1}"    #         d12345abc6789
      19 # Shortest possible match, strips out first 3 characters  abcd12345abc6789
      20 #                                     ^^^^^               |-|
      21 echo '${var1##$pattern1} =' "${var1##$pattern1}"   #                  6789
      22 # Longest possible match, strips out first 12 characters  abcd12345abc6789
      23 #                                    ^^^^^                |----------|
      24
      25 echo; echo; echo
      26
      27 pattern2=b*9            # everything between 'b' and '9'
      28 echo "var1 = $var1"     # Still  abcd12345abc6789
      29 echo
      30 echo "pattern2 = $pattern2"
      31 echo "--------------"
      32 echo '${var1%pattern2}  =' "${var1%$pattern2}"     #     abcd12345a
      33 # Shortest possible match, strips out last 6 characters  abcd12345abc6789
      34 #                                     ^^^^                         |----|
      35 echo '${var1%%pattern2} =' "${var1%%$pattern2}"    #     a
      36 # Longest possible match, strips out last 12 characters  abcd12345abc6789
      37 #                                    ^^^^                 |-------------|
      38
      39 # Remember, # and ## work from the left end (beginning) of string,
      40 #           % and %% work from the right end.
      41
      42 echo
      43
      44 exit 0

Example 10-11. Renaming file extensions:

       1 #!/bin/bash
       2 # rfe.sh: Renaming file extensions.
       3 #
       4 #         rfe old_extension new_extension
       5 #
       6 # Example:
       7 # To rename all *.gif files in working directory to *.jpg,
       8 #          rfe gif jpg
       9
      10
      11 E_BADARGS=65
      12
      13 case $# in
      14   0|1)             # The vertical bar means "or" in this context.
      15   echo "Usage: `basename $0` old_file_suffix new_file_suffix"
      16   exit $E_BADARGS  # If 0 or 1 arg, then bail out.
      17   ;;
      18 esac
      19
      20
      21 for filename in *.$1
      22 # Traverse list of files ending with 1st argument.
      23 do
      24   mv $filename ${filename%$1}$2
      25   #  Strip off part of filename matching 1st argument,
      26   #+ then append 2nd argument.
      27 done
      28
      29 exit 0

Variable expansion / Substring replacement

  These constructs have been adopted from ksh.

`${var:pos}`

  Variable `var` expanded, starting from offset `pos`.

`${var:pos:len}`

  Expansion to a max of `len` characters of variable `var`, from offset `pos`. See Example A-13 for an example of the creative use of this operator.

`${var/Pattern/Replacement}`

  First match of `Pattern`, within `var` replaced with `Replacement`.

  If `Replacement` is omitted, then the first match of `Pattern` is replaced by nothing, that is, deleted.

`${var//Pattern/Replacement}`

  Global replacement. All matches of `Pattern`, within `var` replaced with `Replacement`.

  As above, if `Replacement` is omitted, then all occurrences of `Pattern` are replaced by nothing, that is, deleted.

  Example 10-12. Using pattern matching to parse arbitrary strings

         1 #!/bin/bash
         2
         3 var1=abcd-1234-defg
         4 echo "var1 = $var1"
         5
         6 t=${var1#*-*}
         7 echo "var1 (with everything, up to and including first - stripped out) = $t"
         8 #  t=${var1#*-}  works just the same,
         9 #+ since # matches the shortest string,
        10 #+ and * matches everything preceding, including an empty string.
        11 # (Thanks, Stephane Chazelas, for pointing this out.)
        12
        13 t=${var1##*-*}
        14 echo "If var1 contains a \"-\", returns empty string...   var1 = $t"
        15
        16
        17 t=${var1%*-*}
        18 echo "var1 (with everything from the last - on stripped out) = $t"
        19
        20 echo
        21
        22 # -------------------------------------------
        23 path_name=/home/bozo/ideas/thoughts.for.today
        24 # -------------------------------------------
        25 echo "path_name = $path_name"
        26 t=${path_name##/*/}
        27 echo "path_name, stripped of prefixes = $t"
        28 # Same effect as   t=`basename $path_name` in this particular case.
        29 #  t=${path_name%/}; t=${t##*/}   is a more general solution,
        30 #+ but still fails sometimes.
        31 #  If $path_name ends with a newline, then `basename $path_name` will not work,
        32 #+ but the above expression will.
        33 # (Thanks, S.C.)
        34
        35 t=${path_name%/*.*}
        36 # Same effect as   t=`dirname $path_name`
        37 echo "path_name, stripped of suffixes = $t"
        38 # These will fail in some cases, such as "../", "/foo////", # "foo/", "/".
        39 #  Removing suffixes, especially when the basename has no suffix,
        40 #+ but the dirname does, also complicates matters.
        41 # (Thanks, S.C.)
        42
        43 echo
        44
        45 t=${path_name:11}
        46 echo "$path_name, with first 11 chars stripped off = $t"
        47 t=${path_name:11:5}
        48 echo "$path_name, with first 11 chars stripped off, length 5 = $t"
        49
        50 echo
        51
        52 t=${path_name/bozo/clown}
        53 echo "$path_name with \"bozo\" replaced  by \"clown\" = $t"
        54 t=${path_name/today/}
        55 echo "$path_name with \"today\" deleted = $t"
        56 t=${path_name//o/O}
        57 echo "$path_name with all o's capitalized = $t"
        58 t=${path_name//o/}
        59 echo "$path_name with all o's deleted = $t"
        60
        61 exit 0

`${var/#Pattern/Replacement}`

  If prefix of `var` matches `Pattern`, then substitute `Replacement` for `Pattern`.

`${var/%Pattern/Replacement}`

  If suffix of `var` matches `Pattern`, then substitute `Replacement` for `Pattern`.

  Example 10-13. Matching patterns at prefix or suffix of string

         1 #!/bin/bash
         2 # var-match.sh:
         3 # Demo of pattern replacement at prefix / suffix of string.
         4
         5 v0=abc1234zip1234abc    # Original variable.
         6 echo "v0 = $v0"         # abc1234zip1234abc
         7 echo
         8
         9 # Match at prefix (beginning) of string.
        10 v1=${v0/#abc/ABCDEF}    # abc1234zip1234abc
        11                         # |-|
        12 echo "v1 = $v1"         # ABCDEF1234zip1234abc
        13                         # |----|
        14
        15 # Match at suffix (end) of string.
        16 v2=${v0/%abc/ABCDEF}    # abc1234zip123abc
        17                         #              |-|
        18 echo "v2 = $v2"         # abc1234zip1234ABCDEF
        19                         #               |----|
        20
        21 echo
        22
        23 #  ----------------------------------------------------
        24 #  Must match at beginning / end of string,
        25 #+ otherwise no replacement results.
        26 #  ----------------------------------------------------
        27 v3=${v0/#123/000}       # Matches, but not at beginning.
        28 echo "v3 = $v3"         # abc1234zip1234abc
        29                         # NO REPLACEMENT.
        30 v4=${v0/%123/000}       # Matches, but not at end.
        31 echo "v4 = $v4"         # abc1234zip1234abc
        32                         # NO REPLACEMENT.
        33
        34 exit 0

`${!varprefix*}`, `${!varprefix@}`

  Matches names of all previously declared variables beginning with `varprefix`.

         1 # This is a variation on indirect reference, but with a * or @.
         2 # Bash, version 2.04, adds this feature.
         3
         4 xyz23=whatever
         5 xyz24=
         6
         7 a=${!xyz*}         #  Expands to *names* of declared variables
         8 # ^ ^   ^           + beginning with "xyz".
         9 echo "a = $a"      #  a = xyz23 xyz24
        10 a=${!xyz@}         #  Same as above.
        11 echo "a = $a"      #  a = xyz23 xyz24
        12
        13 echo "---"
        14
        15 abc23=something_else
        16 b=${!abc*}
        17 echo "b = $b"      #  b = abc23
        18 c=${!b}            #  Now, the more familiar type of indirect reference.
        19 echo $c            #  something_else

### Notes

[1] | If $parameter is null in a non-interactive script, it will terminate with a 127 exit status (the Bash error code for "command not found").

Prev | Home | Next
Manipulating Variables | Up | Loops and Branches