Handling errors in shell scripts

Avoiding disasters	Now and then users encounter unexpected errors in shell scripts. Common causes are when files or networks are not available or executables crash because of bugs or bad data. Also, shell scripts can be written with unanticipated dependencies on the environment and can be more or less robust in their syntax and error handling. As an example consider the following bourne/korn shell command sequence: # do not do this! cd $subdir rm -rf * There are two potential problems with this. If the variable "subdir" is not defined, the cd sees no argument and the directory is changed to the users HOME directory - then all files and sub-directories are recursively removed (except dot files and directories). If the variable "subdir" is defined but does not specify a valid directory, the cd will fail and the directory will not be changed - then all files etc. in the current directory will be deleted. Fortunately for the user from which this example was taken, HPCCC staff were able to recover the lost files from a backup. But remember, not all file systems on HPCCC systems have backups.
Strategies	Here are several strategies suggested by HPCCC staff to catch errors using shell syntax. This will lessen the chance of such disasters happening. Probably the best solution in this case is to refer to the variable as cd ${subdir:?} In this case the script will exit with an error message if subdir is not defined. Use && or \|\| to test the exit code (only good if $subdir is defined) cd $subdir \|\| exit 1 # will exit if cd $subdir fails rm -rf * # or continue on Test for existence of $subdir first test -n "$subdir" \|\| exit 1 test -d "$subdir" && cd $subdir \|\| exit 1 [ -d $subdir ] && cd $subdir \|\| exit 1 # same as above for ksh You can also use if rather than && or \|\| but that may be awkward if you want to check many commands if cd $subdir; then rm -rf * else exit 1 fi if test -n "$subdir" then if cd $subdir then rm -rf * else echo cd failed - bailing out exit 1 fi else echo variable '$subdir' is not defined - bailing out exit 1 fi
Additional notes	For this specific case the following command would be better - as long as you did not have permission to remove files in the root directory ;-) rm -rf $subdir/* It would be best if your current directory was not under $subdir ... In bourne or korn shell variable expansion can be modified to catch unset values or use a default if unset. ${parameter:?} - exit if parameter unset or null ${parameter:?word} - word is printed if parameter is unset or null and exit. ${parameter:-word} - word is used if parameter is undefined The korn shell command "set -u" causes errors to be generated if variables are used but not defined. You still need to catch the error. In C shell, the behaviour is different. The script will exit if a command fails (not in a test). If the variable is not defined that will also trigger an error by default. For korn shell (or bash), use trap to set up the shell to catch the error and run a command. The simplest command to run is "exit" or "exit $?" so that the script is aborted. The following command should be executed somewhere early in the script, and will apply to all following commands (which don't have their exit value checked by inbuilt shell control structures (if && \|\| ! etc.) trap 'exit $?' ERR However, this would not have worked in the example in question because there is no error generated. We continue to recommend the use of Korn shell (or bash) for batch jobs, but recommend that care be taken to catch errors.

Avoiding disasters

Now and then users encounter unexpected errors in shell scripts. Common causes are when files or networks are not available or executables crash because of bugs or bad data. Also, shell scripts can be written with unanticipated dependencies on the environment and can be more or less robust in their syntax and error handling. As an example consider the following bourne/korn shell command sequence:

        # do not do this!
        cd $subdir
        rm -rf *

There are two potential problems with this.

If the variable "subdir" is not defined, the cd sees no argument and the directory is changed to the users HOME directory - then all files and sub-directories are recursively removed (except dot files and directories).
If the variable "subdir" is defined but does not specify a valid directory, the cd will fail and the directory will not be changed - then all files etc. in the current directory will be deleted.

Fortunately for the user from which this example was taken, HPCCC staff were able to recover the lost files from a backup. But remember, not all file systems on HPCCC systems have backups.

Strategies

Here are several strategies suggested by HPCCC staff to catch errors using shell syntax. This will lessen the chance of such disasters happening.

Probably the best solution in this case is to refer to the variable as
```
   cd ${subdir:?}
```
In this case the script will exit with an error message if subdir is not defined.

Use && or || to test the exit code (only good if $subdir is defined)

   cd $subdir || exit 1 # will exit if cd $subdir fails
   rm -rf *             # or continue on

Test for existence of $subdir first

   test -n "$subdir" || exit 1

   test -d "$subdir" && cd $subdir || exit 1

   [ -d $subdir ] && cd $subdir || exit 1 # same as above for ksh

You can also use if rather than && or || but that may be awkward if you want to check many commands

    if cd $subdir;
    then rm -rf *
    else exit 1
    fi

    if test -n "$subdir"
    then
        if cd $subdir
        then
            rm -rf *
        else
            echo cd failed - bailing out
            exit 1
        fi
    else
        echo variable '$subdir' is not defined - bailing out
        exit 1
    fi

Additional notes

For this specific case the following command would be better - as long as you did not have permission to remove files in the root directory ;-)
```
   rm -rf $subdir/*
```
It would be best if your current directory was not under $subdir ...

In bourne or korn shell variable expansion can be modified to catch unset values or use a default if unset.

     ${parameter:?}       - exit if parameter unset or null
     ${parameter:?word}   - word is printed if parameter is unset or null and exit.
     ${parameter:-word}   - word is used if parameter is undefined

The korn shell command "set -u" causes errors to be generated if variables are used but not defined. You still need to catch the error.
In C shell, the behaviour is different. The script will exit if a command fails (not in a test). If the variable is not defined that will also trigger an error by default.
For korn shell (or bash), use trap to set up the shell to catch the error and run a command. The simplest command to run is "exit" or "exit $?" so that the script is aborted. The following command should be executed somewhere early in the script, and will apply to all following commands (which don't have their exit value checked by inbuilt shell control structures (if && || ! etc.)
```
    trap 'exit $?' ERR
```
However, this would not have worked in the example in question because there is no error generated.
We continue to recommend the use of Korn shell (or bash) for batch jobs, but recommend that care be taken to catch errors.

CSIRO Advanced Scientific Computing

Handling errors in shell scripts