13.8.1 Bash TAB completion tutorial ¶

When a user presses the [TAB] key while typing commands, Bash will inspect the input to find a relevant “completion specification”, or compspec. If available, the compspec will generate a list of possible suggestions to complete the current word. A custom compsec can be generated for any command using bash completion builtins³⁰⁴ and the bash variables that start with the COMP keyword³⁰⁵.

First, let’s see a quick example of how you can make a completion script in just one line of code. With the command below, we are asking Bash to give us three suggestions for echo: foo, bar and bAr. Please run it in your terminal for the next steps.

$ complete -W "foo bar bAr" echo

The possible completion suggestions are fed into complete using the -W option followed by a list of space delimited words. Let’s see it in action:

$ echo [TAB][TAB]
bar  bAr  foo

Nicely done! Just note that the strings are sorted alphabetically, not in the original order. Also, an arbitrary number of space characters are printed between them (based on the number of suggestions and terminal size, etc.). Now, if you type ‘f’ and press [TAB], bash will automatically figure out that you wanted foo and it be completed right away:

$ myprogram f[TAB]
$ myprogram foo

However, nothing will happen if you type ‘b’ and press [TAB] only once. This is because of the ambiguity: there is not enough information to figure out which suggestion you want: bar or bAr? So, if you press [TAB] twice, it will print out all the options that start with ‘b’:

$ echo b[TAB][TAB]
bar  bAr
$ echo ba[TAB]
$ echo bar

Not bad for a simple program. But what if you need more control? By passing the -F option to complete instead of -W, it will run a function for generating the suggestions, instead of using a static string. For example, let’s assume that the expected value after foo is the number of files in the current directory. Since the logic is getting more complex, let’s write and save the commands below into a shell script with an arbitrary name such as completion-tutorial.sh:

$ cat completion-tutorial.sh
_echo(){
    if [ "$3" == "foo" ]; then
      COMPREPLY=( $(ls | wc -l) )
    else
      COMPREPLY=( $(compgen -W "foo bar bAr" -- "$2") )
    fi
}
complete -F _echo echo

We will look at it in detail soon. But for now, let’s source the file into your current terminal and check if it works as expected:

$ source completion-tutorial.sh
$ echo [TAB][TAB]
foo bar bAr
$ echo foo [TAB]
$ touch empty.txt
$ echo foo [TAB]

Success! As you see, this allows for setting up highly customized completion scripts. Now let’s have a closer look at the completion-tutorial.sh completion script from above. First, the ‘-F’ option in front the complete command indicates that we want shell to execute the _echo function whenever echo is called. As a convention, the function name should be the same as the program name, but prefixed with an underscore (‘_’).

Within the _echo function, we’re checking if $3 is equal to foo. In Bash’s auto-complete, $3 means the word before current cursor position. In fact, these are the arguments that the _echo function is receiving:

$1

The name of the command, here it is ‘echo’.

$2

The current word being completed (empty unless we are in the middle of typing a word).

$3

The word before the word being completed.

To tell the completion script what to reply with, we use the COMPREPLY array. This array holds all the suggestions that complete will show for the user in the end. In the example above, we simply give it the string output of ‘ls | wc -l’.

Finally, we have the compgen command. According to bash programmable completion builtins manual, the command compgen [OPTION] [WORD] generates possible completion matches for [WORD] according to [OPTIONS]. Using the ‘-W’ option asks compgen to generate a list of words from an input string. This is known as Word Splitting³⁰⁶. compgen will automatically use the $IFS variable to split the string into a list of words. You can check the default delimiters by calling:

$ printf %q "$IFS"

The default value of $IFS might be ‘ \t\n’. This means the SPACE, TAB, and New-line characters. Finally, notice the ‘-- "$2"’ in this command:

COMPREPLY=( $(compgen -W "foo bar bAr" -- "$2") )

Here, the ‘--’ instructs compgen to only reply with a list of words that match $2, i.e. the current word being completed. That is why when you type the letter ‘b’, complete will reply only with its matches (‘bar’ and ‘bAr’), and will exclude ‘foo’.

Let’s get a little more realistic, and develop a very basic completion script for one of Gnuastro’s programs. Since the --help option will list all the options available in Gnuastro’s programs, we are going to use its output and create a very basic TAB completion for it. Note that the actual TAB completion in Gnuastro is a little more complex than this and fully described in Implementing TAB completion in Gnuastro. But this is a good exercise to get started.

We will use asttable as the demo, and the goal is to suggest all options that this program has to offer. You can print all of them (with a lot of extra information) with this command:

$ asttable --help

Let’s write an awk script that prints all of the long options. When printing the option names we can safely ignore the short options because if a user knows about the short options, s/he already knows exactly what they want! Also, due to their single-character length, they will be too cryptic without their descriptions.

One way to catch the long options is through awk as shown below. We only keep the lines that 1) starting with an empty space, 2) their first no-white character is ‘-’ and that have the format of ‘--’ followed by any number of numbers or characters. Within those lines, if the first word ends in a comma (‘,’), the first word is the short option, so we want the second word (which is the long option). Otherwise, the first word is the long option. But for options that take a value, this will also include the format of the value (for example, --column=STR). So with a sed command, we remove everything that is after the equal sign, but keep the equal sign itself (to highlight to the user that this option should have a value).

$ asttable --help \
           | awk '/^  / && $1 ~ /^-/ && /--+[a-zA-Z0-9]*/ { \
                    if($1 ~ /,$/) name=$2; \
                    else          name=$1; \
                    print name}' \
           | sed -e's|=.*|=|'

If we wanted to show all the options to the user, we could simply feed the values of the command above to compgen and COMPREPLY subsequently. But, we need smarter completions: we want to offer suggestions based on the previous options that have already been typed in. Just Beware! Sometimes the program might not be acting as you expected. In that case, using debug messages can clear things up. You can add a echo command before the completion function ends, and check all current variables. This can save a lot of headaches, since things can get complex.

Take the option --wcsfile= for example. This option accepts a FITS file. Usually, the user is trying to feed a FITS file from the current directory. So it would be nice if we could help them and print only a list of FITS files sitting in the current directory – or whatever directory they have typed-in so far.

But there’s a catch. When splitting the user’s input line, Bash will consider ‘=’ as a separate word. To avoid getting caught in changing the IFS or WORDBREAKS values, we will simply check for ‘=’ and act accordingly. That is, if the previous word is a ‘=’, we will ignore it and take the word before that as the previous word. Also, if the current word is a ‘=’, ignore it completely. Taking all of that into consideration, the code below might serve well:

_asttable(){
    if [ "$2" = "=" ]; then word=""
    else                    word="$2"
    fi

    if [ "$3" = "=" ]; then prev="${COMP_WORDS[COMP_CWORD-2]}"
    else                    prev="${COMP_WORDS[COMP_CWORD-1]}"
    fi

    case "$prev" in
      --wcsfile)
        COMPREPLY=( $(compgen -f -X "!*.[fF][iI][tT][sS]" -- "$word") )
      ;;
    esac
}
complete -o nospace -F _asttable asttable

To test the code above, write it into asttable-tutorial.sh, and load it into your running terminal with this command:

$ source asttable-tutorial.sh

If you then go to a directory that has at least one FITS file (with a .fits suffix, among other files), you can checkout the function by typing the following command. You will see that only files ending in .fits are shown, not any other file.

asttable --wcsfile=[TAB][TAB]

The code above first identifies the current and previous words. It then checks if the previous word is equal to --wcsfile and if so, fills COMPREPLY array with the necessary suggestions. We are using case here (instead of if) because in a real scenario, we need to check many more values and case is far better suited for such cases (cleaner and more efficient code).

The -f option in compgen indicates we’re looking for a file. The -X option filters out the filenames that match the next regular expression pattern. Therefore we should start the regular expression with ‘!’ if we want the files matching the regular expression. The -- "$word" component collects only filenames that match the current word being typed. And last but not least, the ‘-o nospace’ option in the complete command instructs the completion script to not append a white space after each suggestion. That is important because the long format of an option, its value is more clear when it sticks to the option name with a ‘=’ sign.

You have now written a very basic and working TAB completion script that can easily be generalized to include more options (and be good for a single/simple program). However, Gnuastro has many programs that share many similar things and the options are not independent. Also, complex situations do often come up: for example, some people use a .fit suffix for FITS files and others do not even use a suffix at all! So in practice, things need to get a little more complicated, but the core concept is what you learnt in this section. We just modularize the process (breaking logically independent steps into separate functions to use in different situations). In Implementing TAB completion in Gnuastro, we will review the generalities of Gnuastro’s implementation of Bash TAB completion.