Posted on

Bash Tips & Tricks

bashYesterday on the-list I was asking about some weird behavior about `test`, so it made me want to write about some other Bash scripting pitfalls which I have fallen into before. Maybe some of these you’ve run into before, maybe some you haven’t. Anyways, I hope you find something here helpful. Note that these apply to Bash. If you write scripts with other shells like zsh or ksh, these may or may not apply.

Reading and Writing to the Same File

You cannot read and write to the same file in the same command. One of two things will happen:

  1. The file will be clobbered to zero bytes.
  2. The file will grow until it consumes all available space.

This means you cannot write things like

$ cat file | some_command > file

You can’t do this because the pipe tells the shell to read from the file, but then `some_command` is writing out to it. Which takes precedence? There is not a reliable standard on this matter. Your only option is to create a temporary file.

$ cat file > /tmp/file && some_command /tmp/file

This will guarantee that the contents from the `cat` will be written out (flushed in C library terms) before the other command.

The Error Status of `cd`

Changing directories seems harmless, but you *must* check the return value of the `cd` command. Please always do this! Consider this in a script:

$ cd foo; rm *.php

Now what if `foo` does not exist? The `cd` will fail and you will delete all of the PHP scripts in a directory that you didn’t intend. Whoops. A way to avoid this is to explicitly check the results of `cd` by exiting the script if it fails, e.g.:

$ cd foo || exit 1; rm *.php

This `||` means ‘either change to the directory foo, or if you can’t, exit completely’. I really suggest doing this with every `cd`. If your script depends on changing directories, you do not want that to fail; God forbid you run a series of ‘dangerous’ commands in an unexpected directory.

Looping Over File Names

You may see this a lot online, as a way to loop over file names:

for script in $(ls *.php); do 

Or even:

for script in $(find . -type f -name '*.php'); do 

Both of these forms will break in Bash for any file that has a space, because of word splitting rules. Let’s say you have a file called ‘user processor.php’. Both loops will execute twice, where the variable `$script` is equal to `user` and then `processor.php`. Which is not what you want in either case. The best thing to do is the same file glob:

for script in *.php; do 

This will do the right thing.

Posted on

Changing Directories More Easily

Here is something I have in my Bash config that I have found useful these days. It defines a command called up that lets me move up a given number of directories. For example, up 2 is the same as cd ../.., and up 4 is cd ../../../.., and so on.

function up() {
     cd $(perl -e 'print join("/" => ("..") x shift)' $1)

I found this somewhere online, so I am not taking credit for it. The way this works is we use Perl to create the string ../../.., or however many dots and slashes we need to reach the right parent directory. We can create that string to go up three directories by using the code

("..") x 3

to create the list

(".." ".." "..")

We then use join to insert a slash between each set of dots. This gives us code very close to what is in the function above. The key difference is the use of shift. We don’t know ahead of time how many .. strings to create, since that depends on how many directories upward we want to move. What we want to do then is pass in the number of ..‘s to create as an argument to the Perl script. By default shift will pop off the first command-line argument to the script, which will be our number.

This is how we end up with

perl -e 'print join("/" => ("..") x shift)' $1

Here $1 refers to the first argument of the up shell function. So when we use up 3 we get

perl -e 'print join("/" => ("..") x shift)' 3

which gives us

print join("/" => ("..") x 3)
print join("/" => (".." ".." ".."))
print "../../.."

That string is finally returned as the argument to cd, which moves us up the right number of directories.

Related to this are some aliases I use to treat and as a stack of directories. Bash has two commands called pushd and popd. The former will change to the given directory and put it on the stack. The latter will pop the top of the stack and move to the directory that is now at the top. So I use these aliases to those commands:

alias bd="popd"
alias cd="pushd"
alias rd="popd -n"

The mnemonic for bd is to go ‘back a directory’. The rd alias ‘removes a directory’; it takes the top directory off the stack without switching to it. This is sometimes useful when I end up deleting a directory on the stack, because then ‘bd’ will complain with an error if I try to move back to it.

The command dirs will show you the stack, starting with the current directory on the left. Once you get used to it, I think this is a useful way of moving around directories.