Tuesday, March 24, 2009

Diffing the output of two commands

Today, while doing some server maintenance, I needed to diff the output of two different commands. To be more specified, it was actually the same command, but with different sets arguments. As an example, let's say I wanted to diff the contents of two folders, /tmp/folder1 and /tmp/folder2.

What I had done in the past was:

ls -l /tmp/folder1 > tmp.txt
ls -l /tmp/folder2 | diff tmp.txt
rm tmp.txt


Today I decided there had to be a better way to do that and set out to discover how. What I found was something that I had long forgotten about: named pipes. If you're not familiar with named pipes, check Wikipedia. For a refresher of how they work in Unix, here's an example:

mkfifo mypipe

then in two separate shells type:

ls -l > mypipe

and

cat < mypipe

Now, with a pipe already created, I can accomplish my task a little more quickly and efficiently:

ls -l /tmp/folder1 > mypipe&
cat ls -l /tmp/folder2 | diff < mypipe


Howevery after reading all the way through this great Linux Journal article I learned that what would work even better is the great <(...) syntax:

diff <(ls /tmp/folder1) <(ls /tmp/folder2)

This does exactly what I wanted to do without needing to create any temporary files or named pipes myself. What's happening here is that BASH creates a temporary named pipe and fills it using the output of the command between the parens.

As you can see, temporary pipes make these quick one-offs much, much easier.

Take a look at that awesome article for a peek at more awesome stuff you can do with named pipes.

I'll have to admit that I've not used named pipes much (maybe only once or twice that I can remember), but after this, I'll definitely keep my eyes peeled for more excellent piping opportunities.

Enjoy!

9 comments:

  1. This is awesome. Thanks a lot.

    I love to use this to check my drives (I have four):
    gvimdiff <(smartctl -a /dev/sda) <(smartctl -a /dev/sdb) <(smartctl -a /dev/sdc) <(smartctl -a /dev/sdd)

    Too bad it doesn't work with:
    gvimdiff <smartctrl\ -a\ /dev/sd{a..d}

    I guess that would be /too/ sexy.

    ReplyDelete
  2. It is also awesome to check hashes of downloads. Just copy the md5 or sha1 or what ever hash in to it like this:

    diff <(sha1sum debian-6.0.3-i386-netinst.iso) <(echo "7d9498542435b0272763f480d07abe8044d26c04 debian-6.0.3-i386-netinst.iso")

    Output for me is empty as it is an exact match!

    ReplyDelete
  3. This was very helpful. Thank you.

    ReplyDelete
  4. Exactly what I was looking for!
    Great work!

    ReplyDelete
  5. Can i know similar in AIX ??

    ReplyDelete
  6. also try this:
    diff <(ls {/tmp/folder1, /tmp/folder2})

    ReplyDelete