Friday, November 1, 2013

Unix Shell Scripts Are Like A Human Language...

If you don't use them regularly, it is amazing how quickly the subtle little details start to evaporate away.  I was writing a Linux bash script to do something fairly simple but clever (because I am too lazy to do this by hand).  In this case, I had a list of Stored Procedure Language calls, with the number of calls made by the application, and I wanted to see which SPLs on the list referenced a superfluous Informix table called usp_num.  I did not want to lose the number of calls information, so I wrote a script to read the lines, search in the current workspace for each SPL, and see if there was a reference to usp_num in it, then output the original list, along with whether the SPL was not present in the workspace into which all the SPLs have been loaded from Subversion.  (You do not want to know why SPLs are being called, but are not in the workspace.  Oh my.)
echo "references","SPL","usp_num present","missing SPL"
for line in `tail -n +2 ~/Windows7Documents/splcounts | awk '{print $1,$2}'`
refCount=`echo $line | awk '{print $1}'`
splname=`echo $line | awk '{print $2}'`
# First see if the SPL is even present
if [ -f $splname.sql ]; then
  grep usp_num $splname.sql >/dev/null
  if [ $? = 0 ]; then
    echo $refCount,$splname,"x",
    echo $refCount,$splname," ",
  echo $refCount,$splname.sql,,not found
I had completely forgotten that the for construct treats white space as a line delimiter.  That's why you need the IFS=$'\n' statement up there.


  1. I think I learned that detail of how 'for' works in an implicit manner.

    That is, by trial and error.

    I had a similar, if simpler, task at work this week. I had to comb through some log files and examine the timing of certain events.

    So I used a script that looked something like:
    for i in `ls *.log`; do
    echo $i
    grep "PATTERN_1" $i;
    grep "PATTERN_2" $i;
    echo "======"

    simple to handle, when output is piped into a single text file...

  2. Back in the day, I'd have been much more likely to write a complicated script as a wrapper around a single awk script, than as a bunch of shell commands mixed with single awk statements.

    I found keeping track of all the oddities inherent in shell programming to be simply too much work. (My favorite - remembering under which circumstances a sequence of commands would be executed in a subshell, and would thus be operating in a different environment with a different set of variables.)

    These days, I do anything of consequence in Python.

  3. If you value terseness (over, arguably, clarity), I think you could write the body of the inner if as:

    echo -n $refCount,$splname,
    (grep -q usp_num $splname.sql) && echo "X," || echo " ,"

    But I agree with jdege's comments. Except he inexplicably misspelled "Perl" as "Python".

  4. I love perl scripts: it's like some sadistic genius decided that line noise could actually be put to use.