Академический Документы
Профессиональный Документы
Культура Документы
To make a ksh script (which is a ksh program) crate a new file with a starting line like:
#!/usr/bin/ksh
It is important that the path to the ksh is propper and that the line doesn not have more
than 32 characters. The shell from which you are starting the script will find this line and
and hand the whole script over to to ksh. Without this line the script would be interpreted
by the same typ of shell as the one, from which it was started. But since the syntax is
different for all shells, it is necessary to define the shell with that line.
A script has four types of lines: The shell defining line at the top, empty lines,
commentary lines starting with a # and command lines. See the following top of a script
as an example for these types of lines:
#!/usr/bin/ksh
# Commentary......
file=/path/file
if [[ $file = $1 ]];then
command
fi
The script starts at the first line and ends either when it encounters an "exit" or the last
line. All "#" lines are ignored.
A command starts with the first word on a line or if it's the second command on a line
with the first word after a";'.
A command ends either at the end of the line or whith a ";". So one can put several
commands onto one line:
One can continue commands over more than one line with a "\" immediately followed by
a newline sign which is made be the return key:
The script mus not have a name which is identical to a unix command: So the script must
NOT be called "test"!
After saveing the file give it the execute permissions with: chmod 700 filename.
Variables
Filling in
When filling into a variable then one uses just it's name: state="US" and no blanks. There
is no difference between strings and numbers: price=50.
Using
When using a variable one needs to put a $ sign in front of it: print $state $price.
Arrays
arrname[1]=4 To fill in
print ${arraname[1]} To print out
${arrname[*]} Get all elements
${#arrname[*]} Get the number of elements
Declaration
There are happily no declarations of variables needed in ksh. One cannot have decimals
only integers.
Branching
if then fi
if [[ $value -eq 7 ]];then
print "$value is 7"
fi
or:
if [[ $value -eq 7 ]]
then
print "$value is 7"
fi
or:
if [[ $value -eq 7 ]];then print "$value is 7";fi
if then else fi
if [[ $name = "John" ]];then
print "Your welcome, ${name}."
else
print "Good bye, ${name}!"
fi
case esac
case $var in
john|fred) print $invitation;;
martin) print $declination;;
*) print "Wrong name...";;
esac
Looping
while do done
while [[ $count -gt 0 ]];do
print "\$count is $count"
(( count -= 1 ))
done
until do done
until [[ $answer = "yes" ]];do
print -n "Please enter \"yes\": "
read answer
print ""
done
continue...break
One can skip the rest of a loop and directly go to the next iteration with: "continue".
if [[ $# -eq 0 ]];then
print "No Arguments"
exit
fi
The single Arguments are stored in $1, ....$n and all are in $* as one string. The
arguments cannot
directly be modified but one can reset the hole commandline for another part of the
program.
If we need a first argument $first for the rest of the program we do:
if [[ $1 != $first ]];then
set $first $*
fi
One can iterate over the command line arguments with the help of the shift command.
Shift indirectly removes the first argument.
One can also iterate with the for loop, the default with for is $*:
for arg;do
print $arg
done
Comparisons
To compare strings one uses "=" for equal and "!=" for not equal.
To compare numbers one uses "-eq" for equal "-ne" for not equal as well as "-gt" for
greater than
and "-lt" for less than.
With "&&" for "AND" and "||" for "OR" one can combine statements:
Variable Manipulations
Removing something from a variable
Variables that contain a path can very easily be stripped of it: ${name##*/} gives you just
the filename.
Or if one wants the path: ${name%/*}. % takes it away from the left and # from the right.
%% and ## take the longest possibility while % and # just take the shortest one.
So one can question a string in a variable like: if [[ $var = fo@(?4*67).c ]];then ...
Functions
Description
A function (= procedure) must be defined before it is called, because ksh is interpreted at
run time.
It knows all the variables from the calling shell except the commandline arguments. But
has it's
own command line arguments so that one can call it with different values from different
places in
the script. It has an exit status but cannot return a value like a c funcition can.
Making a Function
One can make one in either of the following two ways:
function foo {
# commands...
}
foo(){
# commands...
}
Return
The return statement exits the function imediately with the specified return value as an
exit status.
Data Redirection
General
Data redirection is done with the follwoing signs: "> >> < <<". Every program has at
least a
To put the error to the same location as the normal output do: command 2>&1
command <<EOF
input1
input2
input3
EOF
From eof to eof all is feeded into the above mentioned command.
Pipes
For a serial processing of data from one command to the next do:
command1 | command2 | command3 ...
e.g. last | awk '{print $1}' | sort -u.
Coprocesses
One can have one background process with which one can comunicate with read -p and
print -p. It is started with command |&. If one uses: ksh |& then this shell in the
background will do everything for us even telnet and so on: print -p "telnet hostname".
Read Input from User and from Files
Read in a Variable
From a user we read with: read var. Then the users can type something in. One should
first print something like: print -n "Enter your favorite haircolor: ";read var; print "". The
-n suppresses the newline sign.
last | sort | {
while read myline;do
# commands
done }
Special Variables
$# Number of arguments on commandline.
$? Exit status of last command.
$$ Process id of current program.
$! Process id of last backgroundjob or background function.
$0 Program name including the path if started from another directory.
$1..n Commandline arguments, each at a time.
$* All commandline arguments in one string.
Trivial Calculations
Simpe calculations are done with either a "let" in front of it or within (( ... )). One can
increment a variable within the (( )) without a "$": (( a+=1 )) or let a+=1.
"grep"
Search for the occurence of a pattern in a file: grep 'pattern' file. If one just wants to know
how often soemthing occurs in a file, then: grep -c 'pattern file. This can be used in a
script like:
if [[ $(grep -c 'pattern' file) != 0 ]];then ......;fi. The condition is fullfilled if the pattern
was found.
"sed"
Sed means stream line editor. It searches like grep, but is then able to replace the found
pattern. If you want to change all occurences of "poor" with "rich", do: sed -e
's/poor/rich/g' filename. Or what is often seen in software packages, that have to be
compiled after getting a propper configuration, is a whole file stuffed with replacements
patterns like: /@foo@/s;;king;g. This file with inumerable lines like that has to be given
to sed with: sed -f sedscript filename. It then precesses each line from file with all the sed
commands in the sedscript. (Of course sed can do much more:-))
"awk"
Awk can find and process a found line with several tools: It can branch, loop, read from
files and also print out to files or to the screen, and it can do arithmetics.
For example: We have a file with lines like: Fred 300 45 70 but hundreds of them. But
some lines have a "#" as the first sign of them and we have to omit these ones for both,
processing and output. And we want to have lines as output like: 415 Fred where 415 is
the sum of 300, 45 and 70. Then we call on awk:
This ignores lines with a "#" at the beginning of the first field and also blank lines. It then
prints the desired sum and the $1 ist only printed after a tab. This is the most trivial use of
awk only.
"perl"
Perl is a much richer programming language then ksh, but still one can do perl commands
from within a ksh script. This might touch Randal, but it's true. Let's say you want to
remove all ^M from a file, then take perl for one line in your ksh script:
Perl can do an infinite amount of things in many different ways. For anything bigger use
perl instead of a shell script.
1. Handling Command Line Arguments
Why is it necessary to write something about command line arguments? The concept is
very easy and clear: if you enter the following command
$ ls -l *.txt
the command "ls" is executed with the command line flag "-l" and all files in the
current directory ending with ".txt" as arguments.
Still many shell scripts do not accept command line arguments the way we are used to
(and came to like) from other standard commands. Some shell programmers do not even
bother implementing command line argument parsing, often aggravating the script's users
with other strange calling conventions.
For examples on how to name command line flags to be consistent with existing UNIX
commands see the table Frequent option names.
• Setting environment variables for script input that could be specified on the
command line.
One example:
:
# AUTORUN must be specified by the user
if [ "$AUTORUN" != yes ]
then
echo "Do you really want to run this script?"
echo "Enter ^D to quit:"
if read answer
then
echo "o.k, starting up memhog daemon"
else
echo "terminating"
exit 0
fi
fi
# start of script...
Consider the script's user who might ponder "What was the name of this variable?
FORCERUN? AUTOSTART? AUTO_RUN? or AUTORUN?"
Don't get me wrong, environment variables do have their place and can make life
easier for the user. A much better way to solve the autorun option would be to
implement a command line flag, i.e. "-f" for "force non-interactive execution".
• Positional parameters.
Example:
:
# process - process input file
ConfigFile="$1"
InputFile="$2"
OutputFile="$3"
This script expects exactly three parameters in exactly this order: the name of a
configuration file with default settings, the name of an input file, and the name of
an output file. The script could be called with the following parameters:
It then reads the configuration file "defaults.cf", processes the input file
"important.dat" and then writes (possibly overwriting) the output file
"output.dat". Now see what happens if you call it like this:
Now the script tries to read the output file "output.dat" as configuration file. If
the user is lucky the script will terminate at this point, before it tries to overwrite
his data file "important.dat" it will be using as the output file!
This script would have been better with the following usage:
The command line option "-c" precedes the default file, the output file is
specified with the "-o" option, and every other argument is taken to be the input
file name.
Our goal are shellscripts, that use "standard" command line flags and options. We will
develop a shell script code fragment that handles command line options well. You may
then use this template in your shell scripts and modify it to fit your needs.
Consider the following command line:
This command line consists of a command ("fgrep") with three flags "-v", "-i" and "-f".
One flag takes an argument ("excludes.list"). After the command line flags multiple file
names ("*.c", "*.h") may follow. At this point we do not know how many file names that
may be; the shell will expand the file name patterns (or "wildcards") to a list of actual file
names before calling the command "fgrep". The command itself does not have to deal
with wildcards.
What happens if there is no file matching the pattern "*.c" in the current directory? In this
case the shell will pass the parameter unchanged to the program.
If we wanted to handle command lines like the above, we must be prepared to handle
The shell sets some environment variables according to the command line arguments
specified:
The name the script was invoked with. This may be a basename without directory
$0 component, or a path name. This variable is not changed with subsequent shift
commands.
$1, $2, The first, second, third, ... command line argument, respectively. The argument
$3, ... may contain whitespace if the argument was quoted, i.e. "two words".
$# Number of command line arguments, not counting the invocation name $0
$@
"$@" is replaced with all command line arguments, enclosed in quotes, i.e. "one",
"two three", "four". Whitespace within an argument is preserved.
$* is replaced with all command line arguments. Whitespace is not preserved, i.e.
$*
"one", "two three", "four" would be changed to "one", "two", "three", "four".
This variable is not used very often, "$@" is the normal case, because it leaves the
arguments unchanged.
The following code segment loops through all command line arguments, and prints them:
:
# cmdtest - print command line arguments
while [ $# -gt 0 ]
do
echo "$1"
shift
done
The shift command "shifts" all command line arguments one position to the left. The
leftmost argument is lost. The following table lists the values of $# and the command line
arguments during the iterations of the while loop:
Now that we can loop through the argument list, we can set script variables depending on
command line flags:
vflag=off
while [ $# -gt 0 ]
do
case "$1" in
-v) vflag=on;;
esac
shift
done
The command line option -v will now result in the variable vflag to be set to "on". We
can then use this variable throughout the script.
Now let's improve this code fragment to handle file names. It would be nice if the script
would handle all command line flags, but leave the file names alone. This way we could
use the shell variable $@ with the remaining command line arguments later on, i.e.
# ...
grep $searchstring "$@"
and be sure that it only contains file names. But how do we recognize file names from
command line switches? That's easy: files do not start with a dash "-" (at least not yet...):
vflag=off
while [ $# -gt 0 ]
do
case "$1" in
-v) vflag=on;;
-*)
echo >&2 "usage: $0 [-v] [file ...]"
exit 1;;
*) break;; # terminate while loop
esac
shift
done
This example prints a short usage message and terminates if an unknown command line
flag starting with a dash was specified. If the current argument does not start with a dash
(and therefore probably is a file name), the while loop is terminated with the break
statement, leaving the file name in the variable "$1".
Now we just need a switch for command line flags with arguments, i.e. "-f filename".
This is also pretty straight forward:
vflag=off
filename=
while [ $# -gt 0 ]
do
case "$1" in
-v) vflag=on;;
-f) filename="$2"; shift;;
-*) echo >&2 \
"usage: $0 [-v] [-f file] [file ...]"
exit 1;;
*) break;; # terminate while loop
esac
shift
done
If the argument $1 is "-f", the next argument ($2) should be the file name. We now
handled two arguments ("-f" and the filename), but the shift after the case construct will
only "consume" one argument. This is the reason why we execute an initial shift after
saving the filename in the variable filename. This shift removes the "-f" flag, while the
second (after the case construct) removes the filename argument.
We still have a problem handling file names starting with a dash ("-"), but that's a
problem every standard unix command interpreting command line switches has. It is
commonly solved by inventing a special command line option named "--" meaning "end
of the option list".
If you for example had a file named "-f", it could not be removed using the command "rm
-f", because "-f" is a valid command line option. Instead you can use "rm -- -f". The
double dash "--" means "end of command line flags", and the following "-f" is then
interpreted as a file name.
Note:
You can also remove a file named "-f" using the command "rm ./-f"
The following (recommended) command line handling code is a good way to solve this
problem:
vflag=off
filename=
while [ $# -gt 0 ]
do
case "$1" in
-v) vflag=on;;
-f) filename="$2"; shift;;
--) shift; break;;
-*)
echo >&2 \
"usage: $0 [-v] [-f file] [file ...]"
exit 1;;
*) break;; # terminate while loop
esac
shift
done
# all command line switches are processed,
# "$@" contains all file names
The drawback of this command line handling is that it needs whitespace between the
option character and an argument, ("-f file" works, but "-ffile" fails), and that multiple
option characters cannot be written behind one switch character, ("-v -l" works, but "-vl"
does not).
Portability:
This method works with all shells derived from the Bourne Shell, i.e. sh, ksh,
ksh93, bash, pdksh, zsh.
Using "getopt"
Now this script processes its command line arguments like any standard UNIX
command, with one exception. Multiple command line flags may be combined with
standard commands, i.e. "ls -l -a -i" may be written as "ls -lai". This is not that easy to
handle from inside of our shell script, but fortunately there is a command that does the
work for us: getopt(1).
The following test shows us, how getopt rewrites the command line arguments "-vl -f
file one two three":
These are the command line flags we would have liked to get! The flags "-vl" are
separated into two flags "-v" and "-l". The command line options are separated from the
file named by a "--" argument.
How did getopt know, that "-f" needed a second argument, but "-v" and "-l" did not? The
first argument to getopt describes, what options are acceptable, and if they have
arguments. An option character followed by a colon (":") means that the option expects
an argument.
Now we are ready to let getopt rewrite the command line arguments for us. Since getopt
writes the rewritten arguments to standard output, we use
vflag=off
filename=
set -- `getopt vf: "$@"`
[ $# -lt 1 ] && exit 1 # getopt failed
while [ $# -gt 0 ]
do
case "$1" in
-v) vflag=on;;
-f) filename="$2"; shift;;
--) shift; break;;
-*)
echo >&2 \
"usage: $0 [-v] [-f file] file ..."
exit 1;;
*) break;; # terminate while loop
esac
shift
done
# all command line switches are processed,
# "$@" contains all file names
This commands do not work with all shells, because the set command doesn't always
return an error code if getopt fails. The line assumes, that getopt sets its return value if
the command line arguments are wrong (which is almost certainly the case) and that set
returns an error code if the command substitution (that executes getopt) fails. This is not
always true.
Why didn't we use getopt in the first place? There is one drawback with the use of getopt:
it removes whitespace within arguments. The command line
Newer shells (Korn Shell, BASH) have the build-in getopts command, which does not
have this problem. This command is described in the following section.
Portability:
The getopt command is part of almost any UNIX system.
Using "getopts"
On newer shells, the getopts command is built-in. Do not confuse it with the older getopt
(without the trailing "s") command. getopts strongly resembles the C library function
getopt(3).
Portability:
The getopts command is an internal command of newer shells. As a rule of thumb
all systems that have the KSH have shells (including the Bourne Shell sh) that
include a built-in getopts command.
• non-interactive mode
rsh -n
-n sort -n
• numeric processing
• process id
ps -p pid
-p mkdir -p
• process path
• quick mode
finger -q, who
-q -q
• quiet mode
• process directories recursively
Note: the flag -R would be better for this purpose.
rm -r
-r • process something in the reverse order sort -r, ls -r
Now you know the standard option names, on to "standard" UNIX commands that do not
use them.
dd - disk dump
The syntax of this command probably is older than UNIX itself. One major
disadvantage is that argument names and file names are written together without
whitespace, i.e. if=mydoc*.txt. The shell will take "if=" as part of the file name,
and cannot expand the wildcards "mydoc*.txt".
With this command option names have more than one character. This makes them
more memorable and more readable. If only all commands would be like this!
And if only -print was a default option!
Often shell script programmers use temporary files in their scripts, and remove them at
the end of the program. This simple and straight forward approach works well as long as
a user does not interrupt the script using a signal (i.e. by pressing ^C or DEL). In this case
the script doesn't have a chance to remove its temporary files before closing.
One example:
:
# viman - start "vi" on a manual page
Tmp=/tmp/viman
This script passes its command line arguments on to the man command, and writes the
result to a temporary file /tmp/viman. Before starting vi on the file, all control
characters are removed ("col -b"), and duplicate or empty lines are removed ("uniq").
After vi terminates, the file is removed.
Consider what happens if two people call this script, one after the other. The first one has
his manual page written to /tmp/viman. Shortly after that the second one has his manual
page written to the same file, overwriting the contents of the first manual page. Now the
first user gets the wrong manual page in the vi editor, and terminates. His instance of the
script removes the file /tmp/viman, and with a little bad luck the first user at the same
time now has an empty file within the vi.
The solution to this problem is clear: each user needs to have a unique temporary file, but
how to do it? We could try to create the temporary file in the directory $HOME. Each user
is (normally) guaranteed to have a unique HOME directory. But even then the user may
overwrite the file if he has a windowing system (like OpenWindows or the Common
Desktop Environment (CDE)) and is logged in more than once with the same HOME
directory.
Steve Bourne (the creator of the Bourne Shell) suggests in The UNIX system to use the
unique process identifier (PID) of the shell script as part of the file name. Since the
process id of the script is always available via the environment variable $$, we could
rewrite the script as follows:
:
# viman - start "vi" with a manual page
Tmp=/tmp/vm$$
But one problem remains: what happens to the temporary file, if the script is terminated
with a signal? In this case, the temporary file may is not removed, because the last line of
the script is never reached!
You may think: "Who cares about files clogging up the /tmp directory? The directory
gets cleaned up automatically anyway!" On the other hand you are reading this text to
become a better shell programmer, and could be excited to come to know there is an easy
way to "trap" signals from a shell script.
Signals may be specified using numbers (0 to 31), "0" being a pseudo-signal meaning
"program termination". The Korn shell also understands names for the signal, i.e. HUP for
HANGUP signal, TERM for the SIGTERM signal etc. Newer kill commands display a
list of signal names if called with the flag -l. The following table lists the most common
signals along with their KSH names:
KSH
Number Comments
name
0 EXIT This number does not correspond to a real signal,
but the corresponding trap is executed before script
termination.
1 HUP hangup
2 INT The interrupt signal typically is generated using the
DEL or the ^C key
3 QUIT The quit signal is typically generated using the
^[ key. It is used like the INT signal but explicitly
requests a core dump.
9 KILL cannot be caught or ignored
10 BUS bus error
11 SEGV segmentation violation
13 PIPE generated if there is a pipeline without reader to
terminate the writing process(es)
15 TERM generated to terminate the process gracefully
16 USR1 user defined signal 1
17 USR2 user defined signal 2
- DEBUG KSH only: This is no signal, but the corresponding
trap code is executed before each statement of the
script.
This means: execute the command "rm -f $Tmp" if the script terminates ("signal" 0), or
after receiving any of the signals 1 (HANGUP), 2 (QUIT), 3 (INTR), or 15 (TERM).
Actually, a good shell script should handle all these signals.
Only one refinement has to be made before we can present The Canonical Way To
Handle Temporary Files ©. Suppose we use the following line in our script:
If somebody sends the SIGTERM signal to our script (i.e. by entering "kill -15
scriptpid"), the following would happen:
1. The script would trap the signal 15, and execute the command "rm -f $Tmp",
thus removing the temporary file.
2. Then it would continue with the next script command. This could cause strange
results, because the (probably needed) temporary file $Tmp is gone. Another point
is that somebody explicitly tried to terminate the script, a fact it deliberately
ignores.
3. Just before the script exits the trap for signal "0" is always performed, resulting in
a second attempt to remove $Tmp. This will result in unwanted error messages
(although in this case it will do no harm).
A better (and the recommended) way to handle the signals is as follows:
The first trap ensures that the temporary file $Tmp is removed at the end of the script
execution. Possible error messages are simply discarded.
The second trap causes our script to terminate after receiving one of the specified signals.
Before the script terminates, the trap for "signal" 0 is executed, effectively removing the
temporary file.
Our original script, now rewritten to handle signals and use unique temporary files looks
as follows:
:
# viman - start "vi" with a manual page
Tmp="${TMPDIR:=/tmp}/vm$$"
${EDITOR:-vi} "$Tmp"
Handling signals requires a bit more overhead; perhaps overkill for simple scripts like
this one but definitely worthwhile for complex scripts.
Solution:
suffix=`date +%Y%m%d%H%M%S`.bak
newFileName=${YourLogName.log}.${suffix}
cp -p YourLogName.log $newFileName
cp /dev/null YourLogName.log
compress $newFileName
A copy of your log is made and compressed and the log file is initialized
(emptied) and ready for more messages. Processing can continue without
interruption. I suppose there is a possibility of a few messages being
dropped (what we don't see won't be missed) , so do this during times of
slow usage with a crontab entry.
if [ -a ${tmpFile} ]; then
rm ${tmpFile};
fi
This, when you use up to 10 or even 5 tmp files gets nasty. A quicker
way
of cleaning
up such tmp files is to use a simple loop, I even perfer to use array's
which are
availible in Korn shell. Here is an example.
clean()
{
tmpfiles[0]=${temp1}
tmpfiles[1]=${temp2}
This way, as you accumulate more and more tmp files, you need only to add
one line to get it cleaned up.
TmpBase=${TMPDIR:=/tmp}/myscript.$$
mkdir "$TmpBase" || exit 1 # create directory
chmod 700 "$TmpBase" || exit 1 # restrict access
D=`dirname "$relpath"`
B=`basename "$relpath"`
abspath="`cd \"$D\" 2>/dev/null && pwd ||
echo \"$D\"`/$B"
Example usage:
Portability:
The "tput" command is available with the "terminfo"
terminal information database
if [ -z "$Host" ]
then
Host=`uname -n`
fi
: ${Host:=`uname -n`}
#/usr/bin/ksh
typeset -i cnt=0
${<arrayname>[<subscript>]}
echo ${myarray[4]}
#!/usr/bin/ksh
typeset -i cnt=0
Have fun.
If you only want the first name, you would declare this variable:
Example:
cat <<!
Please enter your choice
1 - list current directory
2 - list current users
3 - log off
!
echo "
Welcome to Foo.Bar v0.8
=======================
Press enter to continue...
";
}'
An added bonus is the builtins are faster and require less system
resources
because no sub-process is spawned.
basename replacement:
---------------------
$ fullfile="/some/dir/file.txt"
# replaced: file=$(basename $fullfile)
$ file=${fullfile##/*}
$ echo $file
file.txt
dirname replacement:
--------------------
$ fullfile="/some/dir/file.txt"
# replaced: dir=$(dirname $fullfile)
$ dir=${fullfile%/*}
$ echo $dir
/some/dir
tr replacements:
----------------
$ word="MiXeD"
# replaced: word=$(echo $word | tr [A-Z] [a-z])
$ typeset -l word
$ echo $word
mixed
Example:
$ date=
$ read date < /dev/tcp/127.0.0.1/13
$ echo $date
Wed Feb 10 00:45:39 MET 1999
To Reverse a File
Level: Script Submitted by:
URL: none
Programmer radhakrishnan_m@hotmail.com
######## TO PRINT FILE IN REVERSE ORDER BY LINE #############3
if [ $# -ne 1 ]
then
echo "Usage reverse_file <filename> "
exit 1;
fi
set -vx