Unix Shell Scripting

Course Objectives

The participant will learn Linux Introduction Linux file related commands Linux text manipulation commands Environment variables The vi-editor Shell programming Awk programming

Introduction to Linux

Features of Linux System: 1. Simple design, organization and functioning 2. Portability 3. Background processing 4. Hierarchical File System 5. Multi-user 6. Multi-tasking 7. Security 8. Interactive Operating System 9. Communication facilities 10. Utilities
3

Introduction to Linux

Utility Software: 1. Text Manipulation tools: cut, grep, tr, etc. 2. Advanced filters: sed, awk, etc. 3. Document Formatting tools: troff, nroff, etc. 4. Various Programming Languages: C, C++, Java, etc. 5. Interactive Calculators : bc, dc, etc. 6. Advanced tools : lex, yacc.

Linux File System

The Linux file system is the structure in which all the information on your computer is stored. It has hierarchical file structure, where each directory can contain files, as well as other directories. It looks like an upside down tree. Linux looks at everything as a file. At the top is the root directory, represented by a forward slash (/). At the children node of it is a set of common directories in the Linux system, such as /bin, /dev, /home, etc.
5

Linux File System

The diagram shows the Linux file structure / bin/ home/ itp1/ test1.sh etc/ dev/

itp2/ test2.sh

Linux File System

In Windows file systems, drive letters represent different storage devices like A: is a floppy drive, C: is a hard disk, etc. In Linux, all storage devices are in the same file system hierarchy. Windows file systems uses backslash (\) to separate directory names, whereas Linux uses forward slash (/) for the same. Every file and directory in a Linux system has permissions and ownership associated with it. File names have suffixes in Windows, whereas in Linux you can use them as conventions.
7

Linux File System

The chunk of the disk is divided into blocks (1024bytes block, or 4096-bytes block, etc.) in multiple of 512 bytes in size. These blocks are organized into four groupings- boot block, superblock, inode blocks and data blocks.
FS1 FS2
FS3

Boot block

Super block

inode list

Data blocks ..

Linux Users

Users Super user Owner Group Others

1. 2. 3. 4.

Introduction to Linux

Logging on and Logging off. Step 1: telnet <System Address> Step 2: login : ravindra Step 3: password : ****** Login incorrect Step 4: login : ravindra Step 5: password : ****** Step 6: $ Command prompt to execute commands Step 7: $ logout
10

Unix Shells

The Bourne shell /bin/sh (written by S. R. Bourne). Along came the people from UCB and the C-shell /bin/csh was born. Into this shell they put several concepts which were new, (the majority of these being job control and aliasing) and managed to produce a shell that was much better for interactive use. Eventually David Korn from AT&T had the bright idea to sort out this mess and the Korn shell /bin/ksh made its appearance. The Korn shell became part of System V but had one major problem; unlike the rest of the UNIX shells it wasn't free, you had to pay AT&T for it.
11

Unix Shells

Also at about this time the GNU project was underway and they decided that they needed a free shell, they also decided that they wanted to make this new shell POSIX compatible, thus bash (the Bourne again shell) was born. Like the Korn shell bash was based upon the Bourne shells language and like the Korn shell, it also pinched features from the C shell and other operating systems.

12

Unix Shells

The Bourne Shell The Bourne shell is the original UNIX shell program. It is very widely used. You can start the Bourne shellif it hasn't been set as your default startup shellby typing "sh" or "/bin/sh" at the command prompt. This will not spawn a new shell window, but rather will just change your current shell to the Bourne shell. The Bourne shell supports conditional branching in the form of if/then/else statements. In addition, the Bourne shell supports case statements and loops (for, while, and until). The Bourne shell uses the $ as a prompt.
13

Unix Shells

The Korn shell The Korn shell is a much newer variation of the Bourne shell. It supports everything the Bourne shell does, and adds features not available in the Bourne shell. The Korn shell is not a standard offering in UNIX installations. If you have the Korn shell, you can run it by typing ksh or /bin/ksh at the shell prompt. A public-domain version of the Korn shell, called pdksh. The Korn shell was originally written by David Korn and is copyrighted by AT&T. The programming structure of the Korn shell is very similar to that of the Bourne shell. The Korn shell, 14 however, is more interactive.

Unix Shells

The C shell The C shell is a very commonly used shell. Its programming structure closely resembles that of the programming language "C." The C shell uses the "%" as a prompt. The C shell supports all of the features that the Bourne shell supports, and has a more natural syntax for programming. The C shell is more interactive than the Bourne shell, with additional features that aren't available in older shells. The configuration of the C shell is controlled by the .rc and the .login files. 15

Unix Shells

The tc shell The tc shell is a more modern variation of the C shell. It reads the same configuration files that the C shell uses. Tcsh contains command line editing keystrokes that the C shell is missing, and has more "modern" conveniences that the C shell lacks.

16

Unix Shells

The Bourne-Again shell The Bourne-Again shell is a variation of the Bourne shell. It is commonly used in Linux, but is widely available in other standard UNIX distributions. The Bourne Again shell is another modification of the Bourne shell, and uses the $ as a prompt. To start the Bourne Again shell, type "bash" at the shell prompt. The behavior and environment of the Bourne Again shell is controlled by the .bashrc file, which you'll find as a hidden file in your home directory.
17

Unix Shells

Unix Shell application comparison table Application sh csh ksh bash tcsh Job control N Y Y Y Y Aliases N Y Y Y Y Input/Output redirection Y N Y Y N Command history N Y Y Y Y Command line editing N N Y Y Y Vi Command line editing N N Y Y Y Underlying Syntax sh csh ksh sh csh
18

Basic commands in Linux

$ who The command shows who is logged on $ who am I The command shows who are you $ who \ > am \ >i\ We can type a command over two or more lines. A backslash character before the end of the line followed by a new line is taken to be continuation of the line.
19

Commands In Linux

$ man The command formats and displays on-line manual pages. The manual pages are divided into logical grouping of the commands called the sections. Sections are numbered from 1 through 9. For example, commands are 1, system calls are 2, library function are 3, file formats are 5, & management commands are 8. If section is specified, man only looks in that section of the manual. The command man 2 open will display the manual for the system call open.

20

Commands In Linux

Option k

-f

Description searches a set of database files containing short descriptions of system commands for keywords and displays the result on the standard output. gives one line introduction to the command (only complete word matches are displayed).

21

Basic commands in Linux

$ date The command prints or sets the system date and time $ date r TestFile It displays the last modification time of the file TestFile $ date string The super user can invoke the date command with a numeric argument to set the system date. This argument is usually an eight-character string of the form MMDDhhmm (month, day, hour in 24hour format, minutes), and optionally followed by 22 a two-digit year string.

Basic commands in Linux

The command can be used with suitable format specifiers as arguments. Each format is preceded by a + symbol, followed by the % operator, and a single character describing the format. Sequence Interpretation +%a abbreviated weekday name (Mon .. Sun) +%b abbreviated month name (Jan .. Dec) +%d day of month (01 .. 31) +%r time (12- hour) +%T time (24- hour) +%y last two digits of year (00 .. 99) +%D date (mm/dd/yy)
23

Commands In Linux

$ ls The command lists contents of directories Option Description -l list in long format -C multicolumn output -F indicates type of file by /, * -R recursive listing of all subdirectories encountered -a list all files including hidden files
24

Commands In Linux

$ cat The command concatenates files and prints on the standard output $ cat file1 file2 It displays contents of all files specified on the command line one below the other $ cat > file It creates a new file by accepting text from the standard input Press CTRL-d to save and exit the file
25

Commands In Linux

$ mkdir [-p] dirname The command makes a directory dirname. The dirname can contain the full path prefix of the directory to be created. When executed with p option, it wont give any error if the directory dirname already exists. When executed with option p, it makes parent directories in the path if needed (if any parent directory in the path is not available) .

26

Commands In Linux

cd [directory] change working directory to the directory if specified or to the home directory otherwise

27

Commands In Linux

$ pwd The command prints name of your current working directory

28

Commands In Linux

$cp Copies files and directories $ cp file1 file2 It copies file1 to file2 $ cp file1 file2 dest_directory It copies multiple files in the specified existing directory $ cp -r directory1 directory2 dest_directory Recursively copies files from directory1, directory2 etc. to the dest_directory
29

Commands In Linux

$ mv The command changes name of your file or moves your file to the specified destination path $ mv file1 new-file It renames file1 as new-file $ mv file1 file2 dest_directory It moves multiple files to the specified existing directory $ mv directory1 directory2 dest_directory moves one or more directory subtrees to an existing or new dest_directory
30

Commands In Linux

$ rm The command is used to get rid of unwanted files/directories $ rm [-i] file It is interactive removal (option i) of specified

files.

$ rm -r directory It is recursive deletion of all files within the specified directories and also the directories themselves.
31

Commands In Linux

Wild characters ^ beginning of the line $ end of the line ? Matches any character [ ] set of characters character hyphen(-) represents range { } a number enclosed in it specifies the number of times the preceding expression is to be repeated ! represents negation
32

Linux Users

Users Super user Owner Group Others

1. 2. 3. 4.

33

Commands In Linux

$ chmod The command changes file permission. Using absolute value. 4 - Read, 2 - Write, 1 Execute $ chmod 740 zz

34

Commands In Linux

Using Symbolic Mode Code Meaning a all u user g group o other + add remove = assign
35

Commands In Linux

$ umask The command changes initial Permission of newly created file. The value of the argument can be calculated by subtracting the mode you want as default from the current default mode. Assume the current default mode is 0666 and you want it as 0644 then 666 644 = 022 will be the parameter which we have to pass with umask command. $umask 0 sets default mode which is 0666.
36

Text Manipulation Commands

Filters : Filters : A filter is a shell command which takes input from the standard input, processes it, and sends its output to the standard output. At run time, the system supplies data to the filter as standard input. This std input file can not be altered by the program.

37

Text Manipulation Commands

An important characteristic of filter is that all the input, output and error channels have the same structure. They are all unstructured byte streams data delimited by an end-of-data marker. Some widely used filters in Unix are: sort, grep, and wc. Filters allows the users to create the complex programs from the simpler programs. The commands cp, mv and cd are not filter commands.
38

Text Manipulation Commands

wc : word count. $ wc -[wlc] [filename] head : displays first n lines $ head -[n] [filename] tail : displays last n lines $ tail -[n] [filename] split : divides files horizontally $ split -[n] [filename] we get m subfiles of size n (xaa, xab,).

39

Text Manipulation Commands

$
cut -[cfd] [filename] -c characters -f field no -d field separator $ cut -c2-5 sample cuts characters from 2 to 5 from file sample. continued...

40

Text Manipulation Commands

$ cut -c1,2 file cuts 1st and 2nd characters from file. $ cut -d -f1-2 file cuts first 2 fields of the file. $ cut -c2- names cuts from 2nd char to the end of line.

41

Text Manipulation Commands

paste: join files column-wise $ paste -d[field separator] [list of files] uniq: removes duplicate lines from a sorted file $ uniq [dcf] [file] -d only print duplicate lines -c prefix lines with number of occurences -f2 avoid comparing first two fields.

42

Text Manipulation Commands

sort: Sorting text files Self study. Try to find out different options for this command using man pages.

43

Text Manipulation Commands

tee: read from standard input and write to standard output and files $ls l | tee dircontents displays the directory contents on the standard output as well as stores them to file dircontents. nl: number lines of files nl [options] [files]

44

Text Manipulation Commands

tr : Translating Characters. $ tr [options] < [file] -d : deletes specified characters. -cd : do not delete specified characters. -s : substitute characters. $ tr abc ABC < samp : replaces all occurrences of a with A, b with B, c with C.

45

Commands In Linux

grep: it is called as global regular expression

pattern. It searches for a given pattern in the file(s)

$ grep -[cvnl] [pattern] [files] Option Description -c counts the total no of lines containing that pattern. -v displays all the lines not containing that pattern. -n displays lines with line no. -l displays file names containing the pattern.
46

Commands In Linux

fgrep: fast searching for fixed strings It handles fixed character strings as text patterns. It cannot process wild-card matches, character classes. egrep: used to search with full regular expressions. It is called as extended grep. egrep (Ravi|Ravindra) Joshi employee.txt This will search for Ravi Joshi as well as for Ravindra Joshi
47

Commands In Linux

$ ps It shows process status. Display the attribute of a process. Options $ ps -f --> Full option $ ps -f -u itp5 --> gives processes of user itp5 $ ps -a --> all users processes. $ ps -e --> all processes on the system including system processes.
48

Commands In Linux

$ kill The command is used to terminate a process. The command uses one or more PIDs as its arguments. $ Kill 105 It will terminate job with PID 105. $ kill -9 121 The option 9 indicates sure kill signal. $ kill $! The system variable $! Stores the PID of the last background job.
49

Commands In Linux

To execute any process in background just use & at the end of the command. $ sh test & To bring the process in foreground use fg To check out jobs currently running in background use command --> jobs $ jobs

50

The Environment

The Unix system is controlled by a number of shell variables that are separately set by the system some during boot sequence, and some after logging in. These variables are called system variables or environment variables. The set statement displays the complete list of all these variables. Built-in variable names are defined in uppercase.

51

The Environment

The PATH : is a variable that instructs the shell about the route it should follow to locate any executable command. The HOME : when you log in, UNIX normally places you in a directory named after your login name. The MAIL : determines where all incoming mail addressed to the user is to be stored. The PS1 and PS2 : PS1 - your command prompt and PS2-Multi-line command string. The SHELL: determines the type of shell that a user sees on logging in.
52

The Environment

.bash_profile : the script executed during login time. Every time you make changes to it, you should log out and log in again. The .bash_profile must be located in your home directory, and it is executed after /etc/profile, the universal profile for all users. Universal environment settings are kept by the administrator in /etc/profile so that they are available to all users.

53

The Environment

ALIASES : it allows you to assign short-hand names for commands you may be using quite frequently. This is done with the alias statement. Consider following ex. $ alias l=ls -l Aliases are listed when the alias statement is used without argument. The alias feature also allows you to incorporate positional parameters as variables in an alias. For ex. $ alias showdir=cd $1 ; ls -l When you want to see the contents of the directory /home/itp5 use the alias as 54 $ showdir /home/itp5

The Environment

Setting the Environment variable: Environment variables can provide a way of storing information that you need to use often from the shell. Assigning a value to a variable will set an environment variable temporarily. For example: $ x=50 This example will set value 50 to the variable x. Use the export (discussed latter) command to export the value to the shell so that it can be propagated to other shells you may open.
55

The Environment

The variable will not be available if you exit the shell. Add these variables to configuration files (discussed earlier) to set them permanently. If you no longer want a variable to be set, you can use the unset command to erase its value. For example $ unset x This command will cause x to have no value set.

56

The Environment

By default,the values stored in shell variables are local to the shell, i.e., they are available only in the shell in which they are defined. They are not passed on to a child shell. But the shell can also export those variables recursively to all child processes so that, once defined, they are available globally. This is done with the export command. For example, consider a simple script $vi exporttest.sh echo the value of x is $x x=50 echo the new value of x is $x

57

The Environment

Now at the command prompt, first assign value 25 to x, and then execute the above script and observe, we will get following output $ x=25 $ ./exporttest.sh the value of x is the new value of x is 50 $ echo $x $ 25 Because x is a local variable in the login shell, its value cant be accessed in the script exporttest.sh.
58

The Environment

We will use the export command and observe the change. $ x=25 $ export x $ ./exporttest.sh the value of x is 25 the new value of x is 50 $ echo $x 25 When x is exported, its assigned value 25 is available in the script. A reassignment (x=50) in the script which is a sub-shell, is not seen in the parent shell which executed the script. 59

The Environment

Executing commands from a file: . (dot)

If we have a list of commands in a file sample then we can execute it using command . sample It is like executing a shell script, but with following difference: Standard shell scripts cause a new sub shell to be created to run the script. The dot command, uses the same shell and it just needs redirection to take the commands from the file instead of from the keyboard.
60

The Environment

The dot command can change the value of a shell variable in the current shell. The export command lets you pass a variables value from parent shell to child shell, but there is no comparable mechanism for passing a value from child to parent. The dot command, however, creates no child process, so any changes it produces apply to the original shell. It also doesnt require the script to have executable permission.
61

The Environment

For example, consider the simple script $vi dottest.sh echo the value of x is $x x=50 echo the new value of x is $x Now we will observe the change by exporting variable x and executing the script using dot command.

62

The Environment
$ x=25 $ export x $ . dottest.sh $ the value of x is 25 $ the new value of x is 50 $ echo $x $ 50 When x is exported, its assigned value 25 is available in the script. A reassignment (x=50) in the script has also changed the value of x in the parent shell which executed the script.

63

The vi Editor
A vi session begins by invoking the command vi with or without a filename. $ vi After executing this command you will get a full empty screen, each line beginning with a ~(tilde). The last line in the screen is reserved for some commands that you can enter to act on the text. This line is also used by the system to display messages.

64

Modes of operation
The three different modes of operations are: 1. Command mode : This is the mode where you can pass the commands to act on the text, using most of the keys of the keyboard. Example: Key x is used to delete the character at the cursor position. You can switch to this mode using ESC key. contd..

65

Modes of operation
2. Insert (Input) mode: To enter the text, you have to
enter into input mode. Press key i to enter into insert mode from command mode. 3. ex mode or line mode : You have to save your file or switch to another file or make a global substitution in the file. You then have to use ex mode, where you can enter the instruction in the last line of the screen. To enter into this mode press ESC & : Example: ex mode command :wq will save the current file and will quit from the editor.

66

Text insertion commands

1. 2. 3. 4. 5.

Command i a I A o O

6.

Description inserts text at cursor position inserts text after cursor position inserts text at beginning of line inserts text after end of line opens line below current line to insert text opens line above current line to insert text
67

Cursor movement commands

1. 2. 3. 4. 5. 6. 7. 8.

Command h l k j w b 0 $

Description left by one character right by one character up by one line down by one line right by one word left by one word beginning of line end of line
68

Text deletion commands

1. 2. 3. 4. 5. 6. 7.

Command x X dw d0 d$ [n]dd [n]dd p

Description character under cursor character before cursor word beginning to cursor position cursor position to end of line n lines from current line p will paste deleted lines to current cursor position. Equivalent to Ctrl-X and Ctrl-V [in Windows]
69

Text copy commands

1. 2. 3. 4. 5. 6.

Command y y0 y$ yw [n]yy
[n]yy p

Description character beginning to cursor position cursor position to end of line word n lines from current line in to the buffer p will paste copied lines to current cursor position. Equivalent to Ctrl-C and Ctrl-V [in windows]
70

Text modification commands

Command 1. nc [space] 2. c0 Description overwrites next n characters overwrites the portion between beginning of line to cursor position 3. c$ overwrites the portion between cursor to end of line 4. cw overwrites current word 5. :[addr1,addr2]s/pattern1/pattern2[/g] globally replaces pattern1 with pattern2 on the specified lines
71

File related commands

1. 2. 3. 4. 5.

Command Description ZZ or :wq save and exit :w save & continue editing :q! quit without saving :r filnam insert file filnam :[addr1,addr2]w filname write the lines between line number addr1 and line number addr2 in the file filname
contd..
72

File related commands

Command 1. a4yy 2. ap Description copy 4 lines into buffer a paste contents of buffer a to current cursor position. Maximum 26 buffers are available buffer having names a to z select particular columns copy selected into buffer b paste contents of buffer b to current cursor position. contd..
73

1. ctrl-v 2. byy 3. bp

File related commands

1.

2. 3. 4. 5. 6. 7.

Command Description 1,$s/source/target/ substitute string source by string target from line number 1 to last line u undo last change on the line U undo last changes on the line Ctrl-R redo the undone changes. e edit multiple files e# return to original file rew back to first file in command line
contd..
74

Visual mode commands

1. 2. 3. 4. 5. 6.

Command v_u
v_U sp Ctrl-w <Ctrl-w>j <Ctrl-w>k

Description converts the character to lower case converts the character to upper case splitting window toggle between windows moves to lower window moves to upper window
75

Customizing vi

The set command

When the string no is prefixed to any option, it indicates that the option is inoperative. Command Description :Set all displays all set option :set autoindent does automatic indentation :set number shows all line duly numbered :set showmatch helps to locate matching brackets :set tabstop=5 sets tab=5 for display
76

1. 2. 3. 4. 5.

Advanced Commands

The .exrc File and exinit : all sets, maps and

abbreviations can be conveniently stored in the file $HOME/.exrc Vi looks for this file on startup and executes the instructions as ex mode command

77

Shell Programming

When a group of commands has to be executed regularly, they are stored in a file. All such files are called shell scripts, shell programs, or shell procedures. There is no restrictions on extension of these files, but conventionally extension .sh is used for a shell program. You can use the vi editor to create the shell script.

78

Shell Programming

You can execute the shell scripts using either command sh or by just typing shell script name at the prompt (make sure that you have execute permission). $ sh test.sh $ test.sh $ ./test.sh

79

Shell Programming
User-created Shell Variables : variable=value => will assign value to variable. $variable => will print value of the variable. Command echo : it displays message on the screen. Consider following ex. $ var=hello $ echo $var $var $var with this the output will be $ hello hello $var 2. read var => will read value from keyboard to var. $ read var

80

Shell Programming

Operators Arithmetic : + - / * % Relational : -lt -le -gt -ge eq ne Logical : -a -o Expression expr 10 + 20 Assignment statement. x=77 Command substitution: when a command is enclosed in back quotes, the command is replaced by the output it produces. Ex. The command x=`expr $x + 1` will set x=78
81

Shell Programming

Shell Metacharacters:

Wildcard substitution: * ? [] Redirection: > >> < 2> << Piping: | Command substitution: ` ` Sequential commands: semicolon (;) Inserting comment: # Command grouping: () parenthesis Ex. (date; cat employee.txt) > Report This will copy contents of employee.txt with current date to a file Report.
82

Shell Programming

Creating a here document (<< operator): It allows a shell script to get its input from the same file that holds the script. Data used this way is said to be in a here document. To create a here document, we use the << operator. Any command using standard input can also have the input from a here document. The here document symbol (<<) followed by the data, and a delimiter (the termination string).

83

Shell Programming

The example for here document $ wc << EOF >This is for testing the here document. It will >give us the count of lines, words and characters >till the terminator given after here document >symbol, the terminator which is the EOF. We will get output of the command as number of lines, words, characters till first occurrence of the EOF symbol in the script We can use any string as a terminator.
84

Shell Programming

Debugging the scripts Command set is used for debugging purpose The v option of sh causes the shell to echo each
command before it is executed. Command Action set v sets the debugging mode on sh v sample.sh echoes each command of script sample.sh before it is executed set +v turns off v option of sh We can also use x option. What is difference?
85

1. 2. 3.

Shell Programming

Conditional Statement if then else Syntax: if test <condition> then command1 command2 else command3 fi The if statement takes two-way decisions depending on the condition .
86

Shell Programming

String Tests used by the command test String Comparison True if string1 = string2 strings equal string1 != string2 strings not equal -n string string not null -z string string is null

87

Shell Programming

Arithmetic expression comparisons used by the command test Arithmetic comparison True if expr1 -eq expr2 expressions equal expr1 -ne expr2 expressions not equal expr1 -gt expr2 expr1 > expr2 expr1 -ge expr2 expr1 >= expr2 expr1 -lt expr2 expr1 < expr2 expr1 -le expr2 expr1 <=expr2 ! expression expression is false
88

Shell Programming

String Tests used by the command test Test True if -d file file exists and is a directory -e file file exists -f file file exists and is a regular file -r file file exists and is a readable -s file file exists and has a size > 0 -w file file exists and is a writable -x file file exists and is a executable

89

Shell Programming

Conditional Statement - case Syntax: case $value in val1) command1 command2;; val2) command3;; . *) command4;; esac The statement matches an expression for more than one alternative, and permits multi-way branching.
90

Shell Programming

Example of case statement: echo Enter the color read color case $color in Red | red) echo You have selected red color;; Blue | blue) echo You have selected blue color;; *) echo Sorry! Yet to add this color in our list esac
91

Shell Programming

Loop Statement - while Syntax: while test <condition> do command1 command2 done While statement repeatedly performs a set of instructions till the control command returns a true exit status.
92

Shell Programming

Loop Statement until Syntax: until test <condition> do command1 command2 done The set of instructions is executed repeatedly as long as the condition remains false. The until statement complements the while statement.
93

Shell Programming

Unconditional loop for Syntax for <var> in <val1 val2 > do command1 command2 done The loop body (do-done) is executed as many times as there are items in the list. It doesnt test condition but uses a list.
94

Shell Programming

Example of for statement Assume file Users.txt contains names of users. for name in `cat Users.txt` do echo Hello, $name done This statement when executed, will print the message Hello, user-name for each user name from the file.
95

Shell Programming

Command Line parameters (positional parameters) When arguments are specified with a shell script, they are assigned to variables called positional parameters. The first argument is read by the shell into the parameter $1, the second into the parameter $2, and so on. The $# represents total no of arguments passed to the script. The command is assigned to a variable $0. You can use these variables up to $9.
96

Shell Programming

The $* indicates all arguments, in a single variable, separated by the first character in the environment variable IFS The $@ is same as $* except when enclosed in double quotes. The $@ works with string input. The parameter $? stores the exit status of the last command. On success, the command stores value 0 in $?, and a non-zero value if it fails.

97

Shell Programming

set : Assigning values to positional parameters. $ set 23 532 The command assigns value 23 to the positional parameter $1, value 532 to $2. It also sets $#, $*.

98

Shell Programming

shift : shifting arguments to left. The command shift copies the contents of a positional parameter to its immediate lower numbered positional parameter. When called once, contents of $2 are copied to $1, $3 to $2 and so on. $ shift 2 The command does two shifts i.e. $1=$3, $2=$4, and so on. Using shift command we can use more than 9 command line parameters.
99

Shell Programming

Finding the length of the string: The regular expression .* signifies to command expr that it has to print the number of characters matching the pattern, i.e., the length of the entire string. We can also use keyword length with command expr $ expr abcd : .* $4 $ expr length abcd $4
100

Shell Programming

Command grouping using parenthesis ()

Commands are grouped using parenthesis () Using parenthesis, we can collectively redirect the standard output of two commands with a single redirection symbol. For example: $ (date; cat employee.txt) > Report.txt This will redirect the collective output which is current date and employee.txt file contents to Report.txt file.
101

Shell Programming

Command grouping using curly braces {}

The curly braces {} are also used to group commands. The difference between () and {} is that parenthesis () executes the commands group in a sub-shell, while the curly braces {} uses the current shell only. The closing curly brace must be on a separate line by itself, or simply terminate the last command with a semi-colon. For example, observe output of the following: $ pwd ; ( cd /home/ravindra/test ; pwd) ; pwd $ pwd ; { cd /home/ravindra/test ; pwd; } ; pwd
102

Shell Programming

Shell Functions:
A function consists of a group of statements which are executed together as a bunch. For function we need function definition (body of function) and function call. A shell function must precede the statements that call it. The return statement, when present, returns a value representing the success or failure of the function. The return statement is optional.
103

Shell Programming

The syntax for function definition:

function_test() { Command1 Command2 Command3 return value } When function is invoked it executes all the commands enclosed by the curly braces. Use only function name (function_test ) to invoke it.

104

Shell Programming

Shell functions can be defined at a number of places: At the beginning of every script In a separate file, so that other scripts can also use them In the .bash_profile, so that they are available in the current session If we store the functions in a file called function_library then you must include this file in your script using dot command as follows: $ . function_library
105

Shell Programming

The positional parameters made available to shell scripts externally are not available directly to a shell function. We have to store these parameters in the shell variables and then pass them to the function. The parameters are passed on the function call statement itself. These parameters are accessed in the function using system variables $1, $2, etc.

106

Advanced Filter - awk

AWK Programming. Aho, Weinberger, Kernighan A typical and complete awk command specifies address and an action. $ awk /unix/ { print } filename the address section /unix/ selects lines that are processed in the action section { print } the print statement, when used without any field specifiers, prints entire line.

107

Advanced Filter - awk

Splitting a line into fields : awk uses the special variable $0 to indicate the entire line. It also identifies fields by $1, $2, $3, etc. awk uses -F option for indicating the field separator. For ex. password file contains colon (:) separated fields. So if you have to use awk you can use as $ awk -F: /itp/ { print $1, $3, $2 } passwd

108

Advanced Filter - awk

The logical and relational operators. $ awk -F: $1 == itp3 || $1 == itp5 { printf %-20s %-12s , $2, $5} passwd The awk can also handle numbers, both integer and floating type, and all the relational tests can be handled by awk. (ex. <,<=,==, etc) awk has certain built-in variables, like NR- cumulative record no and $0 (whole record) $ awk NR == 3, NR ==5 { print $3, $4 } tex

109

Advanced Filter - awk

Storing awk Programs in a File

Large awk programs should be stored in files. The extension should be preferable .awk to distinguish it from other files.No quotes are used to enclose the awk program. $ vi test.awk NR == 3, NR ==5 {print $3, $4} you can use awk with -f to get result as $ awk -f test.awk text

110

Advanced Filter - awk

The BEGIN and END sections

these are optional, and take the form BEGIN { action } END { action } you can use them to print a suitable heading at the beginning and the average salary at the end. Always start the opening brace in the same line the section (BEGIN or END) begins.

111

Advanced Filter - awk

Built-In variables:
NR cumulative no of records read FS input field separator OFS output field separator NF number of fields in current record FILENAME current input file

112

Advanced Filter - awk

FUNCTIONS : awk has several built-in functions for arithmetic and string operation int(x) returns integer value of x sqrt(x) returns sqare root of x length returns length of complete record length(x) returns length of x substr(s1,s2,s3) returns portion of string of length s3, starting from position s2 in str s1

113

Advanced Filter - awk

CONTROL FLOW : awk has practically all the features of a modern programming language. If statement : if($5 >20000) interest = 0.10 * $5 else interest = 0.05 * $5

114

Advanced Filter - awk

Looping with for for( i=1; i<10; i++) Looping with while while ( i < 10) print $2 * i

115

Thanks! Contact - Sharda Centre - ext. 5160

116