Hissar: by Imran

UNIX
by Imran
HISSAR
INTRODUCTION
HISSAR
Objectives
In this session, you learn about:
The OS The history of Unix The features of UNIX The Unix architecture Process management CPU scheduling Memory management Unix Connectivity
3
HISSAR
Operating System (OS)
OS is a system software OS can be defined as an organized collection of software consisting of procedures for operating a computer
OS provides an environment for execution of programs
OS acts as an interface between the user and the hardware of the computer system.
HISSAR
Operating System
Operating system interacts with user in two ways
Operating system commands Enables user to interact directly with the operating system.
Operating system calls Provides an interface to a running program and the operating system. System calls in UNIX are written in C.
HISSAR
History of UNIX
Ken Thompson of AT&T Bell Laboratories designed UNIX in late 1960s Two versions of UNIX that emerged are AT&T Unix and BSD Unix In 1989, AT&T and Sun Microsystems joined together and developed system V release 4 (SVR4) Two of the main standards mainly in use are POSIX (Portable Operating System Interface) and X/open standard. In 1988, MIT formed Xconsortium developed vendor-neutral Xwindow System.
HISSAR
What is UNIX?
UNICS?
Unix is different. Why?
Power of Unix?
HISSAR
What is Linux?
An open-source UNIX like operating system
Initially created by Linus Torvalds for PC architecture
Developer community world-wide contribute to its enhancement and growth
HISSAR
Features of UNIX
Multi-user, multitasking, timesharing Portability Modularity File structure Security Strong networking support & advanced graphics
HISSAR
Layered Architecture
... cp comp shell as ld vi ed
banner ls
kernel hardware
sort
sh who a.out date grep wc
HISSAR
10
UNIX System Architecture
Unix system follows a layered approach.
It has four layers

The innermost layer is the hardware layer In the second layer, the kernel is placed The utilities and other application programs form the third layer Fourth layer is the one with which the user actually interacts.
HISSAR
11
Kernel
Kernel is that part of the OS which directly makes interface with the hardware system.
Actions:

Provides mechanism for creating and deleting processes Provides processor scheduling, memory, and I/O management Provides inter-process communication.
HISSAR
12
The Shell
A utility program that comes with the UNIX system.
Features of Shell are:

Interactive Processing Background Processing I/O Redirection Pipes Shell Scripts Shell Variables Programming Constructs
13
HISSAR
Process Management
A process is a program in execution
Several processes can be executed simultaneously in a UNIX system.
A process is generally created using the fork( ) system call.
The process that invokes the fork( ) system call is the parent process, and the newly created process is called the child process.
HISSAR
14
CPU Scheduling
Unix uses round-robin scheduling to support its multi-user and time-sharing feature.
Round-robin fashion of scheduling is considered to be the oldest, simplest and widely used algorithm.
Every process is given a time slice (10-100 millisec.)
HISSAR
15
Memory Management
Virtual memory
Swap area Demand paging
HISSAR
16
UNIX file system
UNIX understands everything as a file. It may be disk, terminal, directory or file.
A file is a stream of bytes.
A file will have properties like block special, character special devices etc.
A file system begins with a directory called root

17
HISSAR
UNIX File Management
UNIX uses a hierarchical file system with / as its root.
Every non-leaf node of the tree is called as a directory file.
Every leaf node can either be a file, or an empty directory
HISSAR
18
Unix File System
dev
bin sh
tmp home
etc inittab
var spool
lib bin
usr src
console lp0
ls user1
passwd user2
HISSAR
19
Unix File System
File system is the structure in which files are stored on disk
File in UNIX is sequence of bytes organized in the form of blocks
The size of each block is 512 bytes (depends on architecture)
Block size can be decided while creating the file system structure
HISSAR
20
File System Structure

Boot Block Super Block Inode Block Data Block
Type of the file Link counter Uid, gid, size Date and time of Creation Date and time of access Date and time of modification : :
le ing ct S i re ind
Double indirect
ple Tri irect Ind
Address of datablock Address of datablock : :
Address of the addr block Address of the addr block Address of the addr block
HISSAR
21
UNIX file system - salient features
Hierarchical file structure
Files can grow dynamically
Files will have access permissions
All devices are implemented as files
HISSAR
22
Unix salient features
cont..
A multi-user networked operating system
Operating System Handles files, running other programs, input/output Just like DOS or Windows
Networked Designed for server use Networking is an intrinsic part of the system
Multi-user Every user has different settings and permissions Multiple users can be logged in simultaneous
23
HISSAR
Unix connectivity
Accessing the servers:
Terminal Programs:
Ssh telnet
File Transfer Programs
ftp
24
HISSAR
Types of UNIX Users
Broad classification of users

root (most privileged) Non-root (less privileged)
Group

UNIX allows user IDs to be grouped A single user ID can be member of multiple groups
Differentiating users with respect to file access

Owner Group Others

25
HISSAR
Working With UNIX
User logs in with a valid user ID User logs out to terminate the login session
HISSAR
26
Summary
In this session, you learned about
The functions of OS The History of Unix The features of Unix The Unix Architecture Process management CPU Scheduling Memory management Unix Connectivity
27
HISSAR
UNIX Commands
HISSAR
28
Objectives
In this session, you will learn to:
Command Structure
Use the basic Unix commands
pwd date who ls Man
Use man pages(Online documentation)

29
HISSAR
Command Structure
Shell presents the user with (usually) a text based interface
It accepts a command line from the user
General syntax
Command [options] [arguments]
Parse the command line and execute it with the given parameters as a child process
HISSAR
30
Types of Commands
Internal commands
Built in Commands in Shell
Ex: cd, alias Doesnt require separate process to execute
External Commands
Find exe file in Path
Priority

HISSAR
Built-in External
31
Flow of command execution

1.
the shell first checks to see if it is a built-in command

1.
if so, executes it
2.
If the command name is an absolute pathname,

1.
Eg:
1.
/bin/ls there is not a problem
3.
If the command is neither built-in, nor specified with an absolute pathname, the shell looks in its search path for an executable program or script with the given name.
HISSAR
32
Absolute and Relative Path
Absolute Path
The Pathname start with /
Eg:
cd /root/home/SLS
Relative path
The pathname which start with other than /
Eg:
cd SLS/imran
HISSAR
33
Using . and ..
Current directory
Eg:
Cat > ./SLS/f1.txt
..
Parent Directory
Eg:
Cat > ../mphasis/f1.txt
HISSAR
34
Simple Commands
pwd
Displays the current working directory.
date
Displays the current date and time
HISSAR
35
Simple Commands
who
Displays the names of all the users who have currently logged in
who am i
Displays the name of the current user.
HISSAR
36
Listing the Directory Contents
ls
Syntax :ls [options] [file.] options: -l list in long format -a list all files including those
beginning with a dot -i -s -R -F list inode no of file in first column reports disk blocks occupied by file recursively list all sub directories mark type of each file display files in columns
37
HISSAR
-C
Meta Characters
Meta Characters * ? [] Purpose Example $ ls l *.c file* $ ls l file? $ ls l file[abc] $ cat file1; cat file2 $ cat abc | wc $ (echo ==== x.c ====; cat x.c) > out count=èxpr $count + 1` Match with one or more characters or none Match with any single character Match with any single character within the brackets ; Command separator | Pipe two commands () Group commands Useful when the output of the ommand group c has to be redirected `command` Execute the command enclosed within back quotes. Useful when the output of a command into a variable in a shell script string string
assuming count has value this 3, increments the value of count Quote all characters with no substitution echo expr $count + 1 (ex. no special meaning for ) $ displays expr $count + 1 Quote all cha racters with substitution. echo expr $count + 1 The characters $,\ (back slas and back quote displays expr 3 + 1 h) have special meaning. assuming the variable count has value 3
HISSAR
38
Listing the Directory Contents

$ ls l total 6 -rwxr-xr-x drwxr-xr-x awkpro -rw-r--r-File type File access permissions -rw-------
1 2 1 1
user1 user2 user1

User id
projA projD projA

Group id
12373 4096 12831
Dec 15 14:45 Dec 22 14:00 Dec 12 13:59 11:16
a.out
c core cs
user1
projA projC
File size Date & time of 61440 modification Dec 15 in bytes
File name
count user3 -rw-r--r-- Link 1
255
Dec 20 14:29
HISSAR
39
Getting Help on Commands
The Unix manual, usually called man pages, is available on-line to explain the usage of the Unix system and commands.
Syntax:
man [options] command_name Common Options -k keyword -M path -a show list command synopsis line for all keyword matches path to man pages all matching man pages (SVR4)
info command_name - help for commands help -command_name gives command synatx
40
HISSAR
Summary
In this session, you have learned to
use the basic Unix commands like
pwd date who ls man
use man pages
HISSAR
41
Utilities
HISSAR
42
Objectives
In this session, you will learn how to:
use the Unix utilities such as
cat, echo, touch, more, file, wc, cmp, comm, find
employ redirection operators use filters such as
sort, grep, cut, head, tail, tr, and paste
use backup commands
gzip
HISSAR
43
cat
cat command takes the input from the keyboard, and sends the output to the monitor We can redirect the input and output using the redirection operators $ cat > file1 Type the content here press <ctrl d> $ cat file1 Displays the content of the file $cat >> file1 This will append standard input to the content of file1
HISSAR
44
touch
touch is used to change the time stamp of the file
Syntax: touch [options] file
Options:

-a to change the access time -m to change the modification time -c no create if not exists
touch <file> will change the time of change of the file if the file exists
If the file does not exist, it will create a file of zero byte size.
45
HISSAR
echo & read
echo command is used to print output to the screen echo This is an example This is an example x=10 echo $x 10
read command allows to read input from user and assign it to the variable specified. read x
HISSAR
46
General Purpose Utilities
more
Allows user to view one page-full of information at a time.
file
Used to display the type of the file
tty
Prints the terminals name
HISSAR
47
Wc
A filter used to count the number of lines, words, and characters in a disk file or from the standard input. -l - displays the number of lines
-w - displays the number of words -c - displays the number of characters
HISSAR
48
cmp
Returns the offset and the line number of the first position where the two files differ.
comm

col1 - unique lines of first file col2 - unique lines of second file col3 - common lines
HISSAR
49
find
Lets user to search set of files and directories based on various criteria Syntax: find [path...] [expression] [path]
where to search
[expression]

What type of file to search (specified with type option) What action to be applied (exec, print, etc.) Name of the files (specified as part of name option, enclosed in )
Example find . name *.c -print lists all files with .c extension from the current dir & its subdirectories
HISSAR
50
find
Finding files on the basis of file size
size [+ ]n[bc] n represents size in bytes (c) or blocks (b) of 512 bytes find . size 1000c find . size +1000c find . size 1000c lists all files that are exactly 1000 bytes in size lists all files that are more than 1000 bytes in size lists all files that are less than 1000 bytes in size
HISSAR
51
find
Finding files on the basis of access time (atime) or modified time (mtime)

atime [+-]n mtime [+-]n n represents number of days ( actually 24 * n hours) find . atime 2 find . atime +2 find / mtime 2 lists files accessed exactly 2 days ago lists files accessed more than 2 days ago lists files modified less than 2 days ago
HISSAR
52
find
Applying a command on files matching the criteria with exec and ok options
exec command {} \; command is command to be applied on the matching files (does not prompt user) find . -name *.dat exec ls l {} \; Long listing of all files with .dat extension in the current and its subdirectories
-ok command {} \; Functionality is similar to exec, but prompts user before applying the command on the file matching the criteria.
HISSAR
53
pr
pr

Used to display a file in a format to be printed. Breaks up a file into pages with a header, text and footer area
Options

-l -h -t -n
to alter the length of the file to set the header to suppress the header and the footer to set the line number
HISSAR
54
Standard Files
Standard Input file
Keyboard, file descriptor is 0
Standard Output file
Monitor, file descriptor is 1
Standard Error file
Monitor, file descriptor is 2
HISSAR
55
I/O Redirection
< file > file 2> file 2>&1 $ cat > abc redirect standard input from file redirect standard output to file redirect standard error to file merge standard error with standard output
$ ls l > outfile $ cat xyz abc > outfile 2> errfile $ cat xyz abc > outfile 2>&1
HISSAR
56
Filters
Filters are programs that takes its input from the standard input file, process it, and sends it to the standard output file.
Commonly used filter commands

sort grep cut head tail paste
HISSAR
57
sort
Sorts the contents of the given file based on the first char of each line. -n numeric sort (comparison made according to strings numeric value) -r -o -u +num[-num] example: reverse sort Result stored in specified file To have only unique lines To merge input with the file for sorting To sort according to the field number sort file1
58
HISSAR
grep
grep -Global Regular Expression Printer is used for searching regular expressions
Syntax
grep <options> <pattern> <filename(s)> Options
-v
-c -n -i
Displays only those lines which does not match with the pattern specifies
Displays only the count no of lines which match the pattern specified Displays matching lines with the line numbers at start Matches the pattern specified ignoring the case
Example:
grep v EDS file1
HISSAR
59
Patterns
* - matches 0 or more characters
[^pqr] - Matches a single character which is not p ,q or r
^pqr -Matches pqr at the beginning of the line
pqr$ -Matches pqr at the end of the line
. - Matches any one character
\ - ignores the special meaning. grep New\[abc\] filename
HISSAR
60
Filter Command - head

Displays the first n lines of the file
$ head -3 file1
HISSAR
61
Filter Command - tail
Displays the last n lines of a file
$ tail -3 file1
Can also specify the line number from which the data has to be displayed till the end of file
$ tail +5 file1 HISSAR

62
Filter command - tr
tr - translate filter used to translate a given set of characters
Example : tr [a-z] [A-Z] < filename
This converts standard input read from lower case to upper case.
option -s can be used to squeeze the repeated characters. HISSAR
63
Filter command - tr
Useful options for tr
-s char Squeeze multiple contiguous occurrences of the character into single char
-d char Remove the character
HISSAR
64
Command Piping
Allows the output (only the standard output) of a command to be sent as input to another command. Multiple pipes may appear in one command line.
Example: $ cat * | wc $ cat fil1 | head | wc -l HISSAR
65
Filter Command tee
tee command allows the normal output to the standard output, as well as to a file Useful to capture intermediate output of a long command pipeline for further processing, or debugging purpose. Example

who | tee userlist cat - | tee file1 | wc -l
HISSAR
66
Filter Command cut

Used to extract specified columns of a text
Option remark -c -d -f Examples $ cut -c2-5 file1 $ cut -d | -f2,3 file1 used to extract characters Delimiter for fields Field no.
HISSAR
67
Filter Command paste

Paste is used to fix two cut portions of the file vertically -s -d Pastes the contents of file2 below file1 Specify delimiter
$ paste -d| file1 file2
HISSAR
68
Compression Utilities
gzip, Usage is very similar to compress and pack utilities in Unix: gzip [-vc] filename where -v displays the compression ratio. -c sends the compressed output to standard output and leaves the original file intact.
HISSAR
69
Summary
In this session, you have learned to:
use the Unix Utilities like cat, echo, touch, more, file, wc, cmp, comm, find employ redirection operators use filters like sort, grep, cut, head, tail, tr, and paste backup commands gzip
70
HISSAR
Chapter 3
Files & Directories
HISSAR
71
Objectives

set file permissions using the chmod command use directory-related commands namely mkdir, rmdir, cd commands use file-related commands namely cp, mv, rm commands access advanced file permissions using commands umask create and edit files using the vi editor
HISSAR
72
File Access Permissions
Refers to the permissions associated with a file with respect to the following
Permission Levels

User (owner) (u) Group (wheel, staff, daemon, etc.) (g) World (guest, anonymous and all other users) (o)
Permission Settings

Read (r) Write (w)

73
HISSAR Execute (x)
No read permission does not allow the user to:

List the contents of directory Remove the directory
No Write permission does not allow the user to :

copy files to the directory remove files from the directory rename files in the directory make a subdirectory remove a subdirectory from the directory
move files to, and from the directory HISSAR

74
No execute permission does not allow the user to:
display the contents of a directory file from within the directory
change to the directory
display a file in the directory
copy a file to, or from the directory

75
HISSAR
Changing Permissions - chmod
chmod u+x file_name chmod <category> <operation> <permission> <filename(s)>
Syntax: or chmod <octal number> filename Octal Number 4 - for read 2 - for write 1 - for execution $ chmod 744 xyz HISSAR this sets read, write and execute permissions for owner, read permission for group and others
76
Directory Creation
Command Syntax mkdir [OPTION] DIRECTORY $ mkdir <path>/<directory> $ mkdir m <directory> $ mkdir p <directory1>/<directory2>/<directory3> Example: $ mkdir project1 This creates a directory project1 under current directory Note: Write and execute permissions are needed for the directory in which user wants to create a directory HISSAR
77
Directory Removal
rmdir command removes directory Syntax
rmdir <directory name>
Example Removes project1 directory in the current directory
rmdir project1
Remove multiple directories rmdir pos1 pos2 Remove the directory recursively rmdir p dir1/dir2/dir3
HISSAR removes a directory if it is empty and is not the current directory rmdir
78
Command - cd
cd command is used to change the directory
cd cd .. cd /
- take to the home directory - takes to the parent directory - takes to the root directory
HISSAR
79
File-Related Commands
File Operation Copying a file Moving a file Removing a file Displaying a file and concatenating files Command cp mv rm cat
HISSAR
80
Command - cp
Used to copy files across directories Syntax cp <source file> <new file name>
Example cp file1 file2
HISSAR
81
Command - mv
Used to move a file, or rename a file Preserves the following details
owner id group id permissions Last modification time

82
HISSAR
Command - rm
Used to remove a file
Syntax : rm file(s) suppresses all prompting prompts before deleting destination file
-f -i -r
HISSAR
will recursively remove the file from a directory (can be used to delete a directory along with the content )
83
Command chown & chgrp

$ ls l -rwxr-xr-x 1 user1 training 12373 Dec 15 14:45 a.out -rwxr-xr-x 3 user1 faculty 4096 Dec 24 11:56 awkpro $chown user2 a.out $ls l -rwxr-xr-x 1 user2 training 12373 Dec 15 14:45 a.out -rwxr-xr-x 3 user1 faculty 4096 Dec 24 HISSAR 11:56 awkpro
84
Command - umask
umask value is used to set the default permission of a file and directory while creating
umask command is used to see the default mask for the file permission
Default umask value will be set in the system environment file like /etc/profile
umask 022 will set a mask of 022 for the current session
The file permission after setting this umask value will be 644 And the directory permission will be 755
85
HISSAR
Command Linking files
- ln
Hard Link (in the same filesystem)
$ ln /usr/bin/clear /usr/bin/cls Hard link uses the same inode number
Soft Link (in different filesystems also used to link directories) $ ln s /usr/bin/clear /home/user1/cls
86
HISSAR
Vi Editor
vi is a visual editor used to create and edit text files.

A screen-oriented text editor Included with most UNIX system distributions Command driven
Categories of commands include

Cursor movement Editing commands Search and replace commands vi editor is invoked by the following command:
87
The HISSAR
Navigation
Backspace h j k Space l
the quick w the quick 2 w the quick b
brown fox w w
the quick
brown fox $
brown fox
the quick
brown fox ^
brown fox b b
HISSAR
88
Editing Commands
Text insertion / replacement i a I A o O R s S - inserts text to the left of the cursor - inserts text to the right of the cursor - inserts text at the beginning of the line - appends text at end of the line - opens line below - opens line above - replaces text from cursor to right - replaces a single character with any number of characters
- replaces entire line

89
HISSAR
Editing Commands
Deletion
x 3x dw 2dw dd 2dd
- to delete character at cursor position - to delete 3 characters at cursor position - to delete word - to delete 2 word - to delete a line - to delete 2 lines
HISSAR
90
Editing Commands
Yanking

Y 3Y p P
- copy line into buffer - copy 3 lines into buffer - copy buffer below cursor - copy buffer above cursor
Save and quit

:w :w! :x :q
- to save - to name a file (:w! filename -> save as) - save and quit - cancel changes - cancel and quit
91
HISSAR :q!
Search & Replace Commands

The following commands are applicable for vi editor in Linux /pat searches for the pattern pat and places cursor where pattern occurs. / :%s/old/new repeat last search To change every occurrence in the whole file.
HISSAR
92
Summary
In this session, you have learned how to use file permissions using the chmod command use directory-related commands namely mkdir, rmdir, cd commands use file-related commands namely cp, mv, rm commands access advanced file permissions using commands umask create and edit files using the vi editor
HISSAR
93
Process
HISSAR
94
Objectives
Use process-related commands like
ps, kill, sleep
Start a background process Use background and foreground-related commands like
bg, fg, jobs , nohup
HISSAR
95
Processes
Process - a program in execution When program is executed, a new process is created The process is alive till the execution of the program is complete Each process is identified by a number called pid
96
HISSAR
Login shell
As soon as the user logs in, a process is created which executes the login shell.
Login shell is set for each login in /etc/passwd file.
HISSAR
97
ps
The ps command is used to display the characteristics of a process
It fetches the pid, tty, time and the command which has started the process.

-f -u -a -e
lists the pid of the parent process also. lists the processes of a given user lists the processes of all the users lists all the processes including the system processes
98
HISSAR
Background Process
Enables the user to do more than one task at a time. If the command terminates with an ampersand (&), UNIX executes the command in the background Shell returns by displaying the process ID (PID) and job id of the process
HISSAR
99
Background Process
nohup

Lets processes to continue to run even after logout The output of the command is sent to nohup.out if not redirected
$ nohup command args
$ nohup sort emp.lst & [1] 21356 nohup: appending output to `nohup.out' HISSAR
100
Background Process
wait command
can be used when a process has to wait for the output of a background process The wait command, can be used to let the shell wait for all background processes terminate. $ wait
It is possible to wait for completion of one specific process as well.
HISSAR
101
Controlling Background Processes
jobs
List the background process
fg % <job id>
Runs a process in the foreground
bg %<job id>
Runs a process in the background
HISSAR
102
The kill Command
kill: Kills or terminates a process
kill command send a signal to the process
The default signal is 15 ( SIGTERM)
kill -9 (SIGKILL)
Terminates the process abruptly

103
HISSAR
Summary
In this session, you learned to:
Define a process Use process-related commands like
ps, kill, sleep
Start a background process Use background and foreground-related commands like
bg, fg, jobs
HISSAR
104
Bash Shell Scripting
HISSAR
105
Introduction to Shell
HISSAR
106
Introduction to Shell
content Shell Shell Basic Logging in to the Shell Basics Shell Principles Command execution
HISSAR
107
The Shell
Provides a powerful interface to the UNIX Operating System.
The 'command-line' interface, Comes under different flavours, but all of them do the same thing in slightly different ways.
sh csh bash
Original Unix Bourne Shell BSD Unix C Shell, tcsh Enhanced C Shell Bourne-Again Shell
108
HISSAR
Shell Basics
Shells are command line interpreters.
This is where a user can type in commands to create/del. files.. Etc
Depending on the type of shell you use, the prompt may look different.

tcsh: prevost:~> bash: bash-2.05b$ csh: prevost<31>
To check which shell you are running type:
Echo $SHELL
To change Shell
Eg. If I wanted to use tcsh, just type in tcsh and then enter at the command line
109
HISSAR
Logging in to the shell
In order to be able to use the UNIX shell, you will have to authenticate yourself (tell the system who you are).
This process is commonly called the 'login' process, and it involves two steps. 1. Know your username password.
-
2.
Have a means of communicating with the UNIX shell
HISSAR
110
Bash
Bash Bourne Again shell is a sh-compatible command language interpreter.
It executes commands and read from standard input device as well as from file.
HISSAR
111
Basic Shell Principles (1):

There is a basic syntax for all commands executed at the shell: command argument1 argument2 argument3... command is the name of the actual shell command you wish to execute. Every command may take a certain number of arguments (or operands). For example: cd /mn/proteas/data cd is the actual command and it takes one argument /mn/proteas/data. Always make sure that you have a space between a shell argument(s). command and its
HISSAR
112
Basic Shell Principles (2):
All UNIX shells are case sensitive with regards to both the commands and their arguments, in contrast to versions of Windows/DOS systems. This means that typing: cd /mydirectory/programs is not the same as typing: or even: CD /MYDIRECTORY/PROGRAMS cd /MyDirectory/Programs
Usually, shell commands are lower case, unless otherwise stated.
HISSAR
113
Command processing
Displays the shell prompt and reads the command typed by the user.
Interprets the command and classifies it as an internal (built-in), or an external command.
If it is NOT a built-in command, searches for the command in the PATH-specified directories, and executes that command if it is found.
HISSAR
114
Conclusion
Summary Shell is an interface between a user and the OS The user name and password has to be entered for log in Shell is case sensitive. The command line includes command and arguments Different flavours of shells are Bourne Shell C shell Bash Shell Korn Shell
HISSAR
115
Shell Programming
HISSAR
116
Shell Scripts Basics
HISSAR
117
Shell Scripts Basics

content Bash Shell? Shell Script and Execution Shell Variables Normal variables Environmental variables Built in Variables Interactive Shell Programming Read Echo
HISSAR
118
Bash
Bash (Bourne Again shell) is a sh-compatible command language interpreter. It executes commands and read from standard input device as well as from file. Bash is conformant implementation of IEEE POSIX Shell and Tools specification
General Syntax: bash [options] [Bash_Script]
HISSAR
119
Sample Shell Script

1.
#! /bin/bash # The above line has a special meaning. It must be the # first line of the script. It says that the commands in # this shell script should be executed by the bash # shell (/bin/bash). # ------------------------------------------------------------echo Hello $USER. echo Welcome to programming shell scripts.. # ------------------------------------------------------------120
2.
3.
4.
5.
6.
7.
8.
9.
HISSAR
Sample Shell Script

1.
#!/bin/bash
2.
echo "hello, $USER. I wish to list some files of yours" echo "listing files in the current directory, $PWD" ls # list files
3.
4.
HISSAR
121
Executing Shell Scripts

There are two ways of executing a shell script:
By passing the shell script name as an argument to the shell. For example:
sh script1.sh If the shell script is assigned execute permission, it can be executed using its name. For example:
./script1.sh
HISSAR
122
Shell Programming
Allows

Defining and referencing variables Logic control structures such as if, for, while, case Input and output
HISSAR
123
Shell Variables
A variable is a name associated with a data value, and it offers a symbolic way to represent and manipulate data variables in the shell.

They are an integral part of shell programming Provide ability to store and manipulate data Fully under the control of the shell Can create and destroy the variables as the user wants
HISSAR
124
Shell Variables - Guidelines

Any combination of alphabet, numbers and underscore( _ ) No editing characters like commas, blanks The first character of the name should be alpha or underscore May be of any reasonable length Case sensitive ie abc & Abc are different
HISSAR
125
Shell - Assigning values

Values can be assigned to variable using = equal sign
name=Vijay dirname=/usr2/bin $echo $name displays Vijay
NOTE: values Vijay & /usr2/bin is assigned to variables without giving space. Instead results in error
HISSAR
126
Shell Variables
They are classified as follows

user-defined variables environment variables predefined variables
value assigned to the variable can then be referred to by preceding the variable name with a $ sign.
HISSAR
127
Shell Variables
The shell provides the facility to define normal, and environment variables.
A normal variable can be only used in the shell where it is defined.
An environment variable can be used in the shell where it is defined, plus any child shells invoked from that shell.
HISSAR
128
Using Normal Variables

To define a normal variable, use the following syntax: variable name=value
Examples: x=10 textline_1=This line was entered by $USER textline_2=This line was entered by $USER allusers=`who` usercount=`who | wc l`
HISSAR
129
Using Normal Variables

Once variables are defined, one can use the echo command to display the value of each variable:
echo $x echo $textline_1 echo $textline_2 echo $allusers echo $usercount
HISSAR
130
Using Environment Variables
To define an environment variable, use following syntax: variable_name=value export variable_name
Examples:
$ x=10; export x $ allusers=`who` ; export allusers
HISSAR
131
Working Example
parent.sh #!/usr/bin/bash environ=SYSTEM LOGIC SOLUTIONS; local=Helios matheson; echo From parent: $environ; echo From child: $local; sh child.sh
child.sh #!/usr/bin/bash echo ENVIRONMENT VARIABLE: $environ; echo HISSAR NOT AN ENVIRONMENT VARIABLE: $local;
132
Built-in variables
Following are built-in variables supported

$0, $1$9 $* $@ $? $$ $!
- positional arguments - all arguments - all arguments - exit status of previous command executed - PID of the current process - PID of the last background process
HISSAR
133
Built-in environment variables
PATH HOME PWD SHELL TERM
MAIL USER LOGNAME PS1 PS2
HISSAR
134
Interactive Shell Programming

Two basic words in shell are read & echo read accepts input echo writes the output echo Enter your name \? read name echo Good morning $name Write the above 3 lines in a file name sh01 HISSAR
135

$sh01 Enter your name ? Vijay Good morning Vijay
The \ symbols makes the shell to understand ? as an ordinary character rather than special character Double quotes () may be used to display a string
HISSAR
136

name is a shell variable receives data from keyboard $name will display the contents of the variable
HISSAR
137

echo Enter three values X Y Z read x y z echo $x $y $z
If 1 2 3 entered, x will have 1, 2 to y & 3 to z If 1 2 3 4 are entered, then z will have 3 4 If only 1, 2 are entered, then z will be assigned null value If more arguments entered, last variable will be assigned rest. If less, null will be assigned
138
HISSAR

echo This is new line \nAnd this is second line
The above will be displayed as This is new line
And this is second line
The escape sequence was brought from C language
HISSAR
139

\b is blank, \t is tab char etc.
To make the cursor at the end of the echoed line echo Enter your Name :\c Enter your Name :_ (Cursor waits here) echo \007 gives bell
HISSAR
140
Tips & Traps
All the variables are string variables $a=20. 20 is stored as a string not as number. No arithmetic can be carried on this Use double quotes if the value contains more than one word
$name=Vijay Suri
Can assign more than one variable in a line
$name=Vijay age=20
HISSAR
141
Tips & Traps
Can display more than one variable in a line All variables in a shell script are automatic variable. i.e. they will be created as soon as the shell script execution starts and dies as soon as the execution is over
$echo Name is $name and age is $age
HISSAR
142
Tips & Traps
A null variable can be created by Shell ignore if any shell variable is having null value
$a1= or a1= or a1=
HISSAR
143
Single Quotes versus double quotes
Basically, variable names are expanded within double quotes, but not single quotes. If you do not need to refer to variables, single quotes are good to use as the results are more predictable.
An example
#!/bin/bash echo -n '$USER=' # -n option stops echo from breaking the line echo "$USER" echo "\$USER=$USER" # this does the same thing as the first two lines
The output looks like this (assuming your username is elflord)
$USER=elflord $USER=elflord
144
HISSAR
Unchanging Variable - readonly
User can make a variable unchanged during execution by:
$age=20 $readonly age
The shell does not allow us to change the value
HISSAR
145
Wiping out Variable - unset
A variable can be removed from the shell by using unset command Unset can not be used for system variables $unset PS1 is not allowed
$unset age

HISSAR
146
Positional Parameters
Many occasions, a program expects the variables in a certain fashion. This is achieved through positional parameters from $0 through $9. $0 is the program itself. Thus
$abc par1 par2 par3 par4 assigns abc to $0, par1 to $1 par4 to $4
HISSAR
147
Passing command line arguments
To write a shell by name sh02 which copies one file to another
echo Copying $1 to $2 cp $1 $2
By executing sh02
$sh02 file1 file2 Copying file1 to file2. Positional parameters can not assigned values like $1=100 HISSAR
148
Shell Variables
To know the number of variables given for the shell through $# $abc file1 file The above shell displays 3
echo $#
HISSAR
149
Shell Variables - shift
Shell can handle only 9 variables at a time. To access more than 9, the shift command is used $set You have the capacity to learn the shell programming in a very easy way $echo $10 displays You0 as shell interprets $10 as $1 with 0. $shift 5 : makes the 5th argument as $1 argument. Thus $echo $1 displays learn $* handles all positional parameters
150
HISSAR
Arithmetic in Shell
Write the following in a file and execute: a=10 b=4
echo expr $a + $b echo expr $a - $b echo expr $a \* $b echo expr $a / $b echo expr $a % $b
HISSAR
151
#modulus
Arithmetic in Shell
On execution, the output is:
14 6 40 2 2
HISSAR
152
Arithmetic in Shell
Anything after # sign will be treated as comment expr is the key word for doing arithmetic A multiplication symbol (*) should be preceded by \ Terms in expr should be separated by space Parenthesis may be used for clarity of expression
HISSAR
153
Arithmetic in Shell
Should use pair of parenthesis
expr can handle only integers. Use bc to handle real numbers. A=1.5 b=2.5 echo expr $a + $b | bc
HISSAR
154
UNIX decision loops
There are 4 decision making loops if then fi if then else fi if then elif else fi case - esac
HISSAR
155
UNIX - if then fi
Key word if and the delimiter is fi if <command>
then statements fi e.g.: if cp $1 $2 then echo Copied successfully fi

HISSAR
156
UNIX - if then else fi

The structure of this construct is: if <Condition>
then statements else statements fi

HISSAR
157
UNIX - if then else fi

e.g.: if cp $1 $2 then echo File copied successfully else echo File copy failed fi
HISSAR
158
UNIX - if then elif else fi

Used for multilevel decision making. if <condition> then
statements elif <condition> statements else statements fi

HISSAR
159
UNIX - test
if depends upon the exit status of the command given test verbs translates the result into success or failure There are three tests namely Numerical test String test File test
160
HISSAR
UNIX - test - numerical
Used to compare numerical
-gt = greater than -lt = less than -ge = greater than or equal to -le = less than or equal to -ne = not equal -eq = equal
HISSAR
161
UNIX - test - numerical

if [ $1 -lt 5 ] then echo the value is < 5 elif [ $1 -le 7 ] echo the value is <= or equal 7 else echo the value is > 7 fi
HISSAR
162
Using the test Command

To compare two strings using the test command, following operators are available:
string1 = string2 (equal to, please note it is a single =) string1 != string2 (not equal to) string1 (string is not NULL) -n string1 (string is not NULL and exists)
163
HISSAR

To check a file type/access permissions using the test command, following operators are available:
HISSAR
-s file (file is not empty and exists) -f file (Ordinary file and exists) -d file (file is a directory and exists) -r file (file is readable and exists) -w file (file is write-able and exists) -x file (file is executable and exists)
164

To check a file type/access permissions using the test command, following operators are available:
-b file (file is a block device and exists) -c file (file is a character device and exists) -p file (file is a named pipe and exists)
HISSAR
165
UNIX - test
Use square braces to avoid writing test Provide a space after [ Provide a space before ]
HISSAR
166
UNIX - test - file
The following are the file related flags -s returns True if the file exists and size > 0 -f returns True if the file exists and not directory -d return True if the file exists and is a directory
e.g.: if [ -f $1 ] then echo File exists fi

HISSAR
167
UNIX - test - String
s1 = s2 returns true if both are same -n returns true if string length > 0
if [ $1 = $2 ] then echo Both strings are same else echo $? fi
$? Contains the value of the last command

168
HISSAR
UNIX logical conditions
-a stands for AND condition -o stands for OR condition -! Is negation if <Condition-1> -a <Condition-2> Returns true if both the conditions are true
statements...
if [ $1 -gt 60 ] -a [ $2 -lt 50 ] then
HISSAR
169
UNIX case
To handle multiple choices case value in choice 1) statements;; choice 2) statements;; *) statements;; esac
HISSAR
170
UNIX case
esac is the delimiter of case Used for menus *) is the default choice All choice statements should be terminated by double semicolon(;;)
HISSAR
171
UNIX case example

case $option in 1) echo Financial accounting;; 2) echo Materiel accounting;; *) echo Invalid Opt - Try;; esac
HISSAR
172
UNIX case
The choices many be in any order case statement may be a shell variable or shell argument or output of a command Need not be numbers - may be strings too
e.g.: banana) statements;; orange) statements;;

HISSAR
173
UNIX case
Multiple options can be grouped
e.g.:
case $1 in banana | orange) echo Fruit;; dog | pig) echo Animal;; lion) echo Wild animal;; esac
HISSAR
174
UNIX case
Can use shells pattern matching
e.g.: case $1 in [a-zA-Z) echo Characters; [0-9]) echo NUMBER;; esac
HISSAR
175
UNIX Loop Controls
Provided 3 loop constructs namely: while loop for loop until loop
HISSAR
176
UNIX - while loop

while <condition> do statements done
done is the delimiter of do
HISSAR
177
UNIX - while example

count = 1 while [ $count -le 3 ] do echo Loop value $count count = èxpr $count + 1` done
HISSAR
178
UNIX - while
The loop will continue so long as the condition is TRUE When the condition is false, the next command after done will be executed The condition can be any valid UNIX command The while condition can be simple condition or complex condition The condition should have an exit status. Otherwise, it may go into infinite loop
179
HISSAR
UNIX - until loop

until <condition> do statements done
until continues its loop so long as the condition is false except this, while & until are identical
HISSAR
180
UNIX - for loop

Most frequently used loop for control-var in value1 value2
do statements done
for takes a list of variables
HISSAR
181
UNIX - for example

for word in $* do echo $word done
HISSAR
182
UNIX - break statement
Used to break the current loop and comes out of the loop
Usually associated with if if [ $1 -eq 5 ]
then I = 2 break fi
HISSAR
183
UNIX - continue statement

To take the control to the beginning of the loop bypassing the statements I=1
while [ $I -le 5 ] then do I = èxpr $I + 1` continue done

HISSAR
184
Functions
Shell functions are a way to group commands for later execution using a single name for the group. They are executed just like a "regular" command. Shell functions are executed in the current shell context; no new process is created to interpret them. Functions are declared using this syntax: [ function ] name () { command-list; }
185
HISSAR
Functions
Shell functions can accept arguments Arguments are passed in the same way as given to commands Functions refer to arguments using $1, $2 etc., similar to the way shell scripts refer to command line arguments
HISSAR
186
Functions
Function to convert standard input into upper case toupper() { tr [a-z] [A-Z] } This function can be used as
HISSAR
$ cat abc | toupper

187
Debugging Shell Scripts
Two options help in debugging shell scripts
-v (verbose) option:
causes shell to print the lines of the script as they are read. $ bash v script-file
-x (verbose) option:
prints commands and their arguments as they are executed. $ bash x script-file
HISSAR
188
Example
Shell script checks for a blank/non blank string
read nam while [ -z $nam ] do read nam done echo the string is $nam
the above piece of code keeps accepting string variable nam until it is non zero in length.
HISSAR
189
Exercise
Write a shell script to accept the user and password Note: It should keep on asking for the username and password until the user gives the correct one.
HISSAR
190
Example
----------------------script.sh-------------------------#! /usr/bin/bash usernames=`who | cut d f1` echo Total users logged in = ${#usernames[@]} # for user in ${usernames[*]} do echo $user done -----------------------------------------------------------HISSAR
191
Example
# to check if the current directory is the same as your home directory
curdir=`pwd` if test $curdir != $HOME then echo your home dir is not the same as your pesent working directory else echo $HOME is your current directory HISSAR fi
192
AWK
By Imran Hissar Ahmed
HISSAR
193
Objectives
Awk introduction Knowing which awk Command line of awk and options Patterns and Procedures
HISSAR
194
Introduction
awk is a pattern-matching program for processing files, especially when they are databases.
The new version of awk, called nawk, Every modern Unix system comes with a version of new awk, and its use is recommended over old awk.
Different systems vary in what the two versions are called.
Some have oawk and awk, for the old and new versions, respectively. Others have awk and nawk. Still others only have awk, which is the new version.
195
HISSAR
Which awk?
This example shows what happens if your awk is the old one: $ awk 1 /dev/null awk: syntax error near line 1 awk: bailing out near line 1 awk exits silently if it is the new version.
HISSAR
196
Command-Line Syntax
The syntax for invoking awk

awk [options] 'script' var=value file(s) awk [options] -f scriptfile var=value file(s)
has two forms:

You can specify a script directly on the command line, or you can store a script in a scriptfile and specify it with -f.
HISSAR
197
Recognized options
-Ffs Set the field separator to fs. Fields are referred to by the variables $1, $2,..., $n. $0 refers to the entire record. -v var=value Assign a value to variable var. -v var=12 F {print $1 var} -v v1=LT f a1.awk Example: awk F { print $1; print $2; print $3 } filename cat >awk1 -F { print $1; print $2; }
HISSAR
awk f awk1 filename
198
Patterns and Procedures

awk scripts consist of patterns and procedures:
pattern { procedure } Both are optional.
If pattern is missing, { procedure } is applied to all lines; if { procedure } is missing, the matched line is printed.
HISSAR
199
Patterns
A pattern can be any of the following:
/regular expression/

/and/ /^h/ NF>2 $1~/pattern/ BEGIN{ print START} {print $0} END {print END}
relational expression
pattern-matching expression
BEGIN END
Expressions can be composed of quoted strings, numbers, operators, functions, defined variables, etc..
HISSAR
200
Procedures
Procedures consist of one or more commands, functions, or variable assignments, separated by newlines or semicolons, and contained within curly braces.
Commands fall into five groups:
Variable or array assignments Printing commands Built-in functions Control-flow commands

201
HISSAR
Pattern-Procedures Examples
Print first field of each line:
{ print $1 }
Print all lines that contain pattern:
/pattern/
Print first field of lines that contain pattern:
/pattern/ { print $1 }
Select records containing more than two fields:
HISSAR
NF > 2
202
Interpret input records as a group of lines up to a blank line. Each line is a single field:
BEGIN { FS = "\n"; RS = "" } { print $0 }
Print fields 2 and 3 in switched order, but only on lines whose first field matches the string "URGENT":
$1 ~ /URGENT/ { print $3, $2 }
Count and print the number of pattern found:
HISSAR
/pattern/ { ++x } END { print x }

203
Add numbers in second column and print total:
{ total += $2 } END { print "column total is", total}
Print lines that contain less than 20 characters:
length($0) < 20
HISSAR
204
Print each line that begins with Name: and that contains exactly seven fields:
NF == 7 && /^Name:/
Print the fields of each input record in reverse order, one per line:
{ for (i = NF; i >= 1; i--) print $i }
HISSAR
205
Built-in Variables
Variable
Description
FILENAME Current filename FS NF NR OFS Field separator (a space) Number of fields in current record Number of the current record Output field separator (a space)
HISSAR
206
Built-in Variables
Variable Description
ORS
Output record separator (a newline)
RS
Record separator (a newline)
$0
Entire input record
$n
nth field in current record; fields are separated by FS

207
HISSAR
Operators

Symbol
Meaning
= += -= *= /= %= ^= **= Assignment ?: || && ~ !~ < <= > >= != == (blank)

208
C conditional expression Logical OR Logical AND Match regular expression and negation Relational operators Concatenation
HISSAR
Operators

Symbol +*/% +-! ^ ** ++ -$

209
Meaning Addition, subtraction Multiplication, division, and modulus (remainder) Unary plus and minus, and logical negation Exponentiation Increment and decrement, either prefix or postfix Field reference
HISSAR
User-Defined Functions

function capitalize(input, result, words, n, i, w) { result = "" n = split(input, words, " ") for (i = 1; i <= n; i++) { w = words[i] w = toupper(substr(w, 1, 1)) substr(w, 2) if (i > 1) result = result " " result = result w } return result } # main program, for testing { print capitalize($0) }
HISSAR
210
SED by
Imran Hissar Ahmed
HISSAR
211
INTRODUCTION
HISSAR
212
Objectives
In this session, you learn about:
Introdution to Sed Options with sed Substitution Exaples with shell script
HISSAR
213
Introduction
Sed is Stream Editor

Sed makes only one pass over the input and is consistent
General Syntax
Sed [options] [stream-commands/stream command file] [Input filename]
Eg Sed e 1d filename Sed f f1.sed filename
HISSAR
214
Sed options
-e:
Which is used togive expression in command line
Sed e s/string/replacement_string/ filename
-f:
To give a sed script file instead of a sed statement
Sed f f1.sed filename
-i:
Which is used to store the output file as the extension using the same filename
Eg: Sed e s/5/0000/ i.sed filename
-n:
Which is used to suppress the output from displaying
HISSAR
Sed n e s/string/replacement_string/ filename

215
Sed Command s
s:
Substituting the stream
s/day/night/
Note: Can be used | as a delimitter
s|day|night|
Global replacement with g option
s/day/night/g
Pipimg with The next command
Sed s/day/DAY/ file|sed s/night/NIGHT
The other way
Sed e s/day/DAY/ e s/night/NIGHT/ d1
HISSAR
216
Sed Other Command
Deleting d:
Sed -e 1d file
Deletion of line specified
Specifying a range
Sed 1,4d file
Printing p:
Sed /Lotus/p file
Print selected line
Sed n /lotus/p file
Writing to a file w
Sed s/lotus/LOTUS/ w output_file file
HISSAR
217
Example with shell script
#!/bin/sh echo -n 'what is the value? ' read value sed s/XXX/$value/ <<EOF The value is XXX EOF
HISSAR
218
Summary
In this session, you learned about Stream Editor Options of Sed Substitutions Working sed with Shell The Here Document
HISSAR
219
UNIX Bibliography
UNIX in a Nutshell for BSD 4.3: A Desktop Quick Reference For Berkeley (O'Reilly & Associates, Inc., 1990, ISBN 0-937175-20-X). UNIX in a Nutshell: A Desktop Quick Reference for System V & Solaris 2.0 (O'Reilly & Associates, Inc., 1992, ISBN 0-56592-001-5). The UNIX Programming Environment, Brian W. Kernighan & Rob Pike (Prentice Hall, 1984). HISSAR
220
Lab Exercises
HISSAR
221
Thank you
HISSAR
222

Hissar: by Imran

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Hissar: by Imran

Загружено:

Авторское право:

Доступные форматы

UNIX

Operating System (OS)

OS provides an environment for execution of programs

Operating system interacts with user in two ways

Unix is different. Why?

Initially created by Linus Torvalds for PC architecture

Developer community world-wide contribute to its enhancement and growth

... cp comp shell as ld vi ed

sh who a.out date grep wc

UNIX System Architecture

Unix system follows a layered approach.

It has four layers

A utility program that comes with the UNIX system.

Features of Shell are:

A process is a program in execution

Several processes can be executed simultaneously in a UNIX system.

A process is generally created using the fork( ) system call.

Every process is given a time slice (10-100 millisec.)

Swap area Demand paging

UNIX file system

UNIX understands everything as a file. It may be disk, terminal, directory or file.

A file is a stream of bytes.

A file system begins with a directory called root

UNIX File Management

UNIX uses a hierarchical file system with / as its root.

Every non-leaf node of the tree is called as a directory file.

Every leaf node can either be a file, or an empty directory

Unix File System

Unix File System

File system is the structure in which files are stored on disk

File in UNIX is sequence of bytes organized in the form of blocks

The size of each block is 512 bytes (depends on architecture)

File System Structure

ple Tri irect Ind

Address of datablock Address of datablock : :

UNIX file system - salient features

Hierarchical file structure

Files can grow dynamically

Files will have access permissions

All devices are implemented as files

Unix salient features

A multi-user networked operating system

Accessing the servers:

File Transfer Programs

Types of UNIX Users

Broad classification of users

root (most privileged) Non-root (less privileged)

Differentiating users with respect to file access

Owner Group Others

Working With UNIX

Use the basic Unix commands

pwd date who ls Man

Use man pages(Online documentation)

Shell presents the user with (usually) a text based interface

It accepts a command line from the user

Command [options] [arguments]

Built in Commands in Shell

Ex: cd, alias Doesnt require separate process to execute

Find exe file in Path

Flow of command execution

the shell first checks to see if it is a built-in command

If the command name is an absolute pathname,