Академический Документы
Профессиональный Документы
Культура Документы
1976, first licensed release, Version 6 1978, first portable version, Version 7 1979, Berkeley 3BSD 1983, System V as industry standard 1984, Microsoft releases Xenix 1986, BSD 4.3, AT&T Version 9 1987, SVR4, Mach, 1993, Linux
1
Unix Philosophy
Application Programs
Kernel
Shell
Hardware
3
Kernel
Manages memory and allocates it to each process Schedules work done by the CPU Organizes transfer of data from one part of machine to another Accepts instructions from shell and carries them out Enforces access permission on the file system
Shell
Command interpreter Create customized environments Write shell scripts Define command aliases Manipulate command history File and command completion Edit the command line
File System
Logical method for organizing and storing large amounts of information. Easy to manage. File: basic storage unit. Types:
ordinary file (stores info) directory (holds other files and directories) special file (represents physical devices like printers, terminals, etc) pipe (temporary file for command linkage)
More than one user can run at the same time and more than one task can run at the same time Unix is multiuser multitasking, Window NT is, Windows is not. In Unix, each program is started as a process. A process is a program in execution. Usually only one copy of a program, but there may be many processes running the same program. To each interactive user (window):
only one process in foreground may have several processes in background
7
Processes
kernal mode user mode
kernel
Process 0: Kernel bootstrap. Start process 1. Process 1: create processes to allow login. fork exec /etc/getty condition terminal for login exec /bin/login check password exec shell command interpreter
/etc/init
inetd
lpd
httpd
Init process
Unix Process
last step in booting procedure create other processes to allow the users to login
Getty process
conditions for terminal connection wait for user-id
Login process
check password with the uid execute .profile or .login (depends on default shell) display shell prompt
UNIX Process
Process environment
Process id, parent-process-id, process-group-id Opened files Working directory File creation mask User ID, Group ID Resource limits Environment variables Code
10
Processes
use ps to see the processes that you are running. $ ps PID TTY TIME CMD 221 pts/4 4:15 netscape 201 pts/4 0:05 bash 215 pts/4 1:15 emacs-19
use & to execute a task in background
use ctrl-Z to suspend the foreground task, and then use bg. use fg to move a background task to foreground.
11
12
Command cat cp mv rm lpr lp (Sys V) ls chmod pwd cd mkdir rmdir ps man df du grep
Function Display a file Copies a file Renames a file or moves it Delete files Sends a file to a printer
Day-to-Day Use
Meaning conCATenate CoPy MoVe ReMove Line Printer LiSt Change MODe Print WorkingDir Change Dir MaKe DIR ReMove DIR Process Status Manual Disk File Disk Utilization
13
Lists the contents of a directory Changes the mode of permissions Shows the current directory Change current directory Create a directory Delete a directory Shows the processes on the system Shows info. about commands Shows file system status Shows the space allocation of files Search for patterns in files
Options modify how the command works command [options] [--] [<file> ]
options ::= -option white-space options* option ::= noargoption* argoption | noargoption+ noargoption ::= letter argoption ::= letter whitespace string $cc -po zap zap.c
14
Some Examples
ls [-alRF] file-list
a for listing all files including the dot files l for long format (file type, permissions, #links, owner, group, etc) R for recursive, list subdirectories. F for listing directories with a trailing /
ps [<options>]
List the information about running processes Example: %ps -el # print the info about all processes (e) in the long format (l)
15
For shell command, system programs, and library functions. %man [n] <command>
e.g. %man wait %man man %man -k <keywords> %man 1 wait %man 1 man
On-line Documentation
16
I/O Redirection
Sequential:
%date %ps -ef %who
OR
Concurrent:
%pgm1 & prgm2 > file1 & pgm3 %make > log & %sort +1 pdir; ((pr dir | lpr) & sort +1 local))
18
Each shell program recognizes a set of special characters called meta characters. The metacharacters are used to create patterns to match filenames and command names. Bourne and Korn shell meta/wildcard characters
* ? [a-dA-D] [!AB123] \ matches any string (including null) matches any one character matches any one character in the range matches a char not in this range escape
Assume we have the following files under the current directory: 120, 235, 2#5, a.out, c22, c*2, Doc.1, Doc.2, Doc.3, one.c,two.c, three.c
ls *.c ls [a-z]*[!0-9] ls ??? ls * ls c*2 a.* *.* cd ~foo
20
Filters
Filters can be combined to work together using pipes Pipe: takes stdout of one command and uses it as stdin of another command ls | wc
21
Command Alias
22
Directory
A file that contains information for files distinction between a directory and a file
system can alter the contents of a directory directories can contain both files and other directories info.
23
The superblock is at a fixed location relative to the beginning of the file system. It contains a description of the file system. One can find the location of the I-node table thru superblock. Each entry of the I-node table is an I-node, which uniquely represents a file in the file system.
I-node table file1 file2 free file3 free
superblock
An I-node contains uid, gid, time of creation, modification, and access, permissions, location of the file on the disk.
25
od -b .
Inodes
A filename is a link (links name in directory hierarchy to the inode, thus to the file contents)
26
Directories
can be read as ordinary files (by any program that reads text) cant be created or written as ordinary files (only system can)
od - octal dump cat foo hi there welcome to unix od -c myfile (output in decimal by byte pairs) 0000000 h i t h e r e \n w e l c o m e 0000020 t o u n i x \n 0000031 1st 7-digits are position in file, ordinal number of next character (in octal)
27
Symbolic Links
Can have many links to the same file (inode) rm - does not remove inode, removes directory entry (link) Only when all links are gone is the file (inode) removed ln -command for creating symbolic links ln oldfile newfile (creates another link to the inode)
28
Permissions
user group others #links owner size (time of last mod) (file name)
Problem: how do you write a program which allows different users to access a set of files? E.g. the program passwd
29
Permissions (cont.)
Solution: use the set-uid bit When a user execute a program with the set-uid bit set, the user assume the identity of the owner of the program.
For example
ls -l /bin/passwd -rwsr-xr-x 1 root set-uid 8454 Jan 4 1994 /bin/passwd
Set-uid bit may break the security wall. (users can run the /bin/passwd and act like root) Only special programs can be set-uid program, particularly if the owner is root.
30
Chmod
permissions can be specified as +x, or u+x, or g+r, chmod +s test sets the set-uid bit for file test.
If a directory has x in its permision, the dir is searchable, ie., one can do ls on the directory.
31
Pathnames
Every file and directory in the file system can be identified by a full path name (route from root to file)
/home/sue/email/f1
/ home sue
if cwd is /home/sue:
ls email ls ./email cd .. ls ../fred
docs
f1
email f2
32
a file system
A file system must be mounted before it can be used Root file system is mounted at boot time. A file system can be mounted to a dir. of another mounted fs. Mounting is done in memory
33
Whats Next?
Shell scripts!
34
Some Details
ls [-alRF] file-list
a for listing all files including the dot files l for long format R for recrusive. list the all subdirectories. F for listing directories with a trailing /
date [+format]
%date +%h %d, 19%y Oct 1, 1996
35
wc file-list
more file-list
Browse through text files on page at a time.
36
Cut out selected charaters or fields from each line of a file Examples: %cut -c 1-5,15-20 foo # extract chars 1-5 and 5-20 from each line of foo. %cut -f 1,3 -d moo # extract field 1 and 3 from each line of moo.
37
diff
diff (cont.)
%diff file1 file2 3c3 <bananas ---------------->kumquats %diff file1 file3 1d0 <apples 3a3,4 >kiwis >peaches
40
Takes two sorted text fiels and print common lines and lines which appear exclusively in one file on separate colmns. column1: lines only in file1, column 2: lines only in file2; col 3: comm Example file1 file2 %comm file1 file2 apple apple apple banana cantaloup banana grape grade cantaloup orange kiwi grape lemon kiwi %comm -12 file1 file2 apple grape 41
43
45
key := c (create) | t (table of content) |r (append the file) | u (update the file) options := v (verbose) | b (block) | f (file name follows) | m (use extraction time as the mod file) Create/extracting archive files for backup and transporting files %tar cvf proj.tar proj # create archive file proj.tar from file or dir proj %tar xvf proj.tar # extract files in proj.tar % tar tf proj.tar # list of the filenames in proj.tar without extracting data. %tar cf - proj | (cd /newproj/; tar xvpf -) # copy proj to the directory /newproj/. p to keep all the protection mode. cp -r copies a dir tree but all the time info is gone. Tar preserve the time info. %tar cbf 20 proj.tar /usr/local/proj # avoid using full path names. 46 When you extract the file, tar will insist to put fiels to /usr/local/proj.
47
A stream editor. It copies the lines from the file_list or stdin to stdout, editing the lines in the process.
Examples: %sed -n /hello/p < input > output # copy the lines contains hello. -n suppress stdout so only the lines that matches are copied. %sed 5,7d file1 # delete lines 5 to 7 from file1. File1 is unchanged. %sed s/Unix/UNIX/ doc2 #replace the first occurrence of Unix in each line by UNIX. %sed s/Unix/UNIX/g doc2 # replace all Unix by UNIX
48
49
awk (cont.)
Test data file (awktest):
Line1-field1 this is the las-field This line should not show Unix is simple and difficult Hello world is very simple Next blank line should show I line some other NonUnix This line should not show Bye
50
awk (cont.)
%awk -F -f awkexample awktest
Line1-field1 las-field This is an empty line Line starts with Unix Unix is simple and difficult Hello world is very simple Next blank line should show This is an empty line End with NonUnix This is an empty line 11 lines processed
Command line program: most of the awk commands can be used in he command line. %awk {print $2 $1} # exchange field1 and field2
51
Other Commands
Only owner and root can chown. (Root only in some systems.Why?) Example: %chown -R john testdir # john becomes the owner of all files under testdir.
53
make
A tool for maintaining programming projects make [-isrntqpd] [-f file] [macro-definition] [targets]
It allows the users to specify dependencies among different source and binary files in his/her applications. -i ignore error code returned by a command -s silent mode -r suppress built-in rules -n no execute mode -t touch target file -q question before change -p printout macro definitions and target descriptions -d debug mode 54 -f alernative make file name
make (cont.)
Makefile: prog: x.o y.o z.o cc -o prog x.o y.o z.o -lm x.o: x.c def.h cc -c x.c y.o:y.c def.h cc -c y.c prog z.o:z.c cc -c z.c
z.o
Makefile format
make Macro
Syntax: Name=String. E.g. LIB=/users3/foo/lib Predefined Macros for C
CC=cc AS=as CFLAGS= -O -g LOADLIBS=
57
An Example of make
MYPROG=/usr/local/myprog INCLUDE=$(MYPROG)/include BIN=$(MYPROG)/bin LIB=$(MYPROG)/lib CFLAGS= -g -I$(INCLUDE) .c.o: $(CC) $(CFLAGS) -c $*.c HEADERS=interface.h dbms.h SOURCE =driver.c interface.c dbms.c OBJECT=$(SOURCE:.c=.o) app: ($OBJECT) $(CC) -o app $(OBJECT) -l$(LIB)
(continued)
59
make-example (cont.)
print: @echo print source files # @ suppress the comman line printing @for file in $(SOURCE) \ do \ pr -n $$file; \ # $$ to make a $-sign for the shell command done clean: @rm -f *.o Usage: make app make clean
60
Processes
Process environment
Process id, parent-process-id, process-group-id Opened files, env Working directory, File creation mask code User ID, Group ID, Resource limits, Environment variables Code
61
UNIX Process
62
Processes
use & to execute a task in background
ps - list processes. jobs - list background processes. ctrl-C (cancel foreground job) ctrl-Z (suspend foreground job) bg - move (suspended) job into background. fg pid - move background job to foreground. kill pid - kill the process
-1 (kill process, and children) -9 (kill process, may leave children alive)
63
Shell Process
Upon login: shell process created Any command you type at prompt: new child of your shell process What is your current shell? %echo $SHELL How to switch to another shell? %bash just type shell name How to switch login shell? %chsh user newshell (but wont work here)
64
Shell Metacharacters
> >> < | * ? [ccc] ; & prog > file prog>>file prog<file p1|p2 direct stdout to file append stdout to file take stdin from file connect stdout of p1 to stdin to p2 match string of 0 or more characters match any single character match any single character from ccc ranges like 0-9 or a-e are legal command terminator background process
65
Shell Metacharacters
\c
run commands the output of produces run commands in in a sub-shell arguments to shell file value of shell variable var take c literally (dont evaluate) take literally take literally, after $, ``, \ interpreted comment assign variable var run p1, if successful, run p2 run p1, if unsuccessful, run p2
66
What processes, programs, pipes, and files are used? %cat doc1 > out1; wc -l <out1 %grep root /etc/passwd >OUT2 ; cat <IN >OUT & %cat doc1 | wc -l
67
68
Whats Next?
Shell scripts!
69
Standard command format Recognize meta-characters (handle multiple files) Standard I/O (stdin,stdout, stderr. If file arg is absent use std) Keep messages and prompts to a minimum. Provide verbose options Input/output data should be text whenever possible. Use dot files and environment variables for frequently used info. Use standard library and tools to save coding effort.
70
Shell Script
71
Environment Variables
.profile
PATH=.:$HOME/bin:/bin:/usr/bin:/usr/local TERM=vt100 export TERM PATH HOME -- home directory PATH -- where to look for commands TERM -- terminal type for programs such as the editor CDPATH -- shortcut directories for cd command PS1 PS2 -- prompt
72
73
Another Example
$cat shellvariables #! /bin/sh # show predefined shell variables echo The number of arguments is $# date & echo The process id of date command is $! wait echo The process id of this shell is $$ grep root /etc/passwd echo The return code from grep is $? echo The current set of options is $- $shellvariables one 2 xyz $sh -a shellvariables one 2 xyz
74
Positional parameters
set to shell script arguments. e.g. $my_script a b xy 1 2 3 positional parameters have the values $0 -- my_script $1 -- a $2 -- b $3 -- xy $4 -- 1 2 3 shift command: $shift [n] # parambers $n+1 $n+2 become $1 $2 . # default value of n is 1
75
Quoting
string take string literally $echo * $HOME * $HOME string take string literally, except $,`,\,, \c take c literally
77
Compound commands:
a pipeline, a list, a group (), a command that begins with a certain reserved words: for, if, case, time, I/O redirection applies to the complete command except a pipeline, a list and the time command. 78
test Command or [ ]
if test $# -eq 0 then echo no positional param! fi if [ $# -eq 0 ] then echo no positional param! fi
options: -r | -w | -x | -f | -d file # the file is readable, writeable, executable, a file, or a directory. n1 -eq | -ne | -gt | -ge | -lt | -le n2 # n1 = | <> | > | >= | < | <= n2
79
Test Command
Options
File testing -r file -w file -x file -f file -d file Logical Connectives ! -a -o
80
Parameter substitution
Parameter substitution ${param} ${param:-word} value of param if defined, otherwise word, param remains undefined ${param:=word} value of param if defined, otherwise word, if param undefined, param defined to word ${param:?word} if defined, param, otherwise print word and exit shell ${param:+word} word if param defined, otherwise nothing
81
command grouping:
(command_list) {command_list}
Function definition
function id { compound_list }
82
CX command
Suppose you want a command cx that will take a filename and set its execute permission. For example, $cx foo Need to be able to get the filename from the command line Shell variables: $1 (first argument), $2 (second), $* (all args) chmod +x $1
83
Flow Control
General Format:
if command_list1 then command_list2 elif command_list3 then command_list4 . else command_listn fi check exit code of the command 0 -> normal termination -> True non-0 -> abnormal termination -> False if test -d /usr; then echo its a dir; fi $if test -d /dir >then > echo its a dir >fi >$
84
Looping
What does this command do? $wc -l * 6 file1 10 file2 3 file3 19 total What if you would like things formatted nicely? There are 6 lines in file1 There are 10 lines in file2 There are 3 lines in file3
85
Loops
General Format:
while command(s) do body done check return code of command, if 0, execute body and loop again
while test ! -s file1 do echo file1 does not exist or is still empty sleep 500 done
86
Loops
General Format:
until command do body done until who | grep $1 do echo $1 has not logged in yet sleep 500 done
87
Shell scripts
for f in $* do x=`wc -l $f` echo There are `echo $x |cut -f1 -d ` lines in $f done
88
Examples
if [$NAME = John Doe -a $BAL -gt 5000] then echo ok else echo not ok fi
89
Examples
if test -f $1 then echo First arg is a file elif test -d $1 then echo First arg is a directory fi
90