Академический Документы
Профессиональный Документы
Культура Документы
Overview File System Shell Programming Process Management Inter Process Communication System Administration Utilities
Overview
Kernel Functions
Schedule Processes Keep track of files Control hardware devices
File System
File System
Characteristics of file system Types of files File system layout File structure related system calls
File System
Characteristics of Unix File System Hierarchical Structure Consistent treatment of file data Ability to create and delete files Dynamic growth of files Treatment of peripheral devices as files
File System
Types of Files Regular files Directories Special files Pipes Named Pipes I/O devices
File System
File System Layout Boot block contains bootstrap code to initialize the operating system Super block describes the state of a file system Inode list is a list of inodes Data blocks contain file and administrative data
10
11
OPEN
#include <fcntl.h>
12
CREAT
Creates a new file with the indicated filename and access permission modes. If the file already exists, creat truncates the file.
13
READ
Read returns the number of bytes it read into the buffer. It is generally count or a value less than count if the number of bytes left to be read are less than the count
WRITE
Write writes count bytes of data from user address buffer to the file descriptor fd number = write ( fd, buffer, count ) fd is the file descriptor buffer is the address of the data structure count is the no of bytes to be written number is the no of bytes actually written
15
LSEEK
Changes the position of read-write pointer lseek ( fd, offset, reference ) int fd is the file descriptor long offset is the byte offset int reference indicates where the offset is to be considered from reference values 0 1 2 set the pointer to offset bytes from beginning of file increment the current value of pointer by offset set the pointer to size of file plus offste bytes
16
CLOSE
Close closes a file descriptor obtained from a prior open, creat, dup, pipe or fcntl system call
17
STAT
Stat returns status information about the specified file stat ( filename, statbuffer ) char *filename is the absolute file name struct stat *statbuffer is the address of the data structure in the user process which will contain status information
18
FSTAT
Fstat returns the status information of an open file whose file descriptor is fd fstat ( fd, statbuffer ) int fd is the file descriptor struct stat *statbuffer is the address of the data structure in the user process which will contain status information
19
DUP
Dup duplicates the specified file descriptor, returning the lowest available file descriptor newfd = dup ( fd ) int fd is the file descriptor int newfd is the new file descriptor
20
LINK
Link gives another name filename2 to an existing file filename1 link ( filename1, filename2 ) char *filename1 is an existing file char *filename2 is the new name which will also point to the source file name
21
ACCESS
Checks if calling process has r,w or x permission for a file access ( filename, access_mode ) char *filename is the name of the file access_mode 04 02 01 00 read write read existence
CHMOD
Chmod changes the access permissions of the indicated file to the specified mode
chmod(filename, mode)
char *filename Modes
4000
2000 1000
setuid bit
setgid bit sticky bit
0400
0040 0004
CHOWN
Chown changes the owner and the group of the indicated file to the specified owner and group IDs
chown(filename, owner, group) char *filename int owner, group name of the file
24
UMASK
Sets the file mode creation mask and return the old value. When creating a file, permissions are turned off if the corresponding bits in the mask are set
umask(mask) int mask
25
IOCTL
Ioctl does device-specific operations on the open device whose file descriptor is fd.
ioctl ( fd, cmd, arg_pointer) int fd; int cmd is the device specific command arg_pointer defines a parameter whose type depends on the command
26
SHELL PROGRAMMING
27
What is a Shell
Standard command and programming language Interprets command lines Runs other programs Reads commands either from the prompt or from a file
28
Shell Types
sh (bourne shell) csh ksh (korn shell) bash (bourne again shell)
29
Input/Output Redirection
Create Files Append to files Use files as input to the shell Truncate files
30
Input/Output Redirection
Redirector Operators < > >> 2>&1 | Open the following file as stdin Open the following file as stdout Append to the following file Merge file descriptor 2 with 1 Pipe stdout into stdin
31
File/Directory Commands
cd ls mkdir rmdir cat ln mv cp rm Change directory List directorys contents Create directory Remove directory Concatenate and write files contents Create links to existing files Move files Copy files Remove files
32
Selection Commands
awk cut diff grep sed head tail wc Pattern scanning and processing language Select columns Compare files Select lines or rows Edit streams of data Select lines from top in a file Select lines from bottom in a file Count words, characters and lines in a file
33
Shell Variables
Special Variables Environment Variables Program Variables
34
Special Variables
$# Number of positional arguments $0 Program/command name invoked $1,$2.. Positional parameters passed to the shell $* Expands to $1 $2... $@ Expands to $1 $2.. $? Return code from the last executed command $$ PID of the current shell
35
Environment Variables
HOME PATH PS1 IFS PWD Pathname of the users login directory Shells search path for commands Primary prompt Internal field separator Current working directory
36
Program Variables
Normal variables using assignment operator e.g. age=12
37
Quotes
Single Quotes Double Quotes No evaluation whatsoever No evaluation expect for $, \ and `
Backslash
38
Expr
Evaluates arguments and returns 0 or 1 Returns numeric value from an expression
39
Foreground/Background Execution
Background processing by placing an & at the end Background processes run effectively wrapped around by nohup loops run in background enclosed in parentheses fg command moves a background job into foreground
40
READ
Main input mechanism for a shell program Gets a line from the standard input and can store the value in a variable Can read a file line by line
41
Control Structures
42
If ..then ..else
If <condition>
then <command>
[elif <condition>
<command> else <command>] fi
43
Case
case word in pattern1) action1 ;;
pattern2|pattern3)
action2 ;;
....................................... .......................................
*)
esac
default action ;;
44
While .. do
while <condition> do
commands
done
45
test
Tests if an expression is true Returns a 0 if true else returns a non-zero value
46
test
Integer operators used with test eq is equal to
ne
lt le
is not equal to
is less than less than or equal to
gt
ge
greater than
greater than or equal to
47
test
Syntax test <expression>
or
[ <expression> ]
alternative syntax
48
test
File Operators -d -f file exists and is a directory file exists and is an ordinary file
-r
-s -w
-x
49
Filename Wildcards
*
?
[a-z]
[!0-9] Match any character not in the square brackets +(abc) Match one or more instance of abc *(abc) Match zero or more instances of abc
50
Functions
function identifier {list;} or
identifier() {list;}
51
Command Execution
Grave accents or backquotes `command`
{command}
52
53
Setuid/Setgid Permissions
This powerful feature enables the effective userid/groupid of the calling process to be the same as the userid/groupid of the owner of the file.
54
Setuid/Setgid Permissions
The setuid bit is turned on like this chmod 4000 filename
Sticky Bit
This is set to prevent files from being removed from a directory by arbitrary users. chmod 1000 directoryname
In this case, only the owner of the directory, owner of the file or the superuser can remove a file in the directory
56
Find
Finds files that match a given set of parameters find -name filename Finds files with the filename
find -size n
find -atime n
Of size n
Access time n days ago
57
Find
Operators used with find
-print find -name filename -print Prints the filename
-exec
find -name filename -exec rm -f {} \; Executes command
58
File
Tells about the contents of the file
file filename
59
Copy Files
cp command
example cp file1 file2
60
Copy Directories
cp -r|R command
Recursively copies the files under a directory and its subdirectories
tar command Use tar cvf to create a tar of the subdirectories and files under a given directory e.g. tar cvf tarname path of files/subdirectories
61
Copy Directories
Untar a tar file using
tar xvf tarname untar location
62
Mail
Used for sending mails
e.g. cat msg | mail -s subject recipient
Note that the mail program expects its input from the stdin
63
Cut
Selects columns or fields
-c -f for col for field
-d
for delimiter
64
Sort
Sort divides each lines into fields at whitespace (blanks or tabs) and sorts the lines by fields from left to right
sort +n -m sorts the nth field, stops sorting at the mth field
65
Process Status
Produces a report summarizing execution statistics for the current processes.
E.g. ps -ef gives general stats ps -fu username gives stats about all the processes running under the user username
66
Time
Used to time a command
e.g.
time progname
67
Uptime
Used to get a rough estimate of the system load
e.g. Uptime
68
Cron
Does periodic execution of jobs
crontab -l
Editing cron entries crontab -e e.g 20 10 * * * /$HOME/myscript
69
70
EXEC
Execve executes the program file filename, overlaying the address space of the executing process.Argv is an array of character string parameters to the execed program and envp is an array of character strings that are the environment of the new process
execve(filename, argv, envp) char *filename char *argv[] char *envp[]
71
FORK
Fork creates a new process. The child process is a logical copy of the parent process, except that the parents return value from the fork is the process ID of the child, and the childs return value is 0
72
WAIT
Wait causes the process to sleep until it discovers a child process that had exited or a process asleep in trace mode. If wait_stat is not 0, it points to an address that contains status information on return from the call. wait(wait_stat) int *wait_stat
73
EXIT
Exit causes the calling process to terminate, reporting the 8 low-order bits of status to its waiting parent. The kernel may call exit internally, in response to certain signals exit(status) int status
74
75
SIGNAL
Software Interrupts
76
Classification of Signals
Termination of a process Process induced exceptions Unrecoverable conditions during system calls Unexpected error condition during a system call Tracing execution of a process
77
Handling Signals
Exit on receipt of a signal Ignore the signal Execute a particular function on receipt of a signal
78
Signal Syntax
#include <signal.h> signal(sig,function) int sig; void (*func)() Signal allows the calling process to control signal processing.
79
Signal Syntax
Some values of sig
SIGHUP
SIGINT SIGQUIT SIGKILL SIGFPE SIGBUS SIGPIPE SIGTERM
hangup
interrupt quit kill floating point exception bus error write on a pipe with no reader software termination
80
Signal Syntax
The function is interpreted as SIG_DFL default operation. Process terminates except for some signals ignore the signal.kernel calls a function in the process with signal number as the argument. cant be ignored
SIG_IGN
SIGKILL
81
Kill
Kill sends the signal sig to the process identified by pid
kill(pid,sig)
int pid,sig pid > 0 send signal to process whose PID is pid pid 0 send signal to processes whose process group ID is pid of sender pid -1 if effective UID of sender is of superuser send signal to all processes otherwise to processes whose real UID equals effective UID of the sender
82
83
Types of IPC
Pipes Named Pipes Messages Shared Memory Semaphores Sockets
84
Pipes
Allow data transfer in a FIFO manner Allow synchronization of process execution Use file system for data storage Can be created using pipe() system call Pipe manipulation is done using regular system calls for files e.g. read(), write() Only related processes can access a pipe
85
Pipes
Syntax pipe (file_descriptors) where file_descriptors is a pointer to an array of 2 integers which are the file descriptors for reading and writing the pipe
86
Named Pipes
Similar to pipes Have directory entries open system call used to open named pipes mknod system call to create named pipes Unrelated processes can also communicate using named pipes
87
MESSAGES
Allow processes to send data streams to arbitrary processes Four system calls cover all the operations msgget msgsend msgrec msgctl
88
msgget
Returns the message queue identifier assocaited with the key msgget(key, flag)
89
msgsnd
msgsnd (id, message, size, flag) Sends message to the queue associated with the message queue identifier specified by id message is of the form struct msgbuf
{
long mtype; char mtext[]; } Flags can be IPC_NOWAIT- Return -1 if the message cant be stored MSG_NOERROR- Message bigger than size is shortened with no error returned
90
msgrcv
msgsnd (id, message, size, type, flag)
Reads message from the queue associated with the message queue identifier specified by id
message is of the form
struct msgbuf
{ long mtype; char mtext[]; }
91
msgctl
Msgctl ( id, command, buffer )
IPC_STAT
IPC_SET IPC_RMID
92
SHARED MEMORY
Allows processes to share part of their virtual address space System calls similar to the ones in messages
93
shmget
int shmget ( key, size, flag )
key is the name of the shared memory size is the size of shared memory segment in bytes flag is a combination of IPC_CREAT IPC_EXCL mode
94
shmat
Inserts a shared memory segment into a process address space char *shmat ( id, at_address, flag ) id is the shared memory segment identifier size is the address in process address space where the shared memory segment has to be attached flag is a combination of SHM_RDONLY read only 0 read,write
95
shmdt
Removes a shared memory segment into a process address space
96
shmctl
shmctl ( id, cmd, buf)
97
SEMAPHORE
Non-negative integer count used to coordinate access to the resources Initial semaphore count set to the number of free resources Count decremented or incremented by the threads or processes as and when they acquire or free resources. Threads block at count zero till it becomes positive
98
semget
semget ( key, num, flag )
Returns or creates the semaphore identifier associated with the key key is the name of the semaphore set number defines the number of semaphores in the set flag is a combination of
IPC_CREAT
IPC_EXCL
99
semop
semop ( id, operations, number )
-operation
-flags number is the number of entries in operations
100
semctl
semctl ( id, semnum, cmd, ... ) Performs operations on the semaphore as specified by the cmd id is the semaphore identifier Fourth argument is optional, depending upon the operation requested. If required, then its a buffer. cmd GETVAL, SETVAL, GETPID, GETNCNT, GETZCNT
101
SOCKET
Creates an endpoint for communication and returns a descriptor int socket ( domain, type, protocol ) domain specifies communication domain type type of communication over socket (virtual circuit or datagram) protocol protocol to control the communication
102
103
Close
104
Bind
105
Connect
106
Accept
addr user data array which kernel fills with the address of the connecting client
107
send
Transmits data over a connected socket send ( sd, msg, length, flag)
108
recv
Receives data over a connected socket recv ( sd, buf, length, flag)
109
shutdown
110
System Administration
File System Administration Machine Management System Tuning and Performance Management System Accounting User Management Backup/Restore Activities Setting up Peripheral Devices Routine Maintenance
111
112
113
114
System States
System State 0 s 2 3 5 6 q Description Powerdown State Single user Multiuser Start Remote File Share Switch to firmware Halt & Reboot OS Reexamine (/etc/inittab)
115
SHUTDOWN -I S
INIT S Switching to any state INIT <System State> Switching to multiuser
INIT 2
Turning system off SHUTDOWN -I 0
116
117
119
SYSTEM UTILITIES
120
MAKE
Provides a method for maintaining an up-to-date version of programs that consist of a number of files generated in a variety of ways
121
MAKE
Operations
Find the target in the description file Ensure that all the dependencies (files) of the target exist and are up to date Create the target in case any of the dependencies have been modified
122
MAKE
MakeFile
Comments
Continuation lines
Macro Definition
Lines ending in \
Identifier followed by = sign
123
MAKE
Example MakeFile makedemo #makedemo prog : x.o y.o z.o cc x.o y.o z.o -o prog x.o : x.c defs.h cc x.c y.o : y.c defs.h cc y.c z.o : z.c defs.h cc z.c
124
MAKE
Macros Macro Definition name = value
125
SYSTEM UTILITIES
126
Grep
Searches the named files or the standard input and prints each line that contains an instance of the pattern
127
Grep
Usage grep [options] pattern filenames
Options -n -v prints the line numbers inverts the sense of the test
-i
128
Grep
Example
Locate variable in C source code grep variable *.[ch] See if abc is logged in
129
AWK
Scans a set of input lines, searching for lines that match any set of patterns and take a specified action for each pattern on each line that matches the pattern
130
AWK
Usage
awk pattern-action statements optional list of input files e.g. awk {print $1, $2} file1 file2 OR awk -f programfile optional list of input files where programfile is the file where the pattern-action statements are stored
131
AWK
Fields
Record is the sequence of characters separated by a newline character Fields in a record are separated by sequence of blanks or tabs
$0 refers to the entire record $1 refers to the first field $2 refers to the second field
132
AWK
Printing
{print $1 $3} prints the first and third fields of each of the input line
133
AWK
BuiltIn Variables
ARGC
ARGV
FNR
NR NF OFS ORS
AWK
Examples
AWK
BuiltIn Arithmetic Functions
sqrt(x)
srand(x)
136
AWK
BuiltIn String Functions
returns position of t in s returns length of s splits into array a on FS substitutes s for first r returns suffix of string starting at position p
137
AWK
Control Flow Statements if (expression) stat1 else stat2 while (expression) stat for (expr1;expr2;expr3) stat
SED
Stream Editor
Reads one line at a time from input files Applies the commands from a list of commands one at a time, in order, to each line
139
SED
Usage
sed list of ed commands filenames... Example sed s/UNIX/UNIX(TM)/g filenames replaces UNIX by UNIX(TM) in all the file occurences and writes the result to standard output
140
SED
Commands
Appends text on the output before reading the next input line Branch to the : command bearing the label change lines to the following text delete line; read next input line print read write quit
141