DBT544 Class 1 UDB/400 DB2 Universal Database for the AS/400

Who is Tim McKenna? Cursed Standards What is this course? o download the DBTPPTZIP file in the course library: DBT544TM Note: IBM holds more database patents than all their competitors combined. See Analyzing The Relational Database Vendors for InformationWeek's Dec/2000 industry survey of IBM vs. Oracle vs. Microsoft DBs. See The Future of SQL/400 and the iSeries 400 Database by Paul Conte, NEWS/400 Senior Technical Editor (and the author of our textbook). IBM Partners In Education (PIE) IBM's DB2 UDB for AS/400 Web site AS400 Network SQL/400 and DB Community SQL/400 Developer's Guide textbook Club Tech iSeries DB2 UDB and SQL/400 Newsletter

UDB/400 native database within Qsys.lib Other file systems are supported by the IFS (Integrated File System). iSeries or AS/400 is the only platform that supports all Microsoft/Novell/Unix file systems. SQL runtime support is standard in OS/400. SQL Developers Kit is optional. Paul Conte's FAQ: WHAT IS THE AS/400's DATABASE? I frequently get the question, "What is the AS/400's database?" or "What's the difference between "DB2" and "UDB"? Here are the answers to these and some related questions that come up fairly regularly: a. First off, be sure you understand that "iSeries" is just the new name for the "AS/400." Technically, IBM calls some newer hardware models the iSeries, while older models are AS/400s, but the iSeries and the AS/400 both run the OS/400 operating system and have the SAME database. b. The iSeries (and the AS/400) run one and only one database, which in V4R5 is called DB2 Universal Database for AS/400. I like to use "UDB/400" as the short form, but "DB2/400" means the same thing. In a future release, I imagine IBM will change "for AS/400" to "for iSeries", but it will still be just another release of the same integrated database that's always been on the AS/400. You can learn the product

details at http://www-1.ibm.com/servers/eserver/iseries/db2 [Footnote: There are a couple of Java-based databases - Cloudscape and PointBase - that can run on any machine with a Java Virtual Machine (JVM) and thus can also run on the iSeries and AS/400. But at this point, these aren't practical alternatives to UDB/400.] c. UDB/400 comes as part of OS/400, so EVERY iSeries (and AS/400) system includes it. You don't have to buy UDB/400 as a separate product as you do other database management systems on other platforms. That's cool because it means application developers can count on the database being on any iSeries (or AS/400) on which they want to install their software. You still have to be concerned with release compatibility, of course - you can't run an application that uses a feature introduced in V4R5 on a system running an earlier release (e.g., V4R4) of OS/400. You CAN, however, run an application created on a V4R5 system on a system running an earlier release, as long as the application uses ONLY features available in the earlier release. d. NONE of the other major relational database management systems (RDBMSs) runs on the iSeries. You can't run Oracle, Informix, or Microsoft SQL Server 2000 on an iSeries. e. UDB/400 includes runtime support for both SQL database operations and "traditional" RPG/400, ILE RPG, Cobol, and other high-level language I/O. You do NOT need to buy anything (other than OS/400) to run SQL applications. f. The DB2 Query Manager and SQL Development Kit for AS/400 product is an addon product you can buy to DEVELOP HLL applications that use embedded SQL. You must have this product to translate HLL source files that include SQL statements as part of the program-creation process. You do NOT need this product to RUN a compiled SQL application. The product also includes a green- screen Interactive SQL (ISQL) utility that lets you run SQL statements. ISQL is meant for programmers, not end users. And, last but not least, the product includes Query Manager (QM), another green-screen utility to define reports. QM is intended for end users. You can learn more about the product at http://www.ibm.com/servers/eserver/iseries/db2/db2sql.htm
SQL database file collection (schema) table (relation) DB2/400 library PF IFS drive/dir byte stream file

set element data path dynamic selection

Row (Tuple) Column (Attribute) Index View

record program described field LF OPNQRYF

security

Grant/Revoke

xxxOBJAUT

basic by UserID

Data types character strings packed numbers (use odd lengths) numbers date time date & time binary data Binary Numbers Floating Point Char Decimal Numeric Date Time Timestamp Hex Binary Float A P S L T Z H B F

Field Reference File: like a data dictionary Physical Files contain the data in members. Usually only one member per file. SQL does not support members (must use OVRDBF). Logical File applications:

permanent alternate access path (performance good/bad) restrict usage: e.g. input only security (using selection &/or projection and ObjAut) string handling (using substring)

access multiple file members in a single file access multiple physical files in a single file Logical File techniques:

Selection (restrict/select records) - performance - security Projection (specified columns) - reduces program size (not too many fields in memory) - security (column security available in SQL) join (read only) multi-format (for update) - e.g. Order Header, Order Lines access multiple file members in a single file access multiple physical files in a single file

DBT544 Class 2 DB2 -- the traditional approach Field Reference File: like a data dictionary Physical Files contain the data in members. Usually only one member per file. SQL does not support members directly (must use OVRDBF or create a table ALIAS). Creating a Physical File:

always use a Primary Key (minimal and UNIQUE) avoid use of Relative Record Number processing avoid using FIFO, LIFO, FCFO in Logical Files to deal with duplicate key issues always put a TEXT keyword on the record format (for program documentation) Fields should have these keywords: COLHDG is user interface with data TEXT is the programmer interface; specify if different from COLHDG ALIAS field names for COBOL DFT specifies a default value for new records VALUES, RANGE, COMP to validate input Validating input with DDS keywords

poor user interface only effective if user inputs via a DSPF that adopts the validation

GO CMDCST to use Constraint Commands Validating input with PF Constraints

always enforced requires program error checking techniques consolidates the business rules in one place very poor user interface with DFU Validating input with HLL

only enforced via programs simple to complex checking of business rules allows exceptions to rules rules can be broken by program bugs & utilities (DFU) requires simple program editing & error msg techniques good user interface (as good as you make it)

Run the DSPKEY command to view the records of physical or logical files in key sequence. This is not a standard OS/400 command. See DBT544TM/QCLPSRC for the source code. Is DDS Dead?
DBT544 SQL Introduction

SQL, Structured Query Language, is the direction for new DB2 UDB/400 development. Old DDS interface has been "frozen". In IBM speak, this means it continues to be supported but will not be developed further. The challenge for AS/400 professionals:

support the old DDS systems (they will be around for a while) create new DB using SQL o expect resistence from clients with established systems o requires additional Program Products: Query Manager and SQL Developers Kit

o o o

develop in SQL, even for established systems practice 'backward compatible' techniques i.e. be OS/400, RPG/400, COBOL/400 friendly meaningful table & column names <= 10 char for ease of use in native tools refering to files and fields.
native PF OPNQRYF Query/400 comments permanent data object

SQL base table

result table

temporary

view index

LF

alternate data path

RunSqlStm Run SQL Statement

CrtPF/LF, ChgPF/LF, Go CMDCST

SQL statements & DDS definitions stored in source members RunSqlStm cannot run Select (must use Query Manager) can use to create tables/files but only recommended for testing -- results in limited documentation, hard to maintain if no source. new features / ease of use development will be in Operations Navigator

STRSQL Start Interactive SQL (ISQL)

Query/400 (very limited in comparison)

Operations Navigator

green screen

RunSqlStm source Create Table Label On

DDS source CRTxF COLHDG Column Headings (end user interface to data)

Comment On Drop

TEXT DLTF, CHGPF GRTOBJAUT, RVKOBJAUT

programmer/query user interface to data delete table/file, remove column/field only SQL has column security native requires LF with projection SQL can act on sets of rows (powerful and dangerous) DFU is for end user, one row at a time.

Grant & Revoke

Insert, Update, Delete

UPDDTA

Select * from table [output to file in ISQL] CREATE TABLE x LIKE existing_table plus INSERT -- comments

CPYF

copy a file

* (col 6) comments

possibly the most important source specs.

STRSQL F13=Services, 1. Change session attributes o Date format . . . . . . . . . *ISO YYYYMMDD -- the one, true format o Naming convention . . . . . . *SYS *SYS means library/file syntax *SYS must be specified for ISQL to use the *LIBL Interactive SQL will create new objects in your *CURLIB and look for an unqualified table names in your *LIBL only if your ISQL "Naming convention" is *SYS. *SQL means collection.table syntax textbook uses this format ISQL does not support DftRdbCol option (as other SQL functions do) default collection name is your User ID!

Collection list . . . . . . . *USRLIBL Saves a lot of keystrokes when prompting collections with F4.

Statement processing . . . . . *RUN *RUN The SQL statement is syntax and validity checked, and then run. *VLD The SQL statement is syntax and validity checked only.

all statement elements referred to (collection, table, view, column, library, file, field) verified for existance develop DML statements to save in a source member *SYN The SQL statement is syntax checked only statement is constructed according to the SQL rules develop DDL statements to to be run on other systems

F3/F13 can save the ISQL session statements in a source member. You can edit the statements and run them with the RunSqlStm command.

SQL Catalog: (see textbook page 37, Fig 2.13) QSYS2.Sys* => about the entire AS/400 database -- only one SQL DB on an AS/400 collection.Sys* => about the collection/library use Create Collection libName to create an AS/400 library => get journals for rollback and commitment control => get SQL Catalog about your collection => use CHGLIB or other SQL statements to modify afterward Create Schema use to define entire collection with tables, views, etc. all at once. Good for creating the completely developed collection on a client's AS/400.
SQL Create Table native CRTPF comments

OS/400 SQL table (also a *FILE object)

*FILE object

create with meaningful OS/400 name, then Rename Table OS/400file To longSQLtableName

hard to deal with cryptic SQL created OS/400 system names for objects, e.g. CUSTF00001 can have X members max. records: 10,000 plus 1000 3, must CHGPF or o/ride CRTPF for *NOMAX must CHGPF or o/ride CRTPF to Reuse deleted records

only one member

no maximum size

SQL saves "file full" problems

reuses deleted records

SQL saves RGZPFM

journalling of before/after record changes

up to the programmer to set up or design application audits

* journalling needed for rollback/commitment control * more overhead but good transaction control * very useful with complex entity relationships RPG programs will need to rename SQL record formats

table name = record format name

file name <> record format use ALIAS keyword to create long field/column names

use For clause to assign system field names to SQL columns

field name/alias

distinct type / User-Defined Types: use only for primary/foreign key columns

Field Reference

SQL extremely limited in data dictionary/field reference features

Alter Table Add/Drop/Alter Create {unique} Index

modify DDS and CHGPF

add/delete/change fields or field attributes alternate access path to data only use a table/file alias if you really need it. Why have two names for one thing? Access to multi-members in a file: SQL is permanent alias native is temporary for job remove library/file level objects source members defining tables/files/indexes

DDS and CRTLF

Create Alias

no equivalent

Create Alias FileMbrName For File (Member)

OVRDBF FILE(file name) MBR(member name)

Drop use Restrict keyword

DLTobj

QSQLSRC

QDDSSRC

See DBT544TM/QSQLSRC.@STANDARDS for notes on creating source members to define tables.

DBT544 Data Definition Language If using ISQL (STRSQL cmd),

run CHGCURLIB and EDTLIBL beforehand to adjust your library list for the collections you want to access. Make your current library the name of your most important collection. Any new objects will be created here by default. F13=Services, 1. Change session attributes o Naming convention . . . . . . *SYS *SYS means library/file *SYS must be used in ISQL to use the *LIBL and to have new objects created in your *CURLIB

*SQL means collection.table textbook uses this format ISQL does not support DftRdbCol option (as other SQL functions

do) default collection name is your User ID! o Collection list . . . . . . . *USRLIBL (or default *LIBL) F3/F13 can save the ISQL session statements in a source member for later use by RunSqlStm command When making a new collection:

STRSQL Create Collection exit ISQL you cannot CRTSRCPF QSQLSRC (which will contain your DDL source) in your collection library create your source PF in another library (and optionally copy/move it to your new collection library) create source members, one for each table, index, etc. use RunSqlStm with the Default collection . . . . . . . DFTRDBCOL __________ parameter otherwise you must qualify each DDL statement with the collection name this limits the portability of your specs and ease of test/production library management

See Chap.13, pages 268-276. Indexes About Using Varchar? What is in index? Accelerating Your Queries with Encoded Vector Indexes Indexes support "Order By" clauses in Select statements and "Where" statements in Select statements (and Views) .
DBT544 Views

see DBT544TM/QSQLSRC.VIEWS for examples based on a completed Lab 2 The textbook's explanation of views "with Check Option" is not too detailed. If you want to know more about it, see the SQL Reference manual on this point.

Views allow an alternate projection of columns and/or a selection of rows from the based on table. Views can be based on top of other views. Views provide a way to simplify, as well as restrict, access to data. You can use SQL views to

Select a subset of the rows in a base table o Used to improve performance for commonly or frequently selected rows. e.g. rows for a department; rows for a major customer o Used for security purposes (perhaps With Check Option) to constrain users to only the rows they own. Include only a subset of a base table's columns o Used to hide columns from some users or applications. o Used to reformat columns o Used for security purposes (it may be easier to administer security on a view than on a column by column basis). Derive new view columns based on one or more underlying base table columns o use SubStr and/or ConCat to put a name together o perform calculations such as QTY * PRICE as EXTENDED_PRICE o reformat numeric fields into date data types Join related rows from multiple base tables into a single row in the view o DB Admininstrators like third normal form, users don't. o Views Join Tables Together Combine sets of rows from multiple tables into a single view o UNION operator (Ch.8)

See DBT544TM/QSQLSRC.views source member for examples of creating and using views With Check Options. Reformating Dates

Date, time and TimeStamp data types are output with editing characters format is as per the Date/Time format of the job/ISQL session/RunSqlStm parameter You can extract various components from a Date data type: o Year, Month, Day of a date, timestamp, or duration o Days = number of days since 0001-01-01 => good for interval calculations

DayOfYear = sequential number of day within the year (Julian date) o DayOfWeek = sequential number of day within the week (1-7 = Sun-Sat) o Week = sequential number of week within the year (1-53) o Quarter = 1 to 4 meaning Q1 (Jan-Mar), Q2 (Apr-June), ... You can extract the Date or Time from a TimeStamp data type: o Date(TimeStamp) o Time(TimeStamp) You can reformat the output of a date or time using Char(...): o SELECT CSTLSD as TimeStamp, date(cstlsd) as Date_timestmp, time(cstlsd) as Time_timestmp, char(date(cstlsd),EUR) as Char_date, char(time(cstlsd),USA) as Char_time FROM CUSTPHONE o Char(date/time/timestamp [, ISO, EUR, USA <== to reformat] ) date editing of old numeric format (YYMMDD) requires SubStr & ConCat: from Numeric ==> Digit string ==> Edited Date String o Select cstNXD as YYMMDD, char(cstNXD) as char_string, digits(cstNXD) as digit_string, substr(digits(cstNXD),1,2) as YY, substr(digits(cstNXD),3,2) as MM, substr(digits(cstNXD),5,2) as DD, substr(digits(cstNXD),1,2) ||'/'|| substr(digits(cstNXD),3,2) ||'/'|| substr(digits(cstNXD),5,2) as edited_date from custphone from old numeric format (YYMMDD) to new date data type requires Numeric ==> Digit string ==> Edited Date String ==> Date: o Date format is *YMD (system adds the correct century): o select cstNXD as YYMMDD, DATE( substr(digits(cstNXD),1,2) ||'/'|| substr(digits(cstNXD),3,2) ||'/'|| substr(digits(cstNXD),5,2) ) as date_type from custphone
o o

simple subtraction of two dates gives number of years, months, days (yyyymmdd) as the duration interval

The Age Calculator: CREATE TABLE BIRTHDAY (NAME CHAR (10 ) NOT NULL WITH DEFAULT, BIRTHDATE DATE NOT NULL WITH DEFAULT) o INSERT INTO BIRTHDAY VALUES('name', 'yyyy-mm-dd') (above date in ISO format) or INSERT INTO BIRTHDAY VALUES('name', 'dd.mm.yyyy') (above date in EUR format) or INSERT INTO BIRTHDAY VALUES('name', 'mm/dd/yyyy') (above date in USA format) * system understands any format of a 10 character formatted date string o SELECT name, curdate(), birthdate, digits(curdate() - birthdate) as duration, SmallInt( year(curdate() - birthdate)) as Years, SmallInt(month(curdate() - birthdate)) as Months, SmallInt( day(curdate() - birthdate)) as Days, SmallInt( days(curdate())- days(birthdate)) as Total_Days FROM birthday o SmallInt (small integer) is used above to reduce size of display column report the text week day (Sun - Sat) from the Day of Week o Select birthdate, DayOfWeek(birthdate) as DoW_No, case DayOfWeek(birthdate) when 1 then 'Sunday' when 2 then 'Monday' when 3 then 'Tuesday' when 4 then 'Wednesday' when 5 then 'Thursday' when 6 then 'Friday' when 7 then 'Saturday' else 'who knows?' end as Day_of_birth FROM birthday more compact method to report month o Select birthdate, SUBSTR (
o

'Jan. Feb. MarchAprilMay

June July Aug. Sept.Oct. Nov. Dec. ',

month(birthdate) *5 -4, 5) from birthday o Substring parameters: 1. string of month names, 12 5 char each ... like array of elements 2. extract month number from date element length (length 1) 3. element length Put it all together in a view o Create view birthdayV as SELECT name, curdate() as today, birthdate, digits(curdate() - birthdate) as duration, SmallInt( year(curdate() - birthdate)) as Years, SmallInt(month(curdate() - birthdate)) as Months, SmallInt( day(curdate() - birthdate)) as Days, SmallInt( days(curdate())- days(birthdate)) as Total_Days, SUBSTR (
'Jan. Feb. MarchAprilMay June July Aug. Sept.Oct. Nov. Dec. ',

month(birthdate) *5 -4, 5) as Month, case DayOfWeek(birthdate) when 1 then 'Sunday' when 2 then 'Monday' when 3 then 'Tuesday' when 4 then 'Wednesday' when 5 then 'Thursday' when 6 then 'Friday' when 7 then 'Saturday' else 'who knows?' end as Day_of_birth FROM birthday

DBT544 Security and Administration DB change issues

Record format level check . . . . . . . . . : LVLCHK *YES Specifies whether the levels of record format identifiers are checked to verify that the current record format identifier is the same as that specified in the program that opens the physical file. Programs compile the projection of a table/view into the program object. This is done for efficiency. Tables/files are changed infrequently whereas programs are run very frequently. When a program is run, the system checks its files are still at the same level. The system maintains a level identifier for a file and can detect file description changes. This is not just a version number. It is a code that represents the projection of the file. Changing text or column headings does not change the level id because projection is unaffected. Changing a column's data type/length or adding/dropping a column -- now that changes the projection and the level id. Run DSPFD filename TYPE(*RCDFMT) or DSPFFD to see the level id.
Display File Description (DSPFD filename TYPE(*RCDFMT)) Record Format Level Format Fields Length Identifier BIRTHDAY 2 20 3AD77522FCEFD after adding a new column: Record Format Fields Length BIRTHDAY 3 28 Format Level Identifier 3AC7111B2953B

Run Display Program References (DSPPGMREF) to see the level id compiled with a program.
Display Program References (DSPPGMREF) Record Format Format Level Identifier BIRTHDAY 3AD77522FCEFD Field Count 2

Running a program based on the original version of the table results in: CPF4131 - Level check on file name in library lib with member mbr.
drop the column and the level ID returns to its original value: Record Format Level

Format BIRTHDAY

Fields 2

Length 20

Identifier 3AD77522FCEFD

Program can be run again because levels match.

If a program uses a view, the level stays the same if changes to the based on table do not affect the view. e.g. adding a new column to the table. However, if you change the attributes of columns referenced in the view, a level check will occur. Remember, there is more system overhead when a program uses a view than a table. As a DB Administrator, how do you deal with all this? Build an output file of program references from all relevant libraries: DSPPGMREF PGM(library/*ALL) OUTPUT(*OUTFILE) OBJTYPE(*ALL) OUTFILE(QTEMP/DSPPGMREF) OUTMBR(*FIRST [*ADD,*REPLACE]) See DBT544TM2/DSPPGMREF Query/400 for a display of this output. Build an output file of file description information: DSPFD FILE(LIBRARY/*ALL) TYPE(*RCDFMT) OUTPUT(*OUTFILE) OUTFILE(QTEMP/DSPFD) OUTMBR(*FIRST [*ADD,*REPLACE]) See DBT544TM2/DSPFD Query/400 for a display of this output. See DBT544TM2/PGMFLEVEL Query/400 This query joins the program and file information. Review it to see the impact of changing a table or view. You could modify the query to select only those programs having unequal level IDs. Then, have the query output to a report for the programmers to review. The Application Development Manager/400 is an optional feature on the AS/400 which takes care of these issues (and many others including version control).
Security Built-in Object Level Security

IBM's Secure your database references AS/400 uses object level authority. Authority applies to all members in a File object. Objects exist in a library. A library is an object. Therefore, you need authority to both the object and the library it resides in. every object has an owner who has, by default, all authority to it (*ALL)

then, there is everyone else: *PUBLIC security defines what anyone besides the owner can do to an object Object Authorities govern what you can do with or to the object:
OS/400 Authority SQL privilege this means

Operational

assumed with row privileges

look at the object's attributes; use the object as per Data Authorities or SQL row privileges - most users need only Object Operational authority

to specify security, to move or rename the object, and to add members if the object is a database file, to clear a member. Management With Grant Option - needed for CRTDUPOBJ (! note PDM Object option 3. !) - in SQL, allows table description, index, and constraint definition to control the object's existence and ownership (e.g. Delete or Drop) Add, Clear, Reorganize DB file members; CHGPF to specify the object as the first level in a referential constraint

Existence

All

Alter

Index, Alter

Reference

References

Row privilege or Data Authority: Read data records, and/or Add, Update, Delete records. Users can be given any or all of these authorities to a data file.
OS/400 SQL this means Authority privilege Read Select look at the data create new records/rows - subject to Views With Check Option

Add

Insert

Update

Update

change existing records/rows - subject to view restrictions and/or column security - SQL only: combine Select rows with Update column(s) to restrict changes to certain columns only remove existing records/rows run a program

Delete Execute

Delete Execute

Views can be used to restrict access to a selection of rows and/or columns o With Check Option prevents insert/update to rows outside of selection o use casting on a column to prevent updating of that column: CREATE VIEW DBT544TM2/CUSTMASTV (CUST_NO, CSTNAM, CSTADR, CSTCTY, CSTSTP, CSTZIP, AR_BALANCE, REGION, CSTSLN) AS SELECT dec(cstnum,5,0), CSTNAM, CSTADR, CSTCTY, CSTSTP, CSTZIP, dec(YTDTOT,11,2), dec(CSTREG,3), CSTSLN FROM DBT544TM2/CUSTMAST WHERE cstreg = 10 Library/Collections o no SQL equivalents o Grant/Revoke/Edit Object Authority (GRT/RVK/EDT OBJAUT) o requires minimum *USE (i.e. *OBJOPR + *EXECUTE) o to create a new program, table, view, etc., also requires *ADD o use the QTEMP library for temporary tables, not the collection o other Data Authorities at the library level are not applicable Hierarchy of Authorization Checking

1. explicit object authority by User, i.e. User ID in list of private authorities * authority specified object by object, user by user * difficult to manage: most AS/400s have many users who access many objects (this becomes an N2 problem) * least efficient, most flexible 2. Authorization List (AUTL) * only one list per object * identifies a list of User IDs and their individual authority levels * useful when a collection of people need differing authorities to a number of

3. 4.

5.

6. 7.

objects * each object refers to the Authorization List name, e.g. OPSPRF * changing the AUTL object changes the users and authorities for every object that references the list * low efficiency, very good flexibility object or adopted owner's authority (high efficiency) Primary Group Profile * a user profile created only for the purpose of assigning regular user profiles to it * when the user is a member of a group profile, the user has authority to any object the group profile has been granted * not as flexible as AUTL since all members of a group profile get the same authority * high efficiency if a primary group is nominated for the object (CHGOBJPGP) *PUBLIC authority * applies to everyone else * highly efficient but least flexible user with all object (*ALLOBJ) special authority (i.e. a "super-user") Other Group Profiles * a user can belong to up to 16 group profiles (CHGUSRPRF) * all group profiles' authorities are added together * medium efficiency, similar flexibility to AUTL with combined group profiles

Objects can reference other objects for security checking. Security setup can be simple:

many users belong to one, two or a few group profiles an object specifies authorities for one, two or a few group profiles many objects reference one object with the same security

Even if you cannot change security, you can look at it: run DSPOBJAUT (Display Object Authority). Master & Audit Tables A common application is an Inventory Master table containing a single row per product and a Transaction Audit table containing rows of all the quantity changes that occurred during a day's business.

Inventory Master Quantity On Hand 5 30 26 <= start of day

Audit for Inv. Master & Order applications Purchase /Sales Order Order Quantity

Product ID

Product ID

ABC123 ABC123 ABC123

PO321 SO456

ABC123 ABC123

+25 -4

What minimum SQL privilege (OS/400 authority) should regular users have to these tables, rows, and columns?
privileges Inventory Master All? Alter? References? With Grant Option? Quantity On Hand Update? Audit for Inv. Master & Order applications All? Alter? References? With Grant Option? Purchase /Sales Order Update? Order Quantity Update?

Table

Product ID Column Update? Rows Select? Insert? Update? Delete?

Product ID

Update?

Program Adopted Authority

A common task at the end of the day is moving the daily Audit table data to a history file (SQL select & insert, delete) (OS/400 CPYF & CLRPFM). This end-of-day task requires more privileges than users need to process orders. Do you change users privileges at night? Do you create a super user who runs the day end job? OS/400 allows a program to adopt authority for the running of a program. Steps to adopt authority

create a program to do only those actions requiring additional authority this program should not call any other programs because called programs inherit authority create a user id (with password *NONE) give this user id minimal authority to the program and the object(s) the program acts upon make this user id the owner of the program (CHGOBJOWN cmd) and revoke the original owner's authority change the program to adopt the owner's authority (CHGPGM) Now, the programmer who created the program cannot run it. The users who are allowed to run the program adopt sufficient authority from the special purpose user id for the running of this program only. No one can sign on with the special purpose user id.

Testing things out... Sign on with the DB544A20 / A21 / A35 user ids. They all allow access to the DBT544TMS library. They allow various levels of access to the EMPLOYEE file. Try it out. Even if you cannot change security, you can look at it: run DSPOBJAUT (Display Object Authority). Look at the DBT544TMS library object and the security on the objects within. You can also Display User Profile (DSPUSRPRF) to see what group profiles your user id belongs to. Run the EMPDATACPY and EMPDATACPX programs in the DBT544TMS library to see how adopted authority is used. the source is in QSLPSRC. If you have signed on with a user id that allows you to change or deleted rows, be sure to run the program that refreshes the Employee file.

One thing you cannot do with OS/400 security is restrict a user to only explicitly authorized objects. Every user has *PUBLIC authority. You cannot revoke *PUBLIC authority from a user (they would not be able to run any operating system commands). See the textbook's section on "Database Security Principles and Guidelines". It is an excellent practical summary. Data Base Connectivity

Operations Navigator o GUI interface to the iSeries o browse the different File Systems on the iSeries o grant/revoke permissions (authority) to objects o Basic Operations lets you: Work with messages Work with printer output, ilel output queues (and print your spool files on your local PC printer) o System management, configuration and backup tasks o easy access to File Systems, IFS (Integrated File System) The integrated file system is a part of OS/400 that supports stream input/output and storage management similar to personal computer and UNIX** operating systems, and provides a similar hierarchical folder structure. ASCII (American Standard Code for Information Interchange) stream files like in DOS, OS/2 and basic UNIX systems /[root] your_directories drag and drop files between Windows Explorer and the IFS (avoid FTP) UNIX standard file system, POSIX and XPG compliant /QOpenSys/directories [case sensitive names] native EBCDIC (Extended Binary Coded Decimal Interchange Code) database and library file system is a branch of the IFS /QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR OS/400 File Server to remote iSeries servers (/QFileSvr.400), Sun's NFS (/NFS), Novell's Netware (/QNetWare), Windows NT Server (/QNTC), and optical (/QOPT) file systems are also directly supported. limit the files that are shown for any file system or folder by rightclicking it and then clicking Include. In the Include dialog box, you can specify which files you want to see (for example, specify Dx544Ann* to see only your folders off the /root). FTP using native or IFS naming formats

WS_FTP opens in native format at your User Profile's current library Change Dir to anything beginning with a "/" and you are in the IFS Root system (you cannot get back to native naming format) o iSeries FTP server will automatically recognize the naming format you send it o iSeries FTP client can use either format: native naming format (iSeries FTP cmd is NAmefmt 0 ): librname/filename.mbrname IFS naming format (iSeries FTP cmd is NAmefmt 1 ): /your_directory/filename or /QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR o use AScii transfer mode when exchanging data with the iSeries native or IFS iSeries will convert between ASCII and EBCDIC representations iSeries will not change the end of line or record delimiting characters o use Binary transfer mode when exchanging ASCII data amoung differing ASCII operating systems this avoids the problem of the differing "end of line" or record delimiting characters used by various operating systems. On other systems, AScii transfer mode helps by changing those characters on the receiving system. If it is always transferred in binary mode, you will have no problems with record delimiters. End of line character(s) are: MSDOS: CR/LF (Carriage Return and Line Feed) Mac: CR Unix: LF or newline (same thing) What if the file was transfered in a combination of Binary and AScii modes between many systems before landing in your IFS? Try this: FTP a text file from MSDOS to Unix using Binary (preserving CR/LF) then back to MSDOS using AScii mode -you will have record delimiters of CR/CR/LF. FTP "knows" Unix does not use CR and the destination MSDOS system does, so it added a CR. (This will cause problems for the CPYFRMIMPF command.) ODBC (Open Data Base Connectivity) driver comes with Client Access/400 o generic database access from any client o use with many client programming products, e.g. Visual Basic o use with desktop and decision support products, e.g. Excel, Access o Excel: Data \ Get External Data \ New DB Query \ Employee file o connection type should be Read Only or Read/Call Up/download File transfer utility in Client Access/400
o

o o o

creates a PC file with DB definition for upload use transfer definitions can be saved and run by PC batch/script files See terminal menu: Transfer

GO DATA menu to manage Files, Libraries, Folders, Client Access tasks, and the Integrated File System. GO FILESYS for Integrated File System commands to move data between the native database and PC/UNIX data files in the IFS IFS and Stream Files

use Work Link (WRKLNK) to see directories and files in the IFS use Display File (DSPF) to display a stream file use Edit File (EDTF) to edit a stream file Copy To/From Stream File (CPYTOSTMF or CPYFRMSTMF) o used to transfer 'flat files' between AS/400 native and IFS o cannot deal with packed numerics o function is very similar to FTP using AScii transfer where the iSeries automatically translates between EBCDIC/ASCII character sets. o AS/400 native DB naming: The FROMMBR database-member-pathname must be of the format: /QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR o e.g. to transfer a spool file: CRTPF FILE(SPOOLFILE) RCDLEN(132) CPYSPLF FILE(QSYSPRT) TOFILE(SPOOLFILE) CPYTOSTMF FROMMBR('/qsys.lib/dbt544tm.lib/spoolfile.file/spoolfile.mbr') TOSTMF('/dbt544/spoolfile.txt') you can now transfer the /dbt544/spoolfile.txt to your PC using FTP or Operations Navigator using drag and drop or the iSeries may serve those files directly from the IFS to other systems Copy From Import File (CPYFRMIMPF) o copy a stream file in CSV format to an externally described database file o useful command options: o Replace or add records . . . > *ADD, *REPLACE, *UPDADD *UPDADD = The system updates the duplicate key records and adds the new records to the end of the existing records. (Make sure your DB file has a primary key.) o Record delimiter . . . > *ALL *CRLF *LF *CR *LFCR *EOR *ALL = Any single or double character combination of carriage-return and line-feed.

Copy from record number . . . > Number, *FIRST specify 2 as the number to avoid copying a header record with column names o Replace null values . . . > *NO, *FLDDFT *FLDDFT = If a null value is detected when parsing an import file record, the corresponding field in the database file record is assigned a default value based on the field type or DDS/SQL defined default value. Copy To Import File (CPYTOIMPF) o copy an externally described database file to a stream file in CSV (Comma Separated Variable) format, also known as quote/comma delimited data. o e.g. CPYTOIMPF FROMFILE(CUSTOMER) TOSTMF('/dbt544/customer.csv') MBROPT(*REPLACE) RCDDLM(*CRLF) o Note the Stream file code page (STMFCODPAG) parameter... set it for *PCASCII so the characters are translated from EBCDIC to ASCII o Note the Record delimiter (RCDDLM) parameter...set it for the end of line character(s) in the destination system: MSDOS: *CRLF (Carriage Return and Line Feed) Mac: *CR Unix: *LF for more information, see IFS & Copying data using CL commands
o

Query Manager/400

part of SQL Development Kit licensed program similar to Query/400 with more flexibility but a little less functionality specify a result table with an SQL Select statement. This query is stored as a *QMQRY OS/400 object type. specify a report format for the result table. This form is stored as a *QMFORM OS/400 object type. you can mix and match queries and forms. SQL definitions allow variable names which can be passed as runtime prompted parameters &VARIABLE & flags it as a variable, the NAME should be in upper case outputs to display, printer, or a file. STRQM Starts the Query Manager two Query creation modes are available: o Prompted for end users and programmers developing a query o SQL for programmers and when specifying variables

use GO CMDQM to see all the Query Management commands. A query that prompts for a variable is not truly user friendly * users must know to input character values in single quotes * users must know how to use wildcard ( % _ ) characters in the case of LIKE comparisons A professional implementation uses: query definition with variable(s)
SELECT USERID, STNO, LAB1, LAB2, LAB3, LAB4, LAB5, LAB6, TEST/15*35 as Mid_term, (lab1 + lab2 + lab3 + lab4 + lab5 + lab6 + (TEST/15*35)) as Total FROM MARKS where userid = &USERID or test >= &TESTGE

"MARKS" command to prompt the user for values (just like CL prompt screens)
CMD PROMPT('Check Student Marks (MARKS)') PARM KWD(USERID) TYPE(*CHAR) LEN(10) MIN(1) + PROMPT('AS/400 User ID') PARM KWD(TESTGE) TYPE(*CHAR) LEN(9) MIN(1) + PROMPT('or Test Mark is GE to')

"MARKSCL" CL program which does the processing for the command

/* /* /* /* /* */ /* /* /* This is the processing program for the MARKS command. When compiling the MARKS command, specify the library name of this processing program (so command can be run from anywhere (or ensure the processing program is always in the users' *LIBL) CRTCMD CMD(DBT544TM/MARKS) PGM(DBT544TM/MARKSCL) ... ******** PROGRAM SHOULD BE RUN ADOPTING OWNER'S AUTHORITY, */ I.E. MARKS FILE ACCESSABLE ONLY VIA THIS INTERFACE */ PGM DCL DCL DCL PARM(&USERID &TESTGE ) /* from MARKS command */ VAR(&USERID ) TYPE(*CHAR) VAR(&USERID2) TYPE(*CHAR) VAR(&TESTGE ) TYPE(*CHAR) receives parms LEN(10) LEN(12) LEN(9) */ */ */ */ */

/* must put USERID with quotes into a NEW field */ CHGVAR &USERID2 VALUE('''' *CAT &USERID *TCAT '''') /* run Query Manager query passing parameters for WHERE clause */ STRQMQRY QMQRY(DBT544TM/MARKSS) + QMFORM(DBT544TM/MARKS) + SETVAR((USERID &USERID2 ) + (TESTGE &TESTGE ))

ENDPGM

see DBT544TM/QCLPSRC.MARKS cmd and MARKSCL clp see DBT544TM/MARKS* Query Manager query and form

A professional implementation also includes an automated upload and import of the MARKS data. This example uses FTP to transfer a PC Comma Separated Variable file to the native DB side of the iSeries. PC Batch file to start FTP:
@echo off echo FTP marks from c:\DBT544\dbt544amrk.csv pause c: cd "\DBT544" ftp -n -s:"ftpDBT544marks.txt" ODIN

The FTP "ftpDBT544marks.txt" command file:

USER DB544Ann password AScii delete mckenna/dbt544amrk put dbt544amrk.csv QUOte RCMD CPYFRMIMPF FROMFILE(MCKENNA/DBT544AMRK) TOFILE(DBT544TM2/MARKS) MBROPT(*UPDADD) RCDDLM(*ALL) RPLNULLVAL(*FLDDFT) quit

Object and Row Locking

...because everyone should share their toys and play nicely together. Please see Chap.13 p.239-253 in the textbook. Object Lock

application based locking ensures specific types of access to objects for your job o e.g. allocate a table for mass update or to clear all data prevents other jobs from having inappropriate access to your job's objects o e.g. no updates allowed during month end processing permit only one job at a time to execute a program o e.g. only one month end process o see QCLPSRC/ONEatAtime o test with CALL ONEATATIME ONCE

OS/400 native: ALCOBJ and DLCOBJ Allocate/Deallocate Object o run these sample CL programs in DBT544TM... o CALL LOCK1 on one screen and CALL LOCK2 on another o LOCK3 is the way to allocate & deallocate a number of files separately o LOCK4 is the way to allocate & deallocate a set of files together SQL: LOCK TABLE statement o released with COMMIT or ROLLBACK

DSPJOB OPTION(*JOBLCK) - (or SysReq #3, #12) to display objects and rows that are locked and the name of the job holding the lock. DSPJOB OPTION(*CMTCTL) - to show the isolation level that the program is running, the number of rows being locked during a transaction, and the pending DDL functions. WRKOBJLCK - work object locks to find out who has a lock on an object ==> you need *JOBCTL special authority in your user profile to take any action on other jobs. To avoid deadlock conditions where JobA has allocated FileA and is waiting to allocate FileB and JobB has allocated FileB and is waiting to allocate FileA establish a File Access Sequence. A File Access Sequence is a sequential approach to allocating tables/files (or rows/records within) that all programmers and database people agree to follow, or else. If everyone allocates files in the same sequence, deadlock conditions cannot occur.

typical File Access Sequence contain system wide control values e.g. next CustomerID number, next Order Number

Control files

Audit files

contain details of the transaction that is about to take place

e.g. order data: WAREHOUSE, SKU, QUANTITY, PRICE, in sequence by foreign key dependency e.g. Customer, Customer-Phone, Customer-Accounts/Receivable e.g. Item (by SKU), Inventory (by Warehouse, SKU) contain posted transactions e.g. Order Header, Order Details

Master files

Transaction files

Types of Object Locks native ALCOBJ *Excl *ExclRd *ShrUpd (share update) SQL Lock Table name In Exclusive Mode

allocating job

other jobs

exclusive

no access read only

Exclusive Mode Allow Read exclusive update (default without a Lock Table statement)

read/update

read/update

*ShrNup Share Mode (share no update) *ShrRd (share read) (default without a Lock Table statement)

read (and update if read only no other job has *ShrNup)

read

read/update

Combination Locks other jobs can allocate?

ALCOBJ *Excl Your job's allocation *ExclRd *ShrUpd *ShrNup *ShrRd

*Excl no no no no no

*ExclRd no no no no YES

*ShrUpd *ShrNup no no YES no YES no no no YES YES

*ShrRd no YES YES YES YES

Row Locks

transaction based locking system controls access to individual rows within a table so two applications cannot update the same record at the same time works like *ExclRd on a row system locks row at start of I/O (read for update) system releases row at end of I/O (update) or, if no update, at start of next I/O use CHGPF to adjust the "Maximum record wait time", i.e. system waits X seconds for another application to update a locked row (then issues error to your job); or *IMMED for impatient applications (immediate error); or *NOMAX for users with the patience of Job or applications that cannot perform error handling COBOL and RPG programs typically work with one record per file at a time e.g. In an order processing application, a Control table row is locked for I/O to obtain the next Order Number and to increment/update the value for the next time.

with commitment control, row locking occurs on a set of rows SQL often works with a set of rows at a time. e.g.

Object vs. Row Locking and Commitment Control

Lock Table Customer Exclusive Mode Allow Read Update Customer Set Credit_Limit = Credit_Limit * 1.05 Where CR_Rating = 'GOOD' Commit The above object locking prevents anyone else from modifying any and all rows in the table. However, this may be too restrictive. If the table is in constant use by other applications, your SQL statement may take all day to obtain the object lock. If you do get the lock, no one else may be able to do any work until your job completes. SET TRANSACTION ISOLATION LEVEL NO COMMIT Update Customer Set Credit_Limit = Credit_Limit * 1.05 Where CR_Rating = 'GOOD' Without the Lock Table statement or commitment control, what happens if SQL gets half way through the file then tries to allocate a row that another application has locked and is taking a long time to update? This can easily happen if a user calls up a customer record with DFU, then walks away from the screen without pressing Enter. In the above scenario, some of the records were updated, then your SQL statement failed. After you use WRKOBJLCK to track down the DFU user, how can your SQL statement pick up where it left off? Well, it can't. You are in trouble, and looking for backup tapes, unless you use commitment control. SET TRANSACTION ISOLATION LEVEL READ COMMITTED Update Customer Set Credit_Limit = Credit_Limit * 1.05 Where CR_Rating = 'GOOD' Commit Using row locking and commitment control, the system shares the rows in the table with other applications but allows only one at a time to update a given row or a set of rows if under commitment control. In this SQL statement, the changes are preserved when Commit is done; other applications can now update the locked rows. If something goes wrong, Rollback can be performed which reverses this statement's updates to the table rows.
Commitment Control and Isolation Levels Isolation Level your SQL job data from other SQL jobs

*NONE or *NC or NO COMMIT

no control (Lock Table may be used to ensure integrity at the object level) changed objects with DDL or changed rows with DML are locked until Commit/Rollback * Rows are not locked during update process.

Locked rows and uncommitted changes in other jobs can be seen. These are "dirty reads" which may be rolled back.

*CHG or *UR or READ UNCOMMITTED, READ WRITE

allows "dirty reads"

*CS (Cursor Stability) or READ COMMITTED

changed objects with DDL or changed rows with DML are locked until Commit/Rollback * Rows are locked during update process.

Only committed data can be seen. Data is read on a row by row basis. Your nested Select statements may not be congruent with the outer Select statements. Some rows were not committed by the other job and the updates not read by your nested Select, however the changes were commited by the time the outer Select ran. Only committed data can be seen. Any data your unit of work read or updated cannot be changed by other jobs until your unit of work is committed.

*ALL or *RS (Read Stability) or REPEATABLE READ

as above

Your nested Select statements may not be congruent with the outer Select statements. New rows can be inserted by other jobsat any time and, if committed, read by your job.

*RR (Repeatable Read) or SERIALIZABLE

as above with object locks of *ShrNup on read-only tables and *ExclRd on updated tables.

Only committed data can be seen. No changes are allowed by other jobs to tables in your work unit.

In general, use Read Committed or Repeatable Read if your work unit re-reads rows in the same transaction. See the IBM Manual for more information on Isolation Levels

Use the COMMIT parameter on the CRTSQLxxx, STRSQL, and RUNSQLSTM commands to specify the default isolation level. Use the SET OPTION statement to specify the default isolation level within the source of a module or program that contains embedded SQL. Use the SET TRANSACTION statement to override the default isolation level within a unit of work. When the unit of work ends, the isolation level returns to the value it had at the beginning of the unit of work. Do not use this statement inside an SPL procedure it will conflict with the caller's commitment control. Use the isolation-clause on the SELECT, SELECT INTO, INSERT, UPDATE, DELETE, and DECLARE CURSOR statements to override the default isolation level for a specific statement or cursor. The isolation level is in effect only for the execution of the statement containing the isolation-clause and has no effect on any pending changes in the current unit of work.

Functions and Procedures

See examples in DBT544TM/QSQLSRC. Note: if a function is called with mismatching parameters, the system issues this confusing error message:
function_name in *LIBL type *N not found.

This is because the system looks for a function according to its name and its parameters. The parameters form a signature similar to a file's level check which is based on the file's projection. That is, the number, type and length of all parameters (or fields) must match.
DBT544 Backup

Save Library (SAVLIB) normally backs up to a tape device but it can also backup to disk using a Save File (SAVF). The AS/400 provides for a Save File object to contain a backup. To create the *SAVF object:
CRTSAVF SaveFileName

Save all objects in your library using the following command:

SAVOBJ OBJ(*ALL) LIB(yourLibraryName) DEV(*SAVF) SAVF(SaveFileName)

You can FTP the Save Fileusing Binary transferto any other system. e.g. your PC. Retain the AS/400 file naming. To reload the file on the same, or even another, system... Login to the target AS/400 (or iSeries) machine. Create a Save File with the original name:
CRTSAVF SaveFileName

FTP your Save File to the LIBRARY in the target system containing the empty Save File . *please make sure you are transferring files in binary mode* Once you have transferred the file, use the following command to restore them. Prompt this command to override the library where the saved objects will be restored.
RSTOBJ OBJ(*ALL) SAVLIB(yourlibraryname) DEV(*SAVF)

DB Overrides and Redirection

Procedures and Functions can be found in your library list if ISQL is using *SYS Naming Convention. However, any unqualified table names in those Procedures and Functions are looked for in the default collection in force at the time the Create Procedure/Function statement was run. The EMPRESET procedure in DBT544TM updates a table called EMPLOYEE and was created with this statement: ===> RUNSQLSTM SRCFILE(DBT544TM/QSQLSRC) SRCMBR(EMPRESET) COMMIT(*CHG) NAMING(* SYS) DFTRDBCOL(DBT544TM) If you need to run this procedure against an EMPLOYEE table in a collection other than DBT544TM, you could recreate the procedure specifying another collection. However, this gives us a maintenance problem if the procedure ever needed changing and creates unnecessary objects on the system. Use the OS/400's Override with Data Base File (OVRDBF) command to temporarily use a different library and/or table name than was specified in your procedure, or QM query, or COBOL program...

OVRDBF FILE(EMPLOYEE) TOFILE(DBT544TM2/EMPLOYEE) STRSQL call empreset You are telling the system that, following OVRDBF, any file/table named EMPLOYEE will be referred to DBT544TM2/EMPLOYEE. It is up to the programmer to ensure that the overriding file/table has the same projection as the overridden file/table. The OVRDBF cmd is used frequently in CL programming. It is also used to change *FILE object properties temporarily. e.g. OVRDBF FILE(EMPLOYEE) FRCRATIO(1) WAITFILE(*IMMED) WAITRCD(*IMMED) The OVRDBF cmd is used prior to running SQL statements when you need to redirect SQL to use anything but the first member in a table. Normally, SQL creates only single member tables. Most DB designers follow this practice. Sometimes the rule may be broken or you need to use SQL to access a file with multiple members (such as a PF-SRC). Try using a combination of OVRDBF and SQL to list the contents of different source members in one of your source files. Always DLTOVR as soon as you are done with the override. Use DSPOVR to display overrides still in effect.
Query Optimization

To find out what the system is doing and what you could do better, use the STRDBG command before you run a query with Query/400 (RUNQRY), or Query Manager (STRQMQRY), or SQL. In the job log (DSPJOBLOG or use the SysReq key to get to it), you will find this msg:
**** Starting optimizer debug message for query .

followed by a series of messages telling you what the system did and how you might improve performance. Remember to run ENDDBG after you are done.

DBT544 Exam notes

Format is similar to the Mid-Term test:

create a collection run a native command which will: o create tables in your collection o add objects to the collection o create a source member in the collection for your answers

Know how to

browse an IFS directory display an IFS file recognize delimiters for a string and field separators use 'import file' commands to transfer data TO and FROM CSV and database files Write a Query Manager Query with user prompted variables. You may have to create a QM form to use with the QM Query. use a command to run the QM Query (with the form you created) passing the user variables o Note: you will not have to create a CL program and CMD to run a QM query use DB overrides (redirection) use commitment control DB Integrity (object and row locking): understand the different ways to enforce appropriate sharing/allocation of files/tables. There are native and SQL ways. Functions and Procedure coding: I will supply a shell, you read the comments and complete/debug the code.