MC Press Online

RPG's Persistent User Spaces

Contributed by Paul Tuohy Tuesday, 04 December 2007

Stop worrying about minimizing the overhead of a program. Sometimes it's better to keep a lot of information in memory.

At the recent RPG & DB2 Summit conference, a few of us were chatting about the way our programming styles had changed over the years and what (apart from experience and learning the hard way) had influenced those changes. A lot of the usual suspects—such as mixed case, longer field names, and free format—were discussed. But, for me, one of the items that caused a major shift in the way I program was user spaces.

In order to make proper use of user spaces, you need to change your traditional approach to writing programs. Traditional RPG programmers are accustomed to minimizing the overhead of a program; we've spent a lot of time ensuring that a program utilizes only the memory it actually needs (e.g., how often have you wondered how many elements to assign to an array?).

But the "modernized" RPG programmer takes a different view of minimizing the overhead. Memory is cheap, and the system and programs are now so fast that there is little or nothing to be gained from the minimalist approach. Dare I say it, but it is a PC style of programming: load the data into memory, manipulate it, and save it. Think of how your favorite spreadsheet or word processor program works.

This approach is also reflected in how RPG has changed to accommodate larger field sizes: 256 to 64K for character fields and 999 to 32,767 for the number of elements in an array. And wait until you see what happens with these limits in V6R1!

As with most RPG programmers, I first came across user spaces when using the list APIs. I soon realized that user spaces also had uses in applications and could prove invaluable in a Web environment that was dependent on CGI programs.

Basically, using user spaces taught me that not everything needs to be database based and at times it is better to keep a lot of information in memory.

In this article, I offer a quick review of user spaces and explain how to create and manage them. I will also show how you can use user spaces to handle the problem of persistence in a CGI environment. User Spaces

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27

MC Press Online

A user space is an object (object type *USRSPC) that consists of bytes, the format and content of which are up to you. More precisely (even though it might sound extremely vague), a user space is whatever you want it to be. A user space is "loaded" into memory and is accessed in a program by using pointers and based variables; it is simply a stream of bytes that you can access directly from within a program.

The nearest that traditional programming can come to this concept is a data area, but you have to input and output data areas to and from programs. In contrast, you can immediately change the content of a user space by merely changing the value of a field in your program.

Also, a user space will only be "loaded" in memory once, which means that multiple programs (whether in the same job or multiple jobs) will share the same instance of the user space. This means that multiple programs in different programs are sharing the same portion of memory. So, when program PGMA in job JOBA changes the value of a field, it is actually changing the value of a field in PGMB running in JOBB, without PGMB having to do any I/O operations. The fact that a user space may be common to programs in multiple jobs at the same time (without the programs having to perform any I/O) is a feature that proves very useful in CGI programming.

A user space has a maximum size of 16 megabytes. To create a user space or manipulate its contents, you must use one of the system-supplied APIs, and the only system-supplied command for user spaces is Delete User Space (DLTUSRSPC).

Although user spaces have been on the system for a long, long time, it was only with the advent of pointers in RPG IV that they became a lot easier to use and a lot more beneficial. The Persistence Problem

One of the issues with CGI is that the connection between the browser and the called program is not persistent, i.e., a request is sent from a browser to a program in a CGI job, and the program returns an HTML document to the browser; the connection is now "broken," and the next request to the CGI job could come from a different browser.

On the same basis, when a request is sent from a browser to a program in a CGI job, the program returns an HTML document, and when the browser makes a subsequent request, there is no guarantee that the request will go to the same CGI job.

If there is any data that must be maintained for a browser, a program must have some way of storing it for each requesting browser session and ensuring that the data is available to all CGI jobs. Although this problem could be solved using database files, there would be the requirement for CGI programs to constantly read, write, update, and delete records. User spaces provide a more efficient means of managing "persistent" information without the need for any I/O.

Let's look at an example.

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27

MC Press Online

At System i Developer, there is a small application we use to maintain information for planning our RPG & DB2 Summit conferences. This application is Web-based and is written using CGIDEV2 (details available at easy400.net).

All of the maintenance programs use externalized database processing subprocedures for database maintenance. The basic concept is that the client programs (CGIDEV2 programs in this example) have no idea of the format of the actual database. All access to the data is through getters and setters. If you are not familiar with the concept of externalizing a database, you can get your hands on some sample code here. (These examples do not have the user space processing applied, but the following information should make it easy to implement the required changes if you wish).

Here's the basic format for external processing (speaker data in this example):

NewSpeaker(Speaker:Id); GetSpeakerData(Id:SpeakerData); SetSpeakerData(Id:SpeakerData); ReleaseSpeaker(Id);

The important parameter in all of these subprocedures is the Id. The module containing the database subprocedures contains two arrays: an array of database records it is currently "handling" and an array of single-character fields used to identify the next available record element. The NewSpeaker subprocedure returns the index (Id) where the requested database record (Speaker is the key) is being stored. The Id is then used as a parameter to any Get or Set subprocedure to identify the record to be processed. The client program calls the Release subprocedure when it is finished processing the data.

In order to incorporate this design into a CGI environment, the two arrays (handles and database records) must be common (or available) to any CGI jobs that might end up processing a request from a browser. If the two arrays are placed in a user space, they are both "common" to programs in all CGI jobs that need to access them; all the programs need is the Id to identify which element to process. Therefore, all the browser page has to return is the Id, which is easily achieved using a hidden field in a form. Creating and Managing User Spaces

Since a user space will be required for each table (physical file) in my application, I decided to write a subprocedure that creates a user space (if it does not already exist), loads the user space into memory, and returns a pointer with the location of the user space. This pointer may then be used as the basing pointer for a based variable. Figure 1 shows the prototype for a subprocedure named GetStorageSpace.

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27

MC Press Online

D GetStorageSpace... D PR ExtProc('GetStorageSpace') 10a Const * 10i 0 Const Options(*NoPass)

D SpaceObj D SpacePtr D SetSize D

Figure 1: This is the prototype for the GetStorageSpace subprocedure.

The GetStorageSpace subprocedure has three parameters:

- SpaceObj is the name of the user space. In this example, I am using a predefined library to contain the user spaces. - SpacePtr is the pointer that indicates the location of the user space in memory. This pointer will be used as a basing pointer for one of the two arrays. - SetSize is an optional parameter that indicates the initial size of the user space (all user spaces are set so they will automatically extend).

The information that will be placed in the user space must be based on pointers. Figure 2 shows the definition of the two arrays (mentioned earlier) that will be mapped to the user space used for the Speaker database (Speakers is the record format on the Speaker table).

D StoreRecord
http://www.mcpressonline.com

DS

LikeRec(Speakers)
Powered by Joomla! Generated: 21 December, 2008, 23:27

MC Press Online

D D

Dim(32767) Based(PtrStoreRecord)

D Handles D

1A Dim(32767) Based(PtrHandles)

D PtrStoreRecord S D PtrHandles D HaveStorage S S *

Figure 2: These based arrays will be mapped to the user space.

Figure 3 shows the definition of the CheckStorage subprocedure. This subprocedure calls the GetStorageSpace subprocedure, providing the name of the user space (STSPEAKERS) and the initial size (8M). The returned pointer (PtrHandles) is the basing pointer for the Handles array. The length of the Handles array is added to PtrHandles to calculate the value of the basing pointer (PtrStoreRecord) for the record array. This means that the first 32,767 bytes of the user space will contain the Handles array followed by the record array.

CheckStorage is executed as the first line in the NewSpeaker subprocedure.

P CheckStorage D CheckStorage
http://www.mcpressonline.com

B PI
Powered by Joomla! Generated: 21 December, 2008, 23:27

MC Press Online

/Free If Not HaveStorage; GetStorageSpace('STSPEAKERS' : PtrHandles : 8000000); PtrStoreRecord = PtrHandles + %Elem(Handles); HaveStorage = *On; EndIf; /End-Free P E

Figure 3: Set the basing pointers. The User Space APIs

The GetStorageSpace subprocedure is going to make use of the Create User Space (QUSCRTUS), Change User Space Attributes (QUSCUSAT), and Retrieve Pointer to User Space (QUSPTRUS) APIs. Figure 4 shows the corresponding prototypes, along with the definition of the SpaceAttribute data structure, which is used as a parameter to the QUSCUSAT API:

D CreateUserSpace... D PR ExtPgm('QUSCRTUS') 20A Const 10A Const 10I 0 Const 1A Const 10A Const 50A Const
Powered by Joomla! Generated: 21 December, 2008, 23:27

D UserSpaceName D Attribute D Size D Initial D Authority D Text

http://www.mcpressonline.com

MC Press Online

// Optional Parameter Group 1 D Replace D ErrorCode D 10A Const Options(*NOPASS) Options(*NOPASS) LikeDS(APIError)

// Optional Parameter Group 2 D Domain 10A Const Options(*NOPASS)

// Optional Parameter Group 3 D TransferSize D OptimumAlign 10I 0 Const Options(*NOPASS) 1A Const Options(*NOPASS)

D ChangeUserSpaceAttributes... D PR ExtPgm('QUSCUSAT') 10A 20A Const Const LikeDS(SpaceAttribute) LikeDS(APIError)

D ReturnLibrary D UserSpaceName D Attribute D D ErrorCode

D GetUserSpace

PR

ExtPgm('QUSPTRUS') 20A Const * Options(*NOPASS) LikeDS(APIError)

D UserSpaceName D pSpacePtr D ErrorCode D

D SpaceAttribute DS D NumberOfRecs D ExtendRecord D Key D Length D Extend

Qualified 10I 0 12A 10I 0 Overlay(ExtendRecord) 10I 0 OverLay(ExtendRecord:*Next) 1A OverLay(ExtendRecord:*Next)

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27

MC Press Online

Figure 4: Here's the prototype for the QUSCRTUS, QUSCUSAT, and QUSPTRUS APIs.

The QUSCRTUS API consists of one mandatory parameter group and two optional parameter groups (remember, if one parameter in an optional parameter group is defined, then all parameters in the optional parameter group must be defined). These are the parameters for the QUSPTRUS API:

- UserSpaceName is the name of the user space; the first 10 characters are the object name, and the last 10 are the library name. Remember that object names and library names should be uppercase. - Attribute can be any valid name (it is the object attribute). - Size is the size of the user space in bytes. This can range from 1 to 16,776,704. - Initial is the initialization value for all bytes in the user space. The API documentation recommends that this is set to X'00'. - Authority is the public authority for the object (*ALL, *CHANGE, *EXCLUDE, *LIBCRTAUT, *USE, or the name of an authorization list). - Text is the text description for the object. - Replace indicates whether or not (*YES or *NO, with *NO being the default) you should replace the user space, if it exists. - ErrorCode is the standard error data structure used for APIs. - Domain indicates whether the user space should be placed in the system or the user domain (*SYSTEM or *USER). The default value of *DEFAULT means that the system decides on the domain based on the value of the QALWUSRDMN system value. - TransferSize is the number of pages to be transferred between main storage and auxiliary storage; it may range from 0 to 32. The default value of 0 indicates that the system determines the transfer size. - OptimumAlign indicates whether or not (1 or 0, the default) optimum space alignment is performed for the user space, i.e., the user space is aligned in memory based on the size of a disk page.

These are the parameters for the QUSCUSAT API:

- ReturnLibrary is the name of the library that contains the changed user space object. If the space attributes are
http://www.mcpressonline.com Powered by Joomla! Generated: 21 December, 2008, 23:27

MC Press Online

successfully changed, the name of the library in which the user space was found is returned. - UserSpaceName is the name of the user space; the first 10 characters are the object name, and the last 10 are the library name. - Attribute is a special data structure (described next) containing details of which attributes are to be changed. - ErrorCode is the standard error data structure used for APIs.

There are four user space attributes that may be changed: the size of the user space, the initial value, automatic extendibility (the one we are interested in), and the transfer size request. The attribute data structure (SpaceAttribute in Figure 4) contains the following information:

- NumberOfRecs indicates the number of attribute change request records contained in the data structure (1 to 4). - ExtendRecord contains variable-length records defining each attribute change request. In this example, we are only interested in making a user space automatically extendible, so ExtendRecord is re-mapped as Key (which will be initialized to 3 for automatically extendible), Length (which will be initialized to 1), and Extend (which will be initialized to '1').

These are the parameters for the QUSPTRUS API:

- UserSpaceName is the name of the user space; the first 10 characters are the object name, and the last 10 are the library. - pSpacePtr will contain the address of the user space in memory. - ErrorCode is the standard error data structure used for APIs. The GetStorageSpace Subprocedure

Figure 5 shows the GetStorageSpace subprocedure.

P GetStorageSpace... P B Export

D GetStorageSpace... D PI 10a Const

Powered by Joomla! Generated: 21 December, 2008, 23:27

D SpaceObj
http://www.mcpressonline.com

MC Press Online

D SpacePtr D SetSize D

* 10i 0 Const Options(*NoPass)

D FullSpaceName S D Library D Size S S 10a

20a

10i 0 Inz(10000000)

/Free (1) If %Parms() > 2; Size = SetSize; EndIf; (2) FullSpaceName = SpaceObj + 'SPACELIB';

(3) (4)

GetUserSpace(FullSpaceName : SpacePtr: APIError); If (APIError.BytesAvail > 0) And (APIError.MsgId = 'CPF9801');

(5)

CreateUserSpace( FullSpacename : 'STORERECS' : Size : x'00' : '*ALL' : 'User Space for Record Storage' : '*YES' : APIError);

(6)

SpaceAttribute.NumberOfRecs = 1; SpaceAttribute.Key = 3; SpaceAttribute.Length = 1; SpaceAttribute.Extend = '1'; ChangeUserSpaceAttributes( Library : FullSpaceName : SpaceAttribute

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27

MC Press Online

: APIError); (7) GetUserSpace( FullSpacename : SpacePtr : APIError); EndIf;

Return; /End-Free P E

Figure 5: This is the GetStorageSpace subprocedure.

The main points to note are these (refer to the corresponding numbers in Figure 5):

- If a value is passed for the third parameter, it is used to set the initial size. Otherwise, the initial size defaults to 10M. - The name of the library is appended to the name of the user space. - The QUSPTRUS API is called to retrieve a pointer to the user space. The API error data structure is checked to see if the call to QUSPTRUS failed because the user space does not exist.

The following points apply only if the user space does not exist.

- The QUSCRTUS API is called to create the user space. - Values are set in the SpaceAttribute data structure to indicate that the user space is automatically extendible, and the QUSCUSAT is called to change the attribute of the user space. - The QUSPTRUS is called again to retrieve the pointer to the newly created user space.
http://www.mcpressonline.com Powered by Joomla! Generated: 21 December, 2008, 23:27

MC Press Online

Some Other Thoughts

This example showed a user space being used to make database data persistent across multiple CGI jobs. But it could be any data. A user space could be used to store information from multiple pages as a user enters information (think of the process of booking a flight online). The information is stored in a data structure array, and each requesting browser is assigned a sequence number that corresponds to an element of the data structure array. The sequence number is written to the browser and is returned as a parameter, as with a hidden field on a display file format. When a program in a CGI job receives a request from a browser, it uses the returned sequence number to identify the element containing the required persistent data.

Be very careful about assigning sequence numbers. What happens if a user closes the browser and reopens it or returns to a starting page? You may want to consider separate subprocedures for assigning a sequence number and checking the validity of a sequence number when a request is received; assignment and validity would be based on how long it has been (and that length of time is up to you) since a request was processed for a specific sequence number, which would allow for re-use of sequence numbers.

Having the data structure array mapped to a user space can enhance the process in a number of ways. Firstly, you can exceed the 32,767 element maximum; assuming that the user space can physically store more than 32,767 elements, you can simply change the basing pointer of the data structure array to have it start in a different portion of the user space (or even use a single data structure and use the starting pointer, the size of the data structure, and the sequence number to calculate where in the user space the data structure should be mapped). Secondly, during development and testing, having the data in a user space provides a means of viewing the "persistent" data in the program without having to debug it. All you have to do is write a program to view the data in the user space.

http://www.mcpressonline.com

Powered by Joomla!

Generated: 21 December, 2008, 23:27