ACIF Indexing Introduction 2018

Understanding ACIF
Part 1: Introduction
The ACIF (AFP Conversion and Indexing Facility) utility consists of three separate but
related functions. ACIF can do the following:
- convert line data to AFP
- index data
- collect AFP resources
ACIF accepts as input either line data or AFP and can produce 3 output files:
- the output file, extension .out, which is either line data or AFP
- the index file, extension .ind, which is an AFP file
- the resource file, extension .res, which is an AFP file
There are 3 different combinations of input and output when running ACIF:
1. line data input -> line data output

Set the parameter CONVERT=NO
No resources
Files produced: .out, .ind
2. line data input -> AFP output

Set the parameter CONVERT=YES
An AFP formdef and pagedef are required for conversion
AFP resources can be collected
Files produced: .out, .ind, .res
3. AFP input -> AFP output

An AFP formdef is required to process AFP input
A subset of (2) is mixed mode input, which is line data with AFP records -> AFP output
An AFP formdef and pagedef are required for conversion
Types of ACIF Parameters
ACIF parameters for data:

CC
CCTYPE
TRC
FILEFORMAT
CPGID
ACIF parameters for line data to AFP Conversion:

CONVERT
MCF2REF recommend using CF instead of CPCS
IMAGEOUT recommend using ASIS instead of IOCA
FORMDEF
PAGEDEF
ACIF parameters for Indexing:

TRIGGER
FIELD
INDEX
INDEXOBJ
INDEXSTARTBY
ACIF parameters for collecting resources:

RESTYPE
EXTENSIONS=RESORDER
Tools for working on ACIF issues
Recommended tools needed for working on ACIF problems:
- a hex editor, for example, the HexEdit editor is available at

http://www.hexedit.com/
- arsafpd utility, available in CMOD 9.0

run like this: arsafpd –d -t -i <input file> > <output file>
The arsafpd utility can be used to display the .out file (if it is AFP), .ind file, and
the .res file that are created by ACIF.
Part 2: Understanding the Input Data
FILEFORMAT parameter
The FILEFORMAT parameter is used on MP to describe the input file.

It is not used on z/OS to describe the input file because on z/OS the file system already
contains the information describing the input file. It has a different use on z/OS; please
see the documentation.
The FILEFORMAT parameter has the following values on MP:
- record,n e.g. FILEFORMAT=record,100

Fixed length line data.
Usually these files have no delimiters.
- stream, e.g. fileformat = stream,(newline=X'0A') or (newline=X’0D0A’)

For files created on UNIX or windows platforms.
The delimiter is usually X'0A' or X'0D0A'.
- record e.g. FILEFORMAT=record

Each record has a 2-byte prefix, which contains the length of the record. This
length is exclusive, meaning it does not include the length of the 2-byte prefix
itself. MVS Download adds this prefix when it downloads files.
If the input is AFP, the FILEFORMAT parameter is not needed, except for AFP in record
format.
Types of carriage Controls
ANSI  encoded in EBCDIC ASCII

1 new page x’F1’ x’31’
<space> space one line x’40’ x’20’
0 space two lines x’F0’ x’30’
- space three lines x’60’ x’2D’
+ suppress space x’8F’ x’2B’
MACHINE
Machine carriage controls occur in EBCDIC data created on z/OS.
Here are the most common machine carriage controls:
X'09' Print the line and single space

X'11' Print the line and double space
X'19' Print the line and triple space
X'01' Print the line (do not space)
X'0B' Space one line immediately (do not print)
X'89' Print the line, then skip to channel 1 (top of form, by convention)
X'8B' Skip to channel 1 immediately (do not print)
Since machine carriage controls are binary values, if a file contains them, the file must
always be transferred as binary. Machine carriage controls cannot be converted to
ASCII.
You can find more information on the web about ANSI and Machine carriage controls.
The Carriage Control and the Delimiter
The most common delimiters are x’0A’ and x’0D0A’. Note that the carriage control and
the delimiter are two different things: the delimiter separates the records, the carriage
control is the first byte of each record. The carriage control follows the delimiter, except
at the beginning of the file, where the carriage control is the first byte. Therefore, to
search in a hex editor for the beginning of the next page of a file which uses x’0A’ as the
delimiter, search for x’0AF1’ or x’0A31’.
When analyzing a problem, it is important to know the format of the data. What is it?
Look at the input in a hexadecimal editor to be sure.
In the HexEdit editor, use View->Character Set->EBCDIC to switch from ASCII to
EBCDIC. Use Tools->Options->window tab to set number of columns displayed.
The following pages contain examples of different data formats.

EBCDIC line data with ANSI carriage controls, FILEFORMAT=RECORD,200
ASCII line data with no carriage controls, FILEFORMAT=stream,(newline=X'0D0A')

EBCDIC line data with ANSI carriage controls, FILEFORMAT=RECORD
ASCII line data with no carriage controls and x’0C’, use input exit ASCIINP or
ASCIINPE
AFP Structured Fields
Here is a brief introduction to AFP, which is also called MODCA.
The first nine bytes of an AFP record are called the Structured Field Introducer. Here is
an example:
5A 00 10 D3 A8 A8 00 00 00…
The first byte is always x’5A’.

The second and third bytes are the length (maximum length of 32767).
The fourth byte is always x’D3’.
The fourth, fifth, and sixth bytes are the Structured Field Identifier, for example,
x’D3A8A8’ or x’D3A8AF’.
The seventh byte is the flag byte, and the last two bytes are reserved, usually zeros.
What follows the reserved bytes depends on the type of Structured Field.
The length does not include the 5A.
There are 2 kinds of text in AFP files:

Internal text: encoded in EBCDIC.
Document text: the text in most AFP files is encoded in EBCDIC, but this is changing.
AFP must always be transferred as binary.
Link to MODCA Reference: http://publib.boulder.ibm.com/prsys/pdfs/ha3m6mst.pdf
The MODCA Reference can also be found on the website of the AFP Consortium.
AFP, FILEFORMAT parameter is not needed
Here are some examples in hexadecimal of the AFP Structured Field Introducer of the
most common Structured Fields that you may see at the beginning of an AFP file:
5A 00 10 D3 A8 A8 00 00 00 Begin Document (BDT)

5A 00 5B D3 A8 C6 00 00 00 Begin Resource Group (BRG)
5A 00 20 D3 EE EE 00 00 00 No Operation (NOP)
An AFP Structured Field can begin with a 2-byte length prefix. This is called record
format. Here is an example:
00 11 5A 00 10 D3 A8 A8 00 00 00 …
Note that the length in the 2-byte prefix is one greater than the length in the Structured
Field. This is because the 2-byte prefix includes the x’5A’, but does not include itself.
AFP in RECORD format, this is valid
Mixed Mode: AFP and ASCII line data with ANSI carriage controls,
FILEFORMAT=stream
What is this? It’s AFP that has been transferred as text instead of binary. It is no longer
valid.
Part 3: Understanding the Index File
There are two ways to run ACIF: from arsload or from the command line.
When arsload runs, it calls the indexer specified in the Application on the Index
Information tab.
ACIF can also be run from the command line like this:
>arsacif parmdd=<parameter file name>
To run ACIF from the command line, the inputdd, outputdd, indexdd and resobjdd (if
needed) parameters must be added to the parameter file. For example, here are the
contents of a parameter file named ‘test’:
cc=yes
cctype=a
cpgid=500
formdef=F1010110
reslib=\acif\reslib
inputdd=\acif\input\tle0.afp
outputdd=\acif\output\weirdtle.out
indexdd=\acif\index\weirdtle.ind
To run acif with this parameter file, enter the following on the command line:
>arsacif parmdd=test
This example would create an .out file named weirdtle.out and an .ind file named
weirdtle.ind.
The ACIF level can be determined by the xxxxxxx number in the final ACIF message,
for example:
APK440I ACIF AT PI45471 HAS COMPLETED NORMALLY WITH RETURN

CODE 0.
Or
APK441I ACIF AT PI45471 HAS COMPLETED ABNORMALLY WITH RETURN

CODE 8.
Example – indexing Credit Card Statements
When ACIF indexes and converts a file from line data to AFP, the indexes are extracted
before the conversion to AFP.
Here is our sample input:
EBCDIC line data with ANSI carriage controls, FILEFORMAT=stream,(newline=X'25')

Looking at the data in the Administrator Graphical Indexer:
The trigger and fields displayed in the Graphical Indexer, page 1.
When the trigger is found, ACIF collects the field values that will go into the index file.
When the trigger appears again on a subsequent page, ACIF will look at the field values
again.
When one of the field values changes, a new group will be started.
A group cannot be smaller than a page, i.e. you cannot create a new group except on a
new page boundary.
The trigger and fields displayed in the Graphical Indexer, page 2.
Indexing parameters generated by the GI:

CC=YES
CCTYPE=A
CPGID=500
TRIGGER1=*,65,X'D7C1C7C5F1' /* PAGE1 */
FIELD1=0,2,20,(TRIGGER=1,BASE=0) /* account name */
FIELD2=1,2,11,(TRIGGER=1,BASE=0) /* account number */
FIELD3=1,13,8,(TRIGGER=1,BASE=0) /* date */
FIELD4=1,71,8,(TRIGGER=1,BASE=0) /* balance */
INDEX1=X'D5C1D4C5',FIELD1 /* NAME */
INDEX2=X'C1C3C3D6E4D5E3',FIELD2 /* ACCOUNT */
INDEX3=X'C3D9C46DC4C1E3C5',FIELD3 /* CRD_DATE */
INDEX4=X'C2C1D3C1D5C3C5',FIELD4 /* BALANCE */
FORMDEF=F1ABBB
PAGEDEF=P1ABBB
INDEXOBJ=GROUP
RESTYPE=FDEF,PSEG,OVLY
USERLIB=/arsdata/project/data/credit/resource
Explanation of indexing parameters
TRIGGER1=*,65,X'D7C1C7C5F1' /* PAGE1 */
Look in every record in column 65 for the trigger.
FIELD1=0,2,20,(TRIGGER=1,BASE=0) /* account name */

Field1 occurs on the same line as Trigger1 in column 2, for 20 bytes.
FIELD2=1,2,11,(TRIGGER=1,BASE=0) /* account number */

Field2 occurs one line down from Trigger1 in column 2, for 11 bytes.
The field1 value is associated with the Index “NAME”.
CONVERT does not appear in the parameters because CONVERT=YES is the default.
CPGID should be set to the code page of the input file. The load process will use the
CPGID value to convert the index values to the code page of the database.
FORMDEFS and PAGEDEFS

The PAGEDEF is used to convert line data to AFP. The PAGEDEF cannot be collected
as a resource because, once the data is converted, the PAGEDEF is no longer needed.
A FORMDEF and PAGEDEF are always required for converting line data to AFP.
The FORMDEF is required, but the PAGEDEF is not required for AFP input.
Neither FORMDEF nor PAGEDEF are required for line data processed with the
CONVERT=NO option.
Multiple Group Triggers

trigger1 = *,8,X'D5D6E3C9C6E87E' /* 'NOTIFY=' */
trigger2 = 1,18,X'D7C7D47E' /* 'PGM=' */
trigger3 = 7,4,X'C4C5C6C9D5C540' /* 'DEFINE ' */
Additional Group Triggers are all offset from Trigger1.

All the triggers must be found before ACIF looks for the fields.
Floating Triggers
Fields based on Floating Triggers are picked up independently of fields based on Group
Triggers.
Fields based on Group Triggers are picked up once per group.
Fields based on Floating Triggers can be picked up multiple times within a group.
A field based on a Floating Trigger cannot “break” a group, because a group can’t be
broken except on a page boundary.
A document cannot be loaded using only fields based on floating triggers. The indexing
parameters must contain at least one group trigger and one field based on it.
Example of the correct way to use a floating trigger:
cc=yes
cctype=a
fileformat=record
convert=no
trigger1=*,2,X'D9C5D7D6D9E3' /* REPORT */
trigger2=*,*,X'D9E4D540C4C1E3C5C1',(type=float) /* RUN DATE */
field1=0,9,9
field2=0,9,9,(trigger=2,base=trigger)
index1=X'C6C9C5D3C4F1',field1
index2=X'C6C9C5D3C4F2',field2
The Connection between ACIF and OnDemand
The Index Names match the Application Group Database Names:

The Index File
Use arsafpd to display the Index File.
Index file generated:
...1 BDI (Begin Document Index) NAME (NULL) SFI 0078 D3A8A7 000000
Bxx: CGCSGID Triplet (01):
Bxx: - Coded Graphic Char Set Global ID (GCSGID) = 65535 (FFFF)
Bxx: - Code Page Global ID (CPGID) = 500 (01F4) <- FROM CPGID
Bxx: Fully Qualified Name Triplet (02):
Bxx: - use = 01 (replace first GID name)
Bxx: - name = '*/...*.....>.*.>...*..../._.'
Bxx: - use = 83 (begin document reference)
Bxx: - name = '*/...*.....>.*?.....*..../._.'
Bxx: - use = 0A (begin resource group reference)
Bxx: - name = '*/...*.....>.*...?.║*..../._.'
...3 - TLE (Tag Logical Element) SFI 0032 D3A090 000000
TLE: Fully Qualified Name Triplet (02):
TLE: - use = 0B (attribute name)
TLE: - name = 'NAME'
TLE: Attribute Value Triplet (36):
TLE: - value = 'Smith Cyclery Co '
TLE: Attribute Qualifier Triplet (80):
TLE: - sequence number = 0 (000000)
TLE: - level number = 2147483647 (7FFFFFFF)
...4 - TLE (Tag Logical Element) SFI 002C D3A090 000000
TLE: - name = 'ACCOUNT'
TLE: - value = '000-000-000'
...5 - TLE (Tag Logical Element) SFI 002A D3A090 000000
TLE: - name = 'CRD_DATE'
TLE: - value = '06/03/94'
TLE: - name = 'BALANCE'
TLE: - value = ' 47.72'
The TLE values in the index file become the values in the Content Manager OnDemand
Search Results/Document List:
The indexes are used to retrieve the document. This is why the index file is so important,
and why no data can be loaded without an index file.
The IEL indicates where the groups start and how long each one is:
...2 - IEL (Index Element) SFI 005D D3B2A7 000000

IEL: Object Byte Extent Triplet (57):
IEL: - byte extent of object = 1614 (00064E) <- LENGTH OF GROUP
IEL: Direct Byte Offset Triplet (2D):
IEL: - byte offset = 201 (0000C9) <- WHERE IT STARTS IN THE .OUT FILE
IEL: Object Structured Field Extent Triplet (59):
IEL: - num of structured fields in object = 18 (000012)
IEL: Object Structured Field Offset Triplet (58):
IEL: - structured field offset = 1 (000001)
IEL: Medium Map Page Number Triplet (56):
IEL: - sequence num of page = 1 (000001)
IEL: Fully Qualified Name Triplet (02):
IEL: - use = 0D (begin page group reference)  PAGE GROUP IEL
IEL: - name = 'Smith Cyclery Co 00000001'
IEL: - use = 8D (begin medium map reference)
IEL: - name = 'ABBB '
If we go to offset 201 in the .out file we find a BNG structured field:

ACIF has transformed the line data to AFP, and BNG/ENG structured fields indicate the
start and end of each group.
If we were looking at an .out file containing line data instead of AFP, we would see the
beginning of a record ( and probably a new page carriage control) at the location given by
the IEL group offset.
You may see Page Level IELs in the Index file. These are created by setting the ACIF
parameter INDEXOBJ=ALL. They are only needed (and are required) if the document is
being loaded as Large Object:
...7 - IEL (Index Element) SFI 0044 D3B2A7 000000

IEL: Object Byte Extent Triplet (57):
IEL: - byte extent of object = 1342 (00053E) <- LENGTH OF PAGE
IEL: Direct Byte Offset Triplet (2D):
IEL: - byte offset = 456 (0001C8) <- WHERE IT STARTS IN THE .OUT FILE
IEL: Object Structured Field Extent Triplet (59):
IEL: - num of structured fields in object = 11 (00000B)
IEL: Object Structured Field Offset Triplet (58):
IEL: - structured field offset = 7 (000007)
IEL: Medium Map Page Number Triplet (56):
IEL: - sequence num of page = 1 (000001)
IEL: - use = 87 (begin page reference) <- PAGE IEL
IEL: - name = '00000001'
IEL: - use = 8D (begin medium map reference)
IEL: - name = 'ABBB '
IEL: Process Level Triplet (81):
IEL: - process level = 3 (03)
Summary of Index file information

1. CPGID value is placed into the BDI and it is used to convert the index names and
values when they are loaded into the database, e.g. the index names and values are
in EBCDIC, but the database may be in ASCII.
2. TLE’s contain the index values that will appear in the Search Results screen.
3. Group IEL’s contain the offset of where the group starts in the .out file and the
length of each group.
4. All this information is loaded into CMOD database tables and the index file is
discarded.
What if the index names don’t match the AG database names?
Remember that the TLE name comes from the ACIF INDEX parameter:
Instead of this:

TLE: - name = 'NAME'
We have this:

TLE: - name = 'SILLY NAME'
Then we do the following in the Application:

Part 4: AFP Input
The previous example looked at running ACIF and indexing a line data document.
Another way of running ACIF is when the input is AFP and already contains TLEs
and BNG/ENG pairs. This data is called “fully composed AFP”.
Here is an arsafpd dump of a fully composed AFP file in the correct format for
loading into OnDemand (arsafpd was run without the –t option) :
...1 BDT (Begin Document) NAME (NULL)

...2 - BNG (Begin Named Page Group)
...3 - - TLE (Tag Logical Element) <- TLE’s must appear after the BNG
...4 - - TLE (Tag Logical Element) <- and before the BPG
...5 - - TLE (Tag Logical Element)
...6 - - BPG (Begin Page) NAME (NULL)
...7 - - - BAG (Begin Active Environ Group)
...8 - - - - MCF2 (Map Coded Font 2)
..21 - - - - PGD (Page Descriptor)
..22 - - - - PTD2 (Presentation Text Descrip)
..23 - - - EAG (End Active Environ Group)
..24 - - - BCT (Begin Composed-Text Block)
..25 - - - - PTX (Presentation Text Data)
..35 - - - ECT (End Composed-Text Block)
..36 - - EPG (End Page) NAME (NULL)
..37 - - BPG (Begin Page) NAME (NULL)
..38 - - - BAG (Begin Active Environ Group)
..39 - - - - MCF2 (Map Coded Font 2)
..60 - ENG (End Named Page Group)
..61 - BNG (Begin Named Page Group)
..62 - - TLE (Tag Logical Element)
..65 - - BPG (Begin Page) NAME (NULL)
..66 - - - BAG (Begin Active Environ Group)
..67 - - - - MCF2 (Map Coded Font 2)
.119 - ENG (End Named Page Group)
EDT (End Document) NAME (NULL)
When an input file contains TLE structured fields, if you specify indexing parameters
(such as TRIGGER, INDEX, or FIELD), ACIF will fail with error msg 462 – “A trigger
parameter was specified, but the input file is already indexed.”
ACIF processes a file containing TLE structured fields in the following way:
1. For every BNG in the input, ACIF creates a group IEL structured field in the index
file.
2. ACIF makes a copy of the TLE structured fields from the input and places them into
the index file. The original TLE structured fields remain in the out file.
Therefore, the result of ACIF processing under these circumstances is the creation of an
index file. ACIF can complete normally but the load process into OnDemand may still
fail if the format of the input file is incorrect:
- If the input file contains BNG - ENG pairs with no group level TLE Structured Fields
between them, then the load process will fail with the message : 0 fields submitted, n
expected, where n is the number of fields defined to OnDemand.
For example, “0 fields submitted, 3 expected”
-If the input file does not contain the correct number of TLEs in each group, the load
process will fail with the message: x fields submitted, n expected, where n is the number
of fields defined to OnDemand.
- If the input file does not contain any BNG - ENG pairs, then the load process may run
out of memory looking for the start and end of the groups.
Running ACIF under these circumstances, you may find that the output file is larger than
the input. Why, you may wonder, since they are both AFP files? The answer is because
ACIF updates the AFP and usually ends up increasing the file size. Some of the changes
to the AFP may include the following: creating or adding comments to the BDT
Structured Field, creating or adding group names to the BNG - ENG Structured Fields,
adding TLE Structured Fields (if indexing), and changing obsolete Structured Fields to
current ones (e.g. MCF1 to MCF2, PTD1 to PTD2).
Compare the previous file with the following. This file will not load into OnDemand
because it does not contain any group TLE’s.
...1 BDT (Begin Document) NAME 1_10000

...2 - NOP (No Operation)
...3 - NOP (No Operation)
...4 - BNG (Begin Named Page Group) NAME G0000001
...5 - - BPG (Begin Page) NAME P0000001
...6 - - - BAG (Begin Active Environ Grou NAME PAGAEG
...7 - - - - MCF1 (Map Coded Font 1)
...8 - - - - MPS (Map Page Segment)
...9 - - - - PGD (Page Descriptor)
..10 - - - - CTC (Composed-Text Control)
..12 - - - EAG (End Active Environ Group) NAME PAGAEG
..13 - - - TLE (Tag Logical Element) <- PAGE TLEs
..14 - - - TLE (Tag Logical Element)
..15 - - - TLE (Tag Logical Element)
..16 - - - IPS (Include Page Segment)
..17 - - - IPS (Include Page Segment)
..18 - - - BCT (Begin Composed-Text Block NAME PTXREC
..20 - - - ECT (End Composed-Text Block) NAME PTXREC
..21 - - EPG (End Page) NAME P0000001
..22 - ENG (End Named Page Group) NAME G0000001
IMPORTANT:
When an input file contains BNG/ENG and TLE structured fields, if you specify indexing
parameters (such as TRIGGER, INDEX, or FIELD), ACIF may fail with error message
462 : “A trigger parameter was specified, but the input file is already indexed.”, or with
error message 459 : “Index needed for the Groupname was not found”.
What this means is that you can extract TLEs or you can extract fields from the data
(using trigger, field and index parameters) but you CANNOT use both methods at the
same time. You can't specify a constant field on a FIELD parameter when the AFP file
contains BNG/ENGs and TLEs.
However, if you would like to add a constant field to an AFP file that contains TLEs, you
can specify the ADDTLE parameter:
CC=YES
CCTYPE=A
CPGID=500
FIELD1='MONTHLY'
INDEX1='REPORT_TYPE'
EXTENSIONS=ADDTLE
See the documentation in the Indexing Reference for more information about the
ADDTLE parameter.
TLEs versus NOPs

As mentioned earlier, you can extract index values from line data with TRIGGER,
FIELD and INDEX parameters, or you can extract index values from AFP by extracting
TLE structured fields. The most common way to extract index values from AFP
documents is by extracting the TLEs. However, another way to index the AFP is to
process it as if it were line data and extract the index values from NOPs inside the AFP.
In order to use this method, the AFP must meet the following conditions:
− The AFP file must not contain any BNG/ENG pairs or TLEs.
− The NOPs must occur outside of the BPG/EPG pairs.
Specify the following indexing parameters:

CC=YES
CCTYPE=A
CONVERT=YES
CPGID
FORMDEF
RESTYPE
TRIGGER
FIELD
INDEX
ACIF will extract the index values from the NOP structured fields and place them into
the index file.
Part 5: The ACIF input exits ASCIINP and ASCIINPE
Recall that we use these exits on data of the following format:
ASCII data, no carriage controls, FILEFORMAT=STREAM,(NEWLINE=x’0D0A’)
The problem with these files is that they do not have valid carriage controls. ACIF
requires either ANSI or machine carriage controls. The purpose of these exits is to insert
valid carriage controls.
The asciinp exit makes the following transformations:

Changes x’0C’ -> x’31’ x’31’ is the new page carriage control
Changes x’0D0A’ -> x’0A20’ x’0A’ is the delimiter; x’20’ is the carriage control
The asciinpe exit makes the same transformations and also converts the data to EBCDIC.
If converting to AFP, always convert ASCII data to EBCDIC.
The exit is applied before the indexing takes place.

Before asciinp applied on top; after asciinp applied on bottom.
Notice that after asciinp has processed the data, most of the lines now contain an extra
byte. Therefore, the trigger and field column offsets have changed.
For a file that is being transformed by asciinp, use these parameters for the data:
CC=YES
CCTYPE=Z
CPGID=850
FILEFORMAT=STREAM,(NEWLINE=X’0A’)
Before asciinpe applied on top; after asciinpe applied on bottom.
For a file that is being transformed by asciinpe, and AFP conversion is being done, use
these parameters for the data, along with TRIGGER, FIELD and INDEX parameters:
CC=YES
CCTYPE=Z
CPGID=500
CONVERT=YES
For a file that is being transformed by asciinpe, and AFP conversion is not being done,
use these parameters for the data, along with TRIGGER, FIELD and INDEX parameters:
CC=YES
CCTYPE=A
CPGID=500
CONVERT=NO
To develop indexing parameters for files that use the asciinp and asciinpe exits, we
recommend that the file first be transformed by the exit, and then loaded into the
Administrator Graphical Indexer in order to create the indexing parameters.
Here are sample parameters to run only the exit. Run arsload with the – i option or run
arsacif from the command line.
cc=yes
cctype=z (for asciinp) or cctype=a (for asciinpe)
convert=no
fileformat=stream,(newline=x'0A')
inpexit=<path>asciinpe or inpexit=<path>asciinp
Take the .out file and load it into the Administrator Graphical Indexer. Create the
indexing parameters.
After the correct indexing parameters have been created, run arsload/ACIF with
CONVERT=YES or CONVERT=NO on the original file. Here is an example of
complete ACIF parameters for indexing with CONVERT=YES:
CC=YES
CCTYPE=Z
CPGID=500
MCF2REF=CF
TRC=NO
FORMDEF=F1A10110
PAGEDEF=P1B06683
FILEFORMAT=STREAM,(NEWLINE=X'0A')
TRIGGER1=*,8,X'C481A3857A' /* Date: */
FIELD1=0,1,5
FIELD2=0,13,8
FIELD3=4,12,8
FIELD4=5,12,10
INDEX1=X'526E616D65',FIELD1 /* Rname */
INDEX2=X'5264617465',FIELD2 /* Rdate */
INDEX3=X'556E616D65',FIELD3 /* Uname */
INDEX4=X'4A6E616D65',FIELD4 /* Jname */
DCFPAGENAMES=NO
UNIQUEBNGS=YES
IMAGEOUT=ASIS
INDEXOBJ=ALL
INDEXSTARTBY=1
INSERTIMM=NO
RESTYPE=NONE
INPEXIT=\arc\acifsrc\release\asciinpe
RESLIB=\acif\reslib
For more on the acif input exit, see
http://www.ibm.com/developerworks/data/library/techarticle/0307muir/0307muir.html
NOTE: Starting with CMOD 9.5.0.3, it is possible for ACIF to process data that contains
the x'0C' form feed character without using the asciinp or asciinpe exit. See the
documentation for the ACIF FORMFEED parameter and Technote #1962936.
Part 6: Common User Errors and Problem Determination
If ACIF failed, investigate the ACIF error message. If ACIF completes normally but
arsload failed, note the arsload error message and if necessary look at the index file to see
if it looks reasonable. “Reasonable” means that it contains at least one group IEL, and at
least one group TLE.
Example 1
0425-415 CC=YES
0425-415 CCTYPE=A
0425-415 CONVERT=NO
0425-415 CPGID=500
0425-415 MCF2REF=CPCS
0425-415 TRC=NO
0425-415 FILEFORMAT=RECORD,133
0425-415 TRIGGER1=*,2,X´C3C460F0F7F1´,(TYPE=GROUP)
0425-415 FIELD1=0,115,8,(TRIGGER=1,BASE=0)
0425-415 INDEX1=X´9985978481A385´,FIELD1,(TYPE=GROUP,BREAK=YES)
0425-415 DCFPAGENAMES=NO
0425-415 UNIQUEBNGS=YES
0425-415 IMAGEOUT=ASIS
0425-415 INDEXOBJ=GROUP
0425-415 INDEXSTARTBY=1
0425-415 INSERTIMM=NO
0425-415 RESTYPE=NONE
0425-415 inputdd=/az/arsload/fail/testfile
0425-415 outputdd=/dev/null
0425-415 indexdd=./testfile.ind
0425-415 resobjdd=/dev/null
0425-422 AN ERROR OCCURRED WHILE ATTEMPTING TO READ
/az/arsload/fail/testfile RETURN CODE 310.
RC 310 means that either the FILEFORMAT parameter is wrong or something is wrong
with the input file.
If you see RC 28, it means that ACIF could not find a file.
To look at this type of file in the HexEdit editor, use View->Character Set->EBCDIC and
View->Options to set the record length to 133. Then you can scroll to the right and view
the data in character format. The problem with this particular file is that it is missing
bytes at the end; the last line is not 133 characters long.
Another variation of the same problem is when the last line does not end with the
delimiter. If a file has delimiters, every line must end with one.
For this type of problem ACIF will issue the error message similar to “Unable to read
from offset 18283 for a length of 1200.”
Example 2
0425-408 A VIRTUAL STORAGE REQUEST WAS UNSUCCESSFUL - REQUEST

SIZE 2569
RETURN CODE 8
ACIF running out of storage is usually a problem with carriage controls: either they are
specified incorrectly to ACIF or the document does not have any.
Suggestions for how to deal with a document that does not have carriage controls:
1. If the data uses x’0C’ to indicate the start of a new page, use the ACIF exit asciinp or
asciinpe to insert carriage controls.
2. Use a custom input exit to insert carriage controls.
3. Use the ACIF parameter LINECNT to insert page breaks.
Example 3
0425-415 CC=NO
0425-415 CCTYPE=Z
0425-415 CONVERT=NO
0425-415 CPGID=850
0425-415 LINECNT=66
0425-415 TRC=NO
0425-415 FILEFORMAT=STREAM,(NEWLINE=X´0A´)
0425-415 TRIGGER1=*,1,X´53594D424F4C´,(TYPE=GROUP)
0425-415 TRIGGER2=*,1,X´53594D424F4C´,(TYPE=FLOAT)
0425-415 INDEX1=X´53796D626F6C´,FIELD1,(TYPE=GROUP,BREAK=NO)
0425-415 INDEX2=X´5265706F727444617465´,FIELD2,(TYPE=GROUP,BREAK=NO
)
0425-415 INDEX3=X´537461727444617465´,FIELD3,(TYPE=GROUP,BREAK=NO)
0425-415 INDEX4=X´456E6444617465´,FIELD4,(TYPE=GROUP,BREAK=NO)
0425-415 INDEX5=X´526571756573744964´,FIELD5,(TYPE=GROUP,BREAK=NO)
0425-415 INDEXOBJ=ALL
0425-415 inputdd=testfile.ARD
0425-415 outputdd=/dev/null
0425-415 indexdd=testfile.ind
0425-415 resobjdd=/dev/null
0425-440 ACIF AT PK04939 HAS COMPLETED NORMALLY WITH RETURN
CODE 0.
arsload: 07/19/06 15:53:30 Indexing completed
arsload: 07/19/06 15:53:30 -- Loading started, 209181 bytes to process
Loaded 0 rows into the database
Document compression type used - OD77. Bytes Stored = >0<
arsload: 07/19/06 15:53:30 Loading failed
ACIF completed normally, therefore it produced an index file. Look at the index file to
see if it looks reasonable. The following index file is NOT reasonable; every index file
must have at least one Group IEL and TLE.
...1 BDI (Begin Document Index) NAME (NULL) SFI 0053 D3A8A7 000000
Bxx: CGCSGID Triplet (01):
Bxx: - Coded Graphic Char Set Global ID (GCSGID) = 65535 (FFFF)
Bxx: - Code Page Global ID (CPGID) = 850 (0352)
Bxx: - use = 01 (replace first GID name)
Bxx: - name = '..............|&..........................>.'
Bxx: - use = 83 (begin document reference)
Bxx: - name = '.....>.%%'
...2 EDI (End Document Index) NAME (NULL) SFI 0010 D3A9A7 000000
Notice the following:
0425-415 TRIGGER1=*,1,X´53594D424F4C´,(TYPE=GROUP)
0425-415 TRIGGER2=*,1,X´53594D424F4C´,(TYPE=FLOAT)
The problem is that every field is based on the floating trigger. arsload requires at least
one index based on a Group trigger, which in this case is TRIGGER1. This is because
only fields that are based on Group triggers can be used to break the document into
groups, and the index process must create at least one group. (And recall that a group is
represented by a group IEL in the index file.) In cases like this the customer must
redesign their indexing parameters.
Example 4
0425-415 CC=YES
0425-415 CCTYPE=Z
0425-415 CONVERT=NO
0425-415 CPGID=850
0425-415 TRC=NO
0425-415 FILEFORMAT=STREAM,(NEWLINE=X'0D0A')
0425-415 TRIGGER1=*,123,X'504147452020202020202031',(TYPE=GROUP)
0425-415 FIELD7=0,117,5,(TRIGGER=1,BASE=0,DEFAULT=X'30313031')
0425-415 INDEX1=X'7270746E6F',FIELD1,(TYPE=GROUP,BREAK=YES)
0425-415 INDEX2=X'7270746E616D65',FIELD2,(TYPE=GROUP,BREAK=YES)
0425-415 INDEX6=X'737973646174',FIELD6,(TYPE=GROUP,BREAK=YES)
0425-415 INDEX7=X'73797374696D65',FIELD7,(TYPE=GROUP,BREAK=YES)
0425-415 inputdd=testfile
0425-415 outputdd=testfile.out
0425-415 indexdd=testfile.ind
0425-448 INDEXING WAS REQUESTED, BUT NEITHER 'TRIGGER1' NOR ANY
'FIELD' WAS SATISFIED WITHIN THE PAGE RANGE SPECIFIED BY THE
INDEXSTARTBY PARAMETER.
0425-441 ACIF AT PK10069 HAS COMPLETED ABNORMALLY WITH RETURN
CODE 16.
The problem is that the trigger was not found, because either the trigger value or location
was specified incorrectly.
Example 5
arsload: 06/09/06 10:59:40 -- Indexing started, 2812323 bytes to process

0425-415 CC=YES
0425-415 CCTYPE=Z
0425-415 CONVERT=NO
0425-415 CPGID=850
0425-415 TRC=NO
0425-415 FILEFORMAT=STREAM,(NEWLINE=X'0D0A')
0425-415 TRIGGER1=*,72,X'5041474520202031',(TYPE=GROUP)
0425-415 INDEX1=X'416363744E756D',FIELD2,(TYPE=GROUP,BREAK=YES)
0425-415 INDEX2=X'53746D6E7444617465',FIELD1,(TYPE=GROUP,BREAK=YES)
0425-415 inputdd=C:\arsload\testfile.rpt
0425-415 outputdd=NUL
0425-415 indexdd=.\testfile.rpt.ind
0425-415 resobjdd=NUL
0425-346 DATA IN AN INPUT RECORD OR PAGEDEF RESOURCE IS INVALID: A SKIP TO A
NONEXISTENT CHANNEL = '3A'X ON RECORD NUMBER = 1 WAS DETECTED WITHIN THE
LND STRUCTURED FIELDS. OUTPUT WAS FORCED TO SINGLE SPACING, WHICH MAY
CAUSE BLANK PAGES.
NONEXISTENT CHANNEL = '4C'X ON RECORD NUMBER = 54 WAS DETECTED WITHIN
THE LND STRUCTURED FIELDS. OUTPUT WAS FORCED TO SINGLE SPACING, WHICH
MAY CAUSE BLANK PAGES.
NONEXISTENT CHANNEL = '0C'X ON RECORD NUMBER = 58 WAS DETECTED WITHIN
THE LND STRUCTURED FIELDS. OUTPUT WAS FORCED TO SINGLE SPACING, WHICH
MAY CAUSE BLANK PAGES.
0425-441 ACIF AT PK04939 HAS COMPLETED ABNORMALLY WITH RETURN CODE 8.
arsload: 06/09/06 10:59:40 Indexing failed
The values x’3A’, x’4C’ and x’0C’ which appear in the messages are values that appear
in the first byte of three records, and ACIF knows that the first byte in the record is
supposed to be the carriage control if CC=YES is specified. ACIF is complaining
because these values are not valid ANSI carriage controls.
Notice that CCTYPE=Z was specified. That means that the file should have ANSI
carriage controls encoded in ASCII. The valid ANSI carriage control for a new page is
x’31’, and for a space one line it is a x’20’.
The problem with this file is that CC=YES was specified but the document does not have
carriage controls. The customer should use the acif input exit asciinp to insert carriage
controls.
Example 6
The following problems tend to occur with data that is not consistent.
Problem:
Message from ARSLOAD
2 fields submitted, 3 expected (or, 0 fields submitted, 3 expected)
There are two places to define default values which can be used in the loading process: the Application or
the ACIF field parameter. They are used in different ways.
1. If a field is not found in the data because the field is based off a floating trigger and the trigger was not
found, then there is no TLE for it and ARSLOAD will use the default value, which is defined in the
Application. If no default value was defined, loading will fail with the above message.
2. If the trigger/triggers were found, but the record was not long enough to contain the field, then ACIF will
create a TLE for that field using the default value from the ACIF Field parameter. If the record is not long
enough to contain the field, and no default value has been specified in the ACIF field parameters, ACIF
will fail with message 449: Index Fields Reference Outside of the Record.
Problem:
Message from ARSLOAD
The value ‘’ cannot be converted to a date using the format %d/%m/%y.

Or
The value ‘ ’ cannot be converted to a date using the format %d/%m/%y.
The data may not be consistent, and ACIF may have picked up blanks (or something else) instead of a date.
If blank character removal is set in the Application, the blanks may be removed, leaving a null value.
IMPORTANT: One cannot remove the blanks and then use the default value in the Application.
The default value in the Application is used only when no TLE is created, i.e. when no index value is
found.

ACIF Indexing Introduction 2018

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

ACIF Indexing Introduction 2018

Загружено:

Авторское право:

Доступные форматы

Understanding ACIF

1. line data input -> line data output

2. line data input -> AFP output

3. AFP input -> AFP output

ACIF parameters for data:

ACIF parameters for line data to AFP Conversion:

ACIF parameters for Indexing:

ACIF parameters for collecting resources:

Tools for working on ACIF issues

Recommended tools needed for working on ACIF problems:

- a hex editor, for example, the HexEdit editor is available at

- arsafpd utility, available in CMOD 9.0

The FILEFORMAT parameter is used on MP to describe the input file.

The FILEFORMAT parameter has the following values on MP:

- record,n e.g. FILEFORMAT=record,100

- stream, e.g. fileformat = stream,(newline=X'0A') or (newline=X’0D0A’)

- record e.g. FILEFORMAT=record

Types of carriage Controls

ANSI  encoded in EBCDIC ASCII

Here are the most common machine carriage controls:

X'09' Print the line and single space

The Carriage Control and the Delimiter

The following pages contain examples of different data formats.

ASCII line data with no carriage controls, FILEFORMAT=stream,(newline=X'0D0A')

Here is a brief introduction to AFP, which is also called MODCA.

The first byte is always x’5A’.

The length does not include the 5A.

There are 2 kinds of text in AFP files:

5A 00 10 D3 A8 A8 00 00 00 Begin Document (BDT)

Part 3: Understanding the Index File

APK440I ACIF AT PI45471 HAS COMPLETED NORMALLY WITH RETURN

APK441I ACIF AT PI45471 HAS COMPLETED ABNORMALLY WITH RETURN

Example – indexing Credit Card Statements

Here is our sample input:

EBCDIC line data with ANSI carriage controls, FILEFORMAT=stream,(newline=X'25')

The trigger and fields displayed in the Graphical Indexer, page 1.

Indexing parameters generated by the GI:

FIELD1=0,2,20,(TRIGGER=1,BASE=0) /* account name */

FIELD2=1,2,11,(TRIGGER=1,BASE=0) /* account number */

FORMDEFS and PAGEDEFS

Multiple Group Triggers

Additional Group Triggers are all offset from Trigger1.

Example of the correct way to use a floating trigger:

The Index Names match the Application Group Database Names:

Use arsafpd to display the Index File.

Index file generated:

...2 - IEL (Index Element) SFI 005D D3B2A7 000000

If we go to offset 201 in the .out file we find a BNG structured field:

...7 - IEL (Index Element) SFI 0044 D3B2A7 000000

Summary of Index file information

What if the index names don’t match the AG database names?

...3 - TLE (Tag Logical Element) SFI 0032 D3A090 000000

...3 - TLE (Tag Logical Element) SFI 0032 D3A090 000000

Then we do the following in the Application:

...1 BDT (Begin Document) NAME (NULL)

For example, “0 fields submitted, 3 expected”

...1 BDT (Begin Document) NAME 1_10000

TLEs versus NOPs

Specify the following indexing parameters:

Recall that we use these exits on data of the following format:

ASCII data, no carriage controls, FILEFORMAT=STREAM,(NEWLINE=x’0D0A’)

The asciinp exit makes the following transformations: