Redbooks Paper

Mike Gilley

Data Sorting Considerations with

Lotus Enterprise Integrator™ (LEI) 6
on the IBM iSeries Server

The purpose of this document is to provide an explanation of how the iSeries™ sorts data
and how you can make adjustments to the sorting sequence. This is important because
EBCDIC-based systems sort data alpha characters first, then numeric characters, while
ASCII data is sorted numeric first, then alpha characters. Domino™ for iSeries, although
native on the iSeries server, still sorts data as an ASCII-based system. With this information
you can understand how to change the sort order of data when retrieved from UDB DB2®.
Ideally, this causes the data that is retrieved from UDB DB2 to be presorted before it is
presented to the LEI activity, therefore eliminating the need for using an order metaconnector
against a DB2 connector when retrieving UDB DB2 data from an iSeries server.

By default, the iSeries is not configured to sort data, which causes a sort/order mismatch
when retrieving data from UDB DB2 tables on the iSeries server via Lotus® Domino,
especially when using LEI and Domino Enterprise Connection Services (DECS). This is most
noticeable when running an LEI replication activity from DB2 to Notes®, where unnecessary
record insert/deletes occur. LEI provides an order metaconnector that can be used to address
this specific issue. However, the use of this metaconnector affects the performance of LEI,
and, if the result set is too large, can cause the activity to fail. This can be observed in the LEI
activity log entry as shown in Figure 1.

© Copyright IBM Corp. 2003. All rights reserved. ibm.com/redbooks 1

 05/12/2003 04:03:22 PMActivity started
05/12/2003 04:03:24 PMCreated temporary Notes view '(LEIIndexView0)'
** Supply a view name to create a more efficient permanent view, Connector
'Qcustcdt.nsf Notes Conn', Method -Select-
05/12/2003 04:03:24 PMTrace Statement: <SELECT
CUSNUM,LSTNAM,INIT,STREET,CITY,STATE,ZIPCOD,CDTLMT,CHGCOD,BALDUE,CDTDUE FROM
mgilley.qcustcdt2 ORDER BY LSTNAM>, Connector 'RCHASSLH', Method -Select-
05/12/2003 04:03:24 PMTrace Statement: <SELECT * FROM mgilley.qcustcdt2 WHERE 0 = 1>,
Connector 'RCHASSLH', Method -Insert-
05/12/2003 04:03:24 PMTrace Statement: <INSERT INTO mgilley.qcustcdt2
(CUSNUM,LSTNAM,INIT,STREET,CITY,STATE,ZIPCOD,CDTLMT,CHGCOD,BALDUE,CDTDUE) VALUES
(?,?,?,?,?,?,?,?,?,?,?) >, Connector 'RCHASSLH', Method -Insert-
05/12/2003 04:03:24 PM*Replication Conflict* Action: Insert into Connector B;
Record key values: LSTNAM = "1A"
05/12/2003 04:03:24 PM*Replication Conflict* Action: Insert into Connector B;
Record key values: LSTNAM = "1B"
05/12/2003 04:03:24 PM*Replication Conflict* Action: Insert into Connector B;
Record key values: LSTNAM = "1C"
05/12/2003 04:03:24 PMTrace Statement: <DELETE FROM mgilley.qcustcdt2 WHERE LSTNAM=?>,
Connector 'RCHASSLH', Method -Remove-
05/12/2003 04:03:24 PM*Replication Conflict* Action: Remove from Connector B;
Record key values: LSTNAM = "1A"
05/12/2003 04:03:24 PM*Replication Conflict* Action: Remove from Connector B;
Record key values: LSTNAM = "1B"
05/12/2003 04:03:24 PM*Replication Conflict* Action: Remove from Connector B;
Record key values: LSTNAM = "1C"
05/12/2003 04:03:24 PMActivity finished

Statistics
Elapsed Time: 00:00:02
Records Fetched From All Connectors: 12
Records Inserted From All Connectors: 3
Records Updated From All Connectors: 0
Records Removed From All Connectors: 3

Figure 1 LEI activity log

As you can see in Figure 1, the same three records are added then deleted.

By changing the QNOTES user profile, and configuring it to use a specific sort sequence, you
can avoid the issue entirely. When the data is retrieved, it will be presorted before it is
presented to LEI. Therefore, because the data is presorted, there is no need to use an order
metaconnector against a DB2 connection that references UDB DB2 on the iSeries server.
This is important because it eliminates the need for the metaconnector against the DB2
connection, while it also prevents the unnecessary insert/deletes of records during a
replication activity, therefore enhancing overall LEI performance.

Note: This will have no effect when NON-alpha-numeric data is involved. If

non-alpha-numeric data is involved, then an order metaconnector against the Notes
connection, specifying a character set of ASCII, must be used.

The sort sequence (SRTSEQ), along with language ID (LANGID), is used to determine the
sort sequence table to be used for sorting character data. You can change SRTSEQ to any
one of the values listed in Table 1.

2 Data Sorting Considerations with Lotus Enterprise Integrator (LEI) 6 on the IBM eServer iSeries Server
 Note: Changing the OS/400® system values QSRTSEQ and QLANGID is not
recommended. These are global settings for the iSeries server, and changing the values
will impact other applications running on the iSeries server. All changes should be made
directly to the OS/400 user profiles.

Table 1 Sort sequence (SRTSEQ) values

Value Description

*HEX No sort sequence table is used. The hexadecimal values of the graphic
characters are used to determine the sort sequence (a binary sort). This
is the only sort sequence available for DBCS (Double Byte Character Set)
data.

Note: You can specify values other than *HEX for mixed-byte character
data. When you specify values other than *HEX for mixed-byte character
data, SBCS character data is sorted according to the sort sequence
specified. DBCS character data is sorted by hexadecimal values (binary
sort).

*LANGIDSHR The sort sequence table may use the same weight for multiple graphic
characters. The shared-weight sort table associated with the language
specified in the LANGID parameter is used.

This sort applies only to SBCS (Single Byte Character Set) data.

*LANGIDUNQ The sort sequence table contains uniquely weighted graphic characters.
The unique-weight sort table associated with the language specified in the
LANGID parameter is used.

This sort applies only to SBCS (Single Byte Character Set) data.

Qualified sort sequence The name and library of the sort sequence table to be used. This value
table name allows you to specify a sort sequence table other than those associated
with the language specified in the LANGID parameter.

This sort applies to only SBCS (Single Byte Character Set) data.

With the above settings in mind, you can make the comparisons shown in Table 2 between
the OS/400 user profile sort sequence setting and LEI order metaconnector settings.

Table 2 Comparison between OS/400 user profile sort sequence and LEI order metaconnector
OS/400 user profile sort sequence (SRTSEQ) LEI order metaconnector

*HEX Binary

*LANGIDSHR Sort without case sensitivity

*LANGIDUNQ Sort with case sensitivity

Sort sequences are supported only for SBCS EBCDIC languages. The following tables
(Table 3, Table 4 on page 5, and Table 5 on page 7) show the relative sort sequence weights
for characters sorted using the default sort sequence tables shipped with the iSeries server.
When looking at these tables, consider the following:
򐂰 Several tables shipped with the system represent a single sort sequence, each encoded
with a different coded character set identifier (CCSID) value. Not all of the characters in a
given sort sequence exist in every CCSID in which the sort sequence is encoded.

Data Sorting Considerations with Lotus Enterprise Integrator™ (LEI) 6 on the IBM iSeries Server 3
 򐂰 Use the language identifier (LANGID) parameter and the sort sequence (SRTSEQ)
parameter to access the unique-weight tables (*LANGIDUNQ) or the shared-weight tables
(*LANGIDSHR).
򐂰 When using the relative sort sequence, the relative weights shown in these tables differ
from the actual weights in the sort sequence table on the system. The relative weights
shown in these tables are examples only.
򐂰 The relative unique weight of a character is shown by the order of the characters in the
table. The relative unique weight is determined by assigning a weight of one to the first
character in the table and incrementing by one for each of the following characters until the
end of the table is reached.

Table 3 is a sampling of the common sort sequence for Latin-1. To see the complete table,
refer to the following Web address:
http://publib.boulder.ibm.com/cgi-bin/bookmgr/BOOKS/QB3AWC01/E.2.1"

The Latin-1 common sort sequence tables are used with the following languages:
򐂰 Albanian Afrikaans
򐂰 Brazilian Portuguese
򐂰 Belgian Dutch
򐂰 Belgian French
򐂰 Canadian French
򐂰 Catalan
򐂰 Dutch
򐂰 English Australia
򐂰 English Japan
򐂰 English UK
򐂰 English US
򐂰 English Uppercase
򐂰 French
򐂰 German
򐂰 Irish Gaelic
򐂰 Italian
򐂰 Portuguese
򐂰 Simplified Chinese
򐂰 Swiss French
򐂰 Swiss German
򐂰 Swiss Italian
򐂰 Traditional Chinese

Table 3 Sampling of the common sort sequence for Latin-1

GCGID* Character Shared Weight Unique Weight

SP100000 - 6 6

SP080000 ‘ 7 7

SP140000 ; 8 8

SP130000 . 9 9

SP150000 ? 12 12

SP060000 ( 27 27

SP070000 ) 28 28

SM060000 [ 29 29

4 Data Sorting Considerations with Lotus Enterprise Integrator (LEI) 6 on the IBM eServer iSeries Server
 GCGID* Character Shared Weight Unique Weight

SM080000 ] 30 30

SM050000 @ 37 37

SC040000 ¢ 39 39

SC030000 $ 40 40

SM070000 \ 46 47

SM300000 & 47 48

SM010000 # 48 49

SM020000 % 49 50

ND010000 1 64 69

ND020000 2 65 71

ND030000 3 66 73

ND040000 4 67 75

ND050000 5 68 76

LA010000 a 73 81

LA020000 A 73 82

LA270000 ä 73 90

LB010000 b 74 98

LB020000 B 74 99

LC010000 c 75 100

LC020000 C 75 101

LD010000 d 76 104

LD020000 D 76 104

As Table 3 demonstrates, the shared weights of all forms of an alpha character are the same,
that is, 'a' = 73, 'A' =73, and 'ä' =73. However, the unique weight is different, that is, 'a' = 81, 'A'
= 82, and 'ä'= 90. With this in mind, review Table 4 below, which is an example of a
shared-weight sort sequence, a unique weight sort sequence, and the binary sort sequence
for English code page 00037.

Table 4 Using the English language sort table sequence

Binary sort sequence Shared-weight sort sequence Unique-weight sort sequence
using LANGID(ENU) and using LANGID(ENU) and
SRTSEQ(*LANGIDSHR) SRTSEQ(*LANGIDUNQ)

Jones, Mary Johnson, John Johnson, John

JOHNSON, JOHN JONES, MARTIN Jones, Mary

JONES, MARTIN Jones, Mary JONES, MARTIN

Smith, Ron SMITH, ROBERT Smith, Ron

SMITH, ROBERT Smith, Ron SMITH, ROBERT

Data Sorting Considerations with Lotus Enterprise Integrator™ (LEI) 6 on the IBM iSeries Server 5
 The iSeries server ships with the sort sequence system value (QSRTSEQ) defaulted to
*HEX. The QNOTES user profile setting for SRTSEQ defaults to *SYSVAL, but it is overridden
by the LOCALE definition for *SRTSEQ, which is *HEX for all LOCALEs used by Domino for
iSeries. Figure 2 shows the default user profile settings.

Change User Profile (CHGUSRPRF)

Type choices, press Enter.

Sort sequence . . . . . . . . . SRTSEQ *SYSVAL

Library . . . . . . . . . . .
Language ID . . . . . . . . . . LANGID *SYSVAL
Country or region ID . . . . . . CNTRYID *SYSVAL
Coded character set ID . . . . . CCSID *SYSVAL
Character identifier control . . CHRIDCTL *SYSVAL
Locale job attributes . . . . . SETJOBATR *CCSID
*DATFMT
*DATSEP
*DECFMT
*SRTSEQ
*TIMSEP
Locale . . . . . . . . . . . . . LOCALE *SAME

User options . . . . . . . . . . USROPT *NONE

+ for more values
More...
F3=Exit F4=Prompt F5=Refresh F12=Cancel F13=How to use this display
F24=More keys

Figure 2 Default OS/400 user profile settings

To change the QNOTES user profile to use the *LANGIDSHR sort sequence, issue the
following Change User Profile command:

CHGUSRPRF QNOTES SRTSEQ(*LANGIDSHR) SETJOBATR(*CCSID *DATFMT *DATSEP

*DECFMT *TIMSEP)

This will force QNOTES to use the shared weight table that is specific to the language
specified by the LANGID setting for the QNOTES user profile, and prevents it from using the
*SRTSEQ setting specified by the LOCALE definition.

Note: The above setting has been tested and used when using Domino for iSeries and LEI
for iSeries. It is possible to make the same changes to a user profile that is used to access
iSeries UDB DB2 data from a WINNT-based server; however, this has not been tested.

It is possible to specify a specific table to use, and to even create a custom table; however,
this is rare, and should only be done by someone with considerable iSeries expertise.

Figure 3 is the LEI activity log entry for the previously listed activity, after the change was
made to the QNOTES user profile. As you can see, the replication conflicts no longer occur.

6 Data Sorting Considerations with Lotus Enterprise Integrator (LEI) 6 on the IBM eServer iSeries Server
 05/12/2003 04:08:20 PMActivity started
05/12/2003 04:08:22 PMCreated temporary Notes view '(LEIIndexView0)'
** Supply a view name to create a more efficient permanent view, Connector
'Qcustcdt.nsf Notes Conn', Method -Select-
05/12/2003 04:08:22 PMTrace Statement: <SELECT
CUSNUM,LSTNAM,INIT,STREET,CITY,STATE,ZIPCOD,CDTLMT,CHGCOD,BALDUE,CDTDUE FROM
mgilley.qcustcdt2 ORDER BY LSTNAM>, Connector 'RCHASSLH', Method -Select-
05/12/2003 04:08:23 PMActivity finished

Statistics
Elapsed Time: 00:00:02
Records Fetched From All Connectors: 12
Records Inserted From All Connectors: 0
Records Updated From All Connectors: 0
Records Removed From All Connectors: 0

Figure 3 LEI activity log after changing the QNOTES user profile

Table 5 lists the tables that are shipped with the iSeries server.

Table 5 Tables shipped with the iSeries server

Sort sequence SBCS CCSIDs Shared name Unique name

Arabic 00420 QARA01A4S QARA01A4U

Cyrillic 00880 QCYR0370S QCYR0370U

Cyrillic 0125 QCYR0401S QCYR0401U

Danish 00277 QDAN0115S QDAN0115U

Danish 00500 QDAN01F4U QDAN01F4U

Farsi 01097 QFAR0449S QFAR0449U

Greek 00423 QELL01A7S QELL01A7U

Greek 00875 QELL036BS QELL036BU

Hebrew 00424 QHEB01A8S QHEB01A8U

Icelandic 00871 QISL0367S QISL0367U

Icelandic 00500 QISL01F4S QISL01F4U

Japanese 00290 QJPN0122S QJPN0122U

Japanese 01027 QJPN0403S QJPN0403U

Korean 00833 QKOR0341S QKOR0341U

Latin-1 00037 QLA10025S QLA10025U

Latin-1 00256 QLA10100S QLA10100U

Latin-1 00273 QLA10111S QLA10111U

Latin-1 00280 QLA10118S QLA10118U

Latin-1 00284 QLA1011CS QLA1011CU

Latin-1 00285 QLA1011DS QLA1011DU

Data Sorting Considerations with Lotus Enterprise Integrator™ (LEI) 6 on the IBM iSeries Server 7
 Sort sequence SBCS CCSIDs Shared name Unique name

Latin-1 00297 QLA10129S QLA10129U

Latin-1 00500 QLA101F4S QLA101F4U

Latin-1 00836 QLA10344S QLA10344U

Latin-2 00870 QLA20366S QLA20366U

Norwegian 00277 QNOR0115S QNOR0115U

Norwegian 00500 QNOR01F4S QNOR01F4U

Russian 00880 QRUS0370S QRUS0370U

Russian 01025 QRUS0401S QRUS0401U

Spanish 00284 QESP011CS QESP011CU

Spanish 00500 QESP01F4S QESP01F4U

Swedish/Finnish 00278 QSNF0116S QSNF0116U

Swedish/Finnish 00500 QSNF01F4S QSNF01F4U

Thai 00838 QTHA0346S QTHA0346U

Turkish 00905 QTUR0389S QTUR0389U

Turkish 01026 QTRK0402S QTRK0402U

In summary, by default the iSeries server is not configured to sort data. This causes a
sort/order mismatch when retrieving data from iSeries UDB DB2 tables via Lotus Domino,
especially when using LEI and DECS. By changing the QNOTES user profile, and configuring
it to use a specific sort sequence, the issue can be addressed before it occurs. Because the
data is presorted, there is no need to use an order metaconnector against a DB2 connection
that references iSeries UDB DB2, therefore enhancing the overall performance of LEI.

References
The following manuals are provided here for additional reference:
򐂰 AS/400 National Language Support, SC41-5101-01
򐂰 AS/400 DB2 UDB for AS/400 SQL Programming Information, SC41-5611-04
򐂰 AS/400 DB2 UDB for AS/400 SQL Reference Information, SC41-5612-04
򐂰 AS/400 DB2 UDB for AS/400 Call Level Interface (ODBC) Information, SC41-5806-03

The team that wrote this Redpaper

This Redpaper was produced by a team of specialists from around the world working at the
International Technical Support Organization, Rochester Center.

Mike Gilley is a Software Engineer at the IBM Rochester Center. Mike has been in the
computer industry for 10 years and has been with IBM since November of 1997. He has been
working on the Domino for iSeries support team since 1999, specializing in DECS/LEI/DB2
integration with Domino. He currently holds a Lotus certification level of Principal Application
Developer, specializing in Maintaining Data Access with LEI.

8 Data Sorting Considerations with Lotus Enterprise Integrator (LEI) 6 on the IBM eServer iSeries Server
Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and
distribute these sample programs in any form without payment to IBM for the purposes of developing, using,
marketing, or distributing application programs conforming to IBM's application programming interfaces.

© Copyright IBM Corp. 2003. All rights reserved. 9

This document created or updated on July 2, 2003.

Send us your comments in one of the following ways: ®

򐂰 Use the online Contact us review redbook form found at:
ibm.com/redbooks
򐂰 Send your comments in an Internet note to:
redbook@us.ibm.com
򐂰 Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. JLU Building 107-2
3605 Highway 52N
Rochester, Minnesota 55901-7829 U.S.A.

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AS/400® OS/400® Domino™
DB2® eServer™ Lotus Enterprise Integrator™
IBM® ™ Lotus®
ibm.com® Redbooks™ Notes®
iSeries™ Redbooks(logo) ™

The following terms are trademarks of other companies:

Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other
countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems,
Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic
Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.

10 Data Sorting Considerations with Lotus Enterprise Integrator (LEI) 6 on the IBM eServer iSeries Server