Академический Документы
Профессиональный Документы
Культура Документы
Version 2007.1
Repository Guide
ATG
One Main Street
Cambridge, MA 02142
www.atg.com
Copyright
Copyright 1998-2007 Art Technology Group, Inc. All rights reserved.
This publication may not, in whole or in part, be copied, photocopied, translated, or reduced to any electronic medium or machine-readable
form for commercial use without prior consent, in writing, from Art Technology Group, Inc. (ATG) ATG does authorize you to copy
documents published by ATG on the World Wide Web for non-commercial uses within your organization only. In consideration of this
authorization, you agree that any copy of these documents which you make shall retain all copyright and other proprietary notices
contained herein.
Trademarks
ATG, Art Technology Group, and DYNAMO are registered trademarks of Art Technology Group, Inc.
ATG Wisdom, ATG Dynamo Application Server, ATG Adaptive Scenario Engine, ATG Scenario Personalization, ATG Portal, ATG Commerce,
ATG Content Administration, ATG Data Anywhere Architecture, ATG Search, ATG Response Management, ATG Merchandising, ATG
Knowledge, ATG Self Service, ATG Commerce Assist, ATG Advisor, ATG Forum and ATG Business Control Center are trademarks of Art
Technology Group, Inc.
Microsoft, Windows and Word are the trademarks or registered trademarks of Microsoft Corporation in the United States and other countries.
IBM, AIX, and MQ-Series are the trademarks or registered trademarks of IBM Corporation in the United States and other countries. Oracle is a
registered trademark, and other Oracle product names, service names; slogans or logos referenced in this document are trademarks or
registered trademarks of Oracle Corporation. Adobe Acrobat Reader is a registered trademark of Adobe. CORBA is a trademark of the OMG
(Object Management Group). Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems,
Inc. in the United States and other countries. Primus, and its respective logo, and Primus Knowledge Solutions, are trademarks, registered
trademarks, or service marks of Primus.
This product includes software developed by the Apache Software Foundation (http://www.apache.org/).
EditLive Authoring Software Copyright 2004 Ephox Corporation. All rights reserved. Includes code licensed from RSA Security, Inc.. Some
portions licensed from IBM are available at http://oss.software.ibm.com/icu4j/. Includes software developed by the Apache Software
Foundation (http://www.apache.org/). Contains spell checking software from Wintertree Software Inc.. The Sentry Spell Checker Engine
2000 Wintertree Software Inc..
All other product names, service marks, and trademarks mentioned herein are trademarks of their respective owners. This publication may
not, in whole or in part, be copied, photocopied, translated, or reduced to any electronic medium or machine-readable form for commercial
use without prior consent, in writing, from Art Technology Group (ATG), Inc. ATG does authorize you to copy documents published by ATG
on the World Wide Web for non-commercial uses within your organization only. In consideration of this authorization, you agree that any
copy of these documents which you make shall retain all copyright and other proprietary notices contained herein.
No Warranty
This documentation is provided as is without warranty of any kind, either expressed or implied, including, but not limited to, the implied
warranties of merchantability, fitness for a particular purpose, or non-infringement.
The contents of this publication could include technical inaccuracies or typographical errors. Changes are periodically added to the
information herein; these changes will be incorporated in the new editions of the publication. ATG may make improvements and/or changes
in the publication and/or product(s) described in the publication at any time without notice.
Limitation of Liability
In no event will ATG be liable for direct, indirect, special, incidental, economic, cover, or consequential damages arising out of the use of or
inability to use this documentation even if advised of the possibility of such damages. Some states do not allow the exclusion or limitation of
implied warranties or limitation of liability for incidental or consequential damages, so the above limitation or exclusion may not apply to
you.
ATG One Main Street Cambridge MA 02142
617.386.1000 phone 617.386.1111 fax www.atg.com
Contents
Introduction
13
Repository API
17
Repository Architecture
Repository Items
Item Descriptors
MutableRepository
Elements of the Repository API
atg.repository.Repository
atg.repository.RepositoryView
atg.repository.RepositoryItem
atg.repository.MutableRepository
atg.repository.PropertiesChangedEvent
Cloning Repository Items
Repository Queries
Repository Query API
atg.repository.QueryBuilder
atg.repository.QueryOptions
Repository Query Examples
Repository Queries in the ATG Control Center
Repository Query Language
RQL Overview
Comparison Queries
Text Comparison Queries
Property of Property Queries
Logical Operators
Collection Queries
Includes Item Queries
Is Null Queries
Count Expressions
All Query
Full Text Search Queries
ID-based Queries
Order By Directives
Range Directives
Parameters in Queries
18
18
19
21
21
21
22
23
23
25
26
29
29
29
30
31
34
34
35
35
36
36
37
37
38
38
38
38
38
39
40
40
41
iii
Contents
41
42
42
45
45
47
47
49
50
50
51
53
53
53
54
54
55
55
56
56
57
58
58
59
60
60
60
60
61
62
62
65
67
67
68
68
70
70
71
71
75
iv
Contents
77
77
78
80
80
80
81
81
82
83
83
85
86
87
87
88
88
89
90
90
92
92
92
93
94
95
95
95
98
99
101
101
102
103
103
104
104
105
105
105
107
107
108
108
109
v
Contents
109
110
111
112
112
113
115
115
116
116
117
118
119
119
120
120
123
123
124
125
125
125
126
126
126
127
127
127
128
128
129
129
130
130
130
132
133
133
135
135
136
136
137
137
vi
Contents
Cache Timeout
Cache Statistics
Cache Loading
Automatic Reloading
Loading with the load-items Tag
Loading with the query-items Tag
Loading with the dump-cache Tag
Cache Flushing
Cache Invalidation Service
Enabling the Cache Invalidator
Configuring the SQL-JMS System for Cache Invalidation
Invoking the Cache Invalidator Manually
Using the Cache Invalidator with Distributed Caching
Caching and Item Descriptor Inheritance
138
139
140
140
140
140
141
141
143
143
143
143
144
145
148
148
148
149
150
150
150
151
152
152
152
153
156
157
158
158
159
161
161
162
163
163
164
170
170
174
176
176
178
vii
Contents
expression Tag
rql-filter Tag
Named Query Tags
named-query Tag
rql-query Tag
rql Tag
sql-query Tag
sql Tag
input-parameter-types Tag
returns Tag
dependencies Tag
Development Operation Tags
transaction Tag
rollback-transaction Tag
import-items Tag
add-item Tag
update-item Tag
query-items Tag
print-item Tag
set-property Tag
remove-item Tag
remove-all-items Tag
export-items Tag
load-items Tag
dump-caches Tag
print-ddl Tag
Document Type Definition for SQL Repository Definition Files
Sample SQL Repository Definition Files
Repository Example A: Simple One-to-One
Repository Example B: One-to-One with Auxiliary Table
Repository Example C: One-to-Many with an Array
Repository Example D: One-to-Many with a Set
Repository Example E: One-to-Many with a Map
Repository Example F: One-to-Many Mapping to Other Repository Items
Repository Example G: Ordered One-to-Many
Repository Example H: Many-to-Many
Repository Example I: Multi-Column Repository IDs
Configuring the SQL Repository Component
Registering a Content Repository
SQL Repository Component Properties
178
178
178
179
179
179
179
180
180
180
180
180
181
182
182
182
183
184
184
185
186
186
187
187
188
188
189
196
196
197
198
200
201
202
204
205
206
208
209
209
221
221
222
223
223
225
viii
Contents
14 Repository Loader
Repository Loader Architecture
FileSystemMonitorService
FileSystemMonitorScheduler
LoaderManager
TypeMapper and TypeMappings
Content Handlers and Back Ends
User Interface Elements
Using the Repository Loader
Repository Loader Form Handlers and Administration Pages
Error Policies
Repository Loader RMI Client
RLClient Hints
Using a Repository Loader Manifest
Repository Loader Manifest File Document Type Definition
Repository Loader Manifest File Tags
Configuring the Repository Loader Components
Configuring the FileSystemMonitorService
Configuring the LoaderManager
Configuring the FileSystemMonitorScheduler
Configuring the TypeMapper
Configuring a TypeMapping
Configuring the Xml2RepositoryContentHandler
Choosing your Configuration
Repository Loader Example
User Example Item Type
Example Item Pathnames
Type Mappings and Content Handlers
Example TypeMapper
xml2repository Schemas
Running the RL Example
225
226
227
227
228
228
229
230
230
231
233
233
235
235
235
235
236
236
236
236
237
237
238
239
239
240
241
241
242
243
244
244
247
247
248
250
251
251
252
253
253
255
255
257
ix
Contents
16 Composite Repositories
Use Example
Primary and Contributing Item Descriptors
Item Inheritance and Composite Repositories
Transient Properties and Composite Repositories
Non-Serializable Items and Composite Repositories
Property Derivation
Configuring a Composite Repository
Property Mappings
Excluding Properties
Link Methods
Creating Composite and Contributing Items
Missing Contributing Items
Configuring the Composite Repository Component
Composite Repository Queries
Composite Repository Caching
Composite Repository Definition File Tag Reference
composite-repository-template Tag
header Tag
item-descriptor Tag
primary-item-descriptor Tag
contributing-item-descriptor Tag
attribute Tag
property Tag
primary-item-descriptor-link Tag
link-via-id Tag
link-via-property Tag
Composite Repository Document Type Definition
Sample Composite Repository Definition File
17 Secured Repositories
Secured Repository Features
Access Rights
Creating a Secured Repository
Modifying the Underlying Repository
Configuring the Secured Repository Adapter Component
Register the Secured Repository Adapter
Writing the Secured Repository Definition File
Group Membership
ACLs and Personae
ACL Syntax
Standard Access Rights
ACL Examples
259
261
263
263
263
264
264
264
264
265
265
266
266
266
267
267
268
269
269
269
269
270
272
273
273
274
275
276
276
277
279
285
285
286
287
288
289
290
290
291
292
292
293
293
x
Contents
18 LDAP Repositories
Overview: Setting Up an LDAP Repository
LDAP Directory Primer
Hierarchical Tree Structure
LDAP Data Representation
Hierarchical Entry Types
Directory Schema
LDAP and JNDI
LDAP Sources
LDAP Repository Architecture
LDAP Repository Items and Repository IDs
Item Descriptors and LDAP Object Classes
Item Descriptor Hierarchies and Inheritance
Id and ObjectClasses Properties
Additional Property Tag Attributes
New Item Creation
Repository Views in the LDAP Repository
Repository View Definition
LDAP Repository View Example
LDAP Repository Queries
ID Matching Queries
Unsupported Queries in the LDAP Repository
Configuring the LDAP Repository Components
/atg/adapter/ldap/LDAPRepository
/atg/adapter/ldap/InitialContextPool
/atg/adapter/ldap/InitialContextEnvironment
/atg/adapter/ldap/LDAPItemCache
/atg/adapter/ldap/LDAPItemCacheAdapter
/atg/adapter/ldap/LDAPQueryCache
/atg/adapter/ldap/LDAPQueryCacheAdapter
LDAP Password Encryption
294
294
294
295
295
296
296
296
297
297
297
298
301
305
305
307
308
308
309
310
310
311
312
313
313
314
314
316
318
319
320
321
321
322
323
323
324
324
325
326
327
329
330
330
331
331
xi
Contents
Index
332
332
332
332
333
333
334
334
335
336
338
338
339
340
340
341
342
347
xii
Contents
1 Introduction
Data access is a large part of most Internet applications. The ATG Data Anywhere Architecture provides a
unified view of content and data across a business for organizations and their customers. The core of the
ATG Data Anywhere Architecture is the Repository API. Through the Repository API, you can employ a
single approach to accessing disparate data types, including SQL databases, LDAP directories, content
management systems, and file systems.
The ATG Data Anywhere Architecture offers several advantages over the standard data access methods
such as Java Data Objects (JDO), Enterprise JavaBeans (EJB), and Java Database Connectivity (JDBC).
Among the differences:
1.
Data source independence The ATG Data Anywhere Architecture provides access to
relational database management systems, LDAP directories, and file systems using the
same interfaces. This insulates application developers from schema changes and also
storage mechanism. Data can even move from a relational database to an LDAP
directory without requiring re-coding. Java Data Objects support data source
independence, but it is up to vendors to provide an LDAP implementation.
2.
Fewer lines of Java code Less code leads to faster time-to-market and reduced
maintenance cost. Persistent data types created using ATG Data Anywhere are
described in an XML file, with no Java code required.
3.
Unified view of all customer interactions A unified view of customer data (gathered
using web applications, call center applications, and ERP systems) can be provided
without copying data into a central data source. This unified view of customer data
leads to a coherent and consistent customer experience.
4.
5.
6.
Fine-grained access control You can control who has access to which data at the
data type, data object, even down to the individual property using Access Control Lists
(ACLs).
7.
13
1 - Introduction
access. A development team is free to use EJBs along side of ATG technology, but the
easiest way to leverage investment in ATG technology is to follow the example set by
our solution sets. The ATG solution sets satisfy all of their data access needs using
repositories.
With the ATG Data Anywhere, the application logic created by developers uses the same approach to
interact with data regardless of the source of that data. One of the most powerful aspects of this
architecture is that the source of the data is hidden behind the ATG Repository abstraction. It would be
easy to change from a relational data source to an LDAP directory since none of the application logic
would need to change. Once data is retrieved from a data source it is transformed into an object-oriented
representation. Manipulation of the data can then be done using simple getPropertyValue and
setPropertyValue methods. The Repository API ties in closely with ATGs targeting APIs, so you can
retrieve items from the repository based on a variety of targeting rules, as well as retrieving specific
identified items.
The figure below provides a high-level overview of the ATG Data Anywhere Architecture.
14
1 - Introduction
15
1 - Introduction
16
1 - Introduction
2 Repository API
The ATG Repository API (atg.repository.*) is the foundation of persistent object storage, user
profiling, and content targeting in ATG products. A repository is a data access layer that defines a generic
representation of a data store. Application developers access data using this generic representation by
using only interfaces such as Repository and RepositoryItem. Repositories access the underlying data
storage device through a connector, which translates the request into whatever calls are needed to access
that particular data store. Connectors for relational databases and LDAP directories are provided out-ofthe-box. Connectors use an open, published interface, so additional custom connectors can be added if
necessary.
Developers use repositories to create, query, modify, and remove repository items. A repository item is
like a JavaBean, but its properties are determined dynamically at runtime. From the developers
perspective, the available properties in a particular repository item depend on the type of item they are
working with. One item might represent the user profile (name, address, phone number), while another
may represent the meta-data associated with a news article (author, keywords, synopsis).
The purpose of the Repository interface system is to provide a unified perspective for data access. For
example, developers can use targeting rules with the same syntax to find people or content.
Applications that use only the Repository interfaces to access data can interface to any number of backend data stores solely through configuration. Developers do not need to write a single interface or Java
class to add a new persistent data type to an application
Each repository connects to a single data store, but multiple repositories can coexist within ATG products,
with various applications and subsystems using different repositories or sharing the same repository.
Because of this approach, applications that use only the Repository API to access data can interface to any
number of back-end data stores solely through configuration. For example, the security system can be
directed to maintain its list of usernames and passwords in a SQL database by pointing the security
system at a SQL repository. Later, the security system can be changed to use an LDAP directory by
reconfiguring it to point to an LDAP repository. Which repositories you use depends on the data access
needs of your application, including the possible requirement to access data in a legacy data store.
The ATG platform includes the following models for repositories:
SQL Repositories, which use ATGs Generic SQL Adapter (GSA) connector to perform a
mapping between ATG and data stored in a SQL database. You can use a SQL
repository to access content, user profiles, application security information, and more.
SQL Profile Repository, included in the ATG Personalization module, which uses the
Generic SQL Adapter connector to perform a mapping for user data contained in a SQL
database. See the ATG Personalization Programming Guide.
17
2 - Repository API
LDAP Repository, which uses the ATG LDAP connector to access user data in an LDAP
directory. See the LDAP Repositories chapter.
Composite Repository, which provides a means for using more than one data store as
the source for a single repository. See the Composite Repositories chapter.
When you store content in a repository, in addition to the meta-information about a document, you need
access to the physical piece of content that is the document and path information that tells you where
the document is stored. To handle this problem, we have developed the content-specific repository
extensions located in the atg.repository.content package, which is described in the ATG API
Reference and in the SQL Content Repositories chapter in this ATG Repository Guide.
Repository Architecture
A data store may contain many diverse types of objects. The repository is not the data store itself; instead,
it is a collection of JavaBeans whose properties can be found and stored in the data store. The mission of
the repository is to provide a mechanism to retrieve the data elements and a run-time representation of
the available meta information about each object. This goal is achieved through three main conceptual
parts of the Repository API:
Repository Items
Item Descriptors
Repository Queries
For example, a repository might track elements of an organization. Each employee would have a
corresponding repository item, as would each department. An employee item descriptor would specify all
of the properties that an employee repository item could possess; a department item descriptor would
specify all the possible properties of a department. An application can build queries that return the
appropriate employee or department repository items as they are needed by the application.
Repository Items
A repository is a collection of repository items. In general, a repository item (a JavaBean component
implementing atg.repository.RepositoryItem or one of its sub-interfaces) corresponds to the
smallest uniquely identifiable entity in the underlying data store. In the SQL repository, for example, a
repository item often corresponds roughly to a row in a table. In the SQL profile repository, each user
profile is a repository item.
Each repository item is made of properties. These properties store the data that makes up a repository
item. Each property has a name, such as id, firstName, or lastName. In the SQL repository, these
properties correspond roughly to columns of a table. The properties available to a type of repository item
are defined in the repositorys item descriptors.
18
2 - Repository API
Each repository item must have an identifier, which is called a repository ID. The repository ID must
uniquely identify the repository item from all other repository items of the same type. The repository will
typically be configured to find the repository ID from some elements of the underlying data. In the SQL
repository, for instance, each item descriptor must specify the columns that act as the repository ID
(which will usually be the same as the tables primary key). Depending on the repositorys configuration,
the repository ID may or may not be exposed as a property of the repository item.
Properties of repository items may be single-valued or multi-valued. In some repository implementations,
such as the SQL repository, a propertys value may refer to one or more other repository items. This
enables a repository item to use properties that are complex data structures. It also lets a repository items
property refer to items in other repositories.
The combination of item descriptors, properties, identifiers, and items allows a repository to read
application data from the underlying data store, and to write application data back to the data source.
Some repositories allow certain properties or even entire item descriptors or repositories to be exposed as
read-only. Sometimes properties can even act as translators between the underlying data source and the
Java application. For example, you might have a database column that stores first names of users. Your
ATG application, however, wants to use first names that are exclusively uppercase. You can define a
repository property named firstNameUpperCase that takes the first name value from the database and
returns it as an uppercased version. The Repository API allows for this kind of flexibility without modifying
any application code.
Item Descriptors
Each repository item belongs to one of several item types. An item type describes all the properties that
are common to the repository items of that type. The Repository API allows you to perform a task similar
to JavaBean introspection to find out information about the properties of a repository item. Each item
type is described by a Repository item descriptor (also called a RepositoryView).
An item descriptor implements the atg.repository.RepositoryItemDescriptor interface and may
subclass atg.repository.ItemDescriptorImpl. The item descriptor gives a name to the type, and
also describes the properties for that type. The name of each property is defined in the item descriptor, as
well as the class of the Java object used to represent that type (Integer, String, etc.). The item descriptors
exposed by a repository depend on a combination of the underlying data store and the configuration of
the repository. In the SQL repository, for example, each database table might have its own repository item
descriptor. Another alternative might join multiple tables into a single item descriptor. Repositories can
support multiple item descriptors. For example, a SQL repository instance that supports a commerce
application might have different item descriptors representing users, products, product categories,
orders, etc.
Note the use of three closely related terms: item type, item descriptor, and Repository View. An item type
is a collection of repository items that share a common set of properties. For example, a bookstore
application might use item types for customers, books, and authors. Each item type is defined by an item
descriptor. There is a one-to-one correspondence between item descriptors and Repository Views.
The figure below shows an example of two item descriptors used by a repository that stores customer
information.
19
2 - Repository API
Inside each repository, there can be several types of items defined by item descriptors and for each item
type there can be several repository items. The definition of each type of item is described in a repository
definition file using XML. In this example, the Visitor Profile Repository defines two types of items (user
and address).
Developers can model relationships between types of items as shown in the following figure. In this
example, the user item type has a property named address. The value of the address property is a
repository item of another item type named address.
item-descriptor user
Repository Item Joe (id 101)
Repository Item Sue (id 102)
The ItemDescriptor mechanism is built upon ATG Dynamic Beans (atg.beans.*) system (described in
the Nucleus: Organizing JavaBean Components chapter of the ATG Programming Guide), which allows you
to describe properties for Java objects without defining the getX and setX methods for each property
required by the JavaBean specification. This interface is used to describe a set of dynamic properties that
occur together and have consistent behavior and semantics. An item descriptor essentially acts like a
BeanInfo where one can get access to the PropertyDescriptors that compose the repository item.
(For information about BeanInfos and PropertyDescriptors, see the JSDK 2 API documentation for
java.beans.BeanInfo and java.beans.PropertyDescriptor.)
Most repositories support simple property types such as Strings and Integers. ATG repositories can also
use the Java Collections Framework to model complex relationships between items using familiar objectoriented concepts. You can store a list of addresses as a Set, List, Map, or array, whichever make sense
for your applications needs. Some repositories support complex property types, in which the value of the
property is itself another repository item or collection of repository items. For example, a repository might
have item descriptors for types Person and Address. The Person item descriptor might have an
addresses property that exposes the list of Addresses that a Person might have. This property might be
of type RepositoryItem[], and the repository items in that array will use the Address item descriptor.
This allows repositories to represent one-to-one, one-to-many, or many-to-many relationships.
20
2 - Repository API
The information stored in ItemDescriptor components is usually not needed for development of your
ATG application. This property metadata is available for applications that need to expose a user interface
that allows one to explore and navigate the state of a repository. The ATG Control Center uses the
ItemDescriptors of the repository to automatically constrain the user interface with appropriate
selections.
MutableRepository
The base interfaces of a repository define an immutable data store. It provides a read-only version of the
elements contained in the repository. Extensions of the Repository interfaces provide facilities to create,
update, and remove items from a repository. See atg.repository.MutableRepository. The design goal for
updates was to allow transactional integrity across a set of changes in a high performance manner. Thus
when an item needs to be updated, a clone of the object is returned and changes are made to the cloned
object. Any modifications to this object are not observed by the repository until the object is submitted
for an update action.
Generally, implementations of repositories use caches to improve performance. Items retrieved out of a
repository either through a query process or directly by ID from the repository are cached. Typically, cache
policies are based on least recently used (LRU) design patterns or time-based expiration. See, for example,
the SQL Repository Caching chapter.
atg.repository.Repository
The atg.repository.Repository interface is the base definition of any repository implementation.
This interface provides methods to access RepositoryItems, RepositoryViews and
ItemDescriptors, corresponding to the three main elements of the repository.
Given a unique ID or set of IDs, you can retrieve items from the repository using the following methods:
21
2 - Repository API
Depending on the repository implementation, item IDs may take different forms. In SQL repositories, a
repository item ID by default is numeric and is auto-generated through the IdGenerator service. See ID
Generators in the Core Dynamo Services chapter of the ATG Programming Guide. The SQL repository also
supports composite repository item IDs. In that case, you can retrieve items from the repository using
these methods:
RepositoryItem getItem(CompositeKey pId, String pDescriptorName)
RepositoryItem [] getItems(CompositeKey [] pIds, String pDescriptorName)
In other cases, an item ID might be the path of the document, as in some of the file-system based
repositories.
The Repository API includes the RepositoryItemDescriptor interface, a subinterface of
atg.beans.DynamicBeanInfo (see Dynamic Beans: DynamicBeanInfo in the Nucleus: Organizing
JavaBean Components chapter of the ATG Programming Guide). This lets you access the dynamic bean
info of the available repository items, such as the property descriptors and property names, using this
method:
RepositoryItemDescriptor getItemDescriptor(String pName)
You can get the list of all the ItemDescriptors that are available from the itemDescriptorNames
property.
atg.repository.RepositoryView
If you do not have an exact repository ID, you can search for items in the repository through a
RepositoryView. Item descriptors and RepositoryViews often have a one-to-one relationship and
often have the same name. You can find out what views are available through the viewNames property of
the Repository component. This is useful if you need to build a system to navigate and view the entire
contents of a repository. The IDs for items in different item types may or may not overlap. There may be
no view that can return all items in the repository, but if there is, it will be the default view. If you need to
use the default view, you can use the view named by the defaultViewName property. Alternatively, you
can create properties for your own services that allow you to explicitly name the view your code is
interested in using. Once you have a name, you can retrieve that view through the RepositoryView
getView(String pName) method. From this returned object you can then build and execute a query.
The RepositoryView can also implement atg.repository.RepositoryViewContainer if the
repository needs to express a hierarchy of RepositoryViews. For example, a document management
system may have a root view for all documents. From that you may have sub-document types like
white papers, promo blurbs, images, etc. Further, the sub-view images may also have a refinement
for specific image types like JPEG and GIF. You can see what attributes are available for building
queries by accessing the itemDescriptor property of the RepositoryView. This describes all the
property information about all the items that can be returned by this view.
Sets of repository items can be gathered by queries, which you can build using the
atg.repository.QueryBuilder interface. This QueryBuilder object can be retrieved from the views
queryBuilder property. Once a query is built, each RepositoryView implementation translates the
22
2 - Repository API
internalized data structure into its native query language, for example SQL, and then return an array of
repository items that match the supplied query criteria. This is the mechanism by which the targeting
engine understands how to translate the rule syntax into the appropriate method calls of the
QueryBuilder.
Once a query is built from the QueryBuilder, it is executed via the various executeQuery methods
defined in the RepositoryView interface, such as:
RepositoryItem[] executeQuery(Query pQuery)
These overloaded executeQuery methods allow range queries and sorting criteria to be added to the
execution of the query. The methods return either:
an array of RepositoryItems, for elements that match the where clauses of the
query, or
atg.repository.RepositoryItem
The atg.repository.RepositoryItem interface is the immutable interface that represents an element
from a repository. Each RepositoryItem is uniquely identified through its repositoryId property. The
ItemDescriptor that describes the dynamic bean info about the item is available through the
itemDescriptor property. Given the repository item, you can also know what repository it came from
with the repository property. To retrieve the attributes of the RepositoryItem, use the
getPropertyValue(String pPropertyName) method. You can retrieve subproperties using the
DynamicBeans.getSubPropertyValue method, which takes a hierarchy property name of the form
propertyName1.subPropertyName2.subSubPropertyName3.
A Dynamic Bean property mapper has been registered for the RepositoryItem interface. This allows you
to reference the names of these properties as though they were JavaBean properties in the ATG Servlet
Bean syntax. See the Dynamic Beans section of the Nucleus: Organizing JavaBean Components chapter of
the ATG Programming Guide.
atg.repository.MutableRepository
Some repository services implement MutableRepository, a subclass of Repository. The SQL repository
implements this interface. The MutableRepository interface defines functions for four operations:
creating, adding, updating, and removing repository items.
createItem
There are two createItem methods:
createItem(String pDescriptorName)
createItem(String pId, String pDescriptorName).
Each of these requires a DescriptorName parameter, which should be the name of the RepositoryView
or ItemDescriptor that describes the repository item you wish to create. Each repository has a default
ItemDescriptor, which may allow your code to use the defaultViewName property of the repository to
23
2 - Repository API
supply this value. One of the createItem methods takes a potential unique ID to use for the
MutableRepositoryItem to create. If you do not supply an ID, one will be automatically generated and
guaranteed to be unique.
In the SQL profile repository, for example, the createItem methods return a transient instance of a
MutableRepositoryItem. At this point, the profile does not exist persistently in a data store. The item
exists only as the object reference you are returned. You may attempt to re-fetch the object (if the users
session has not expired or the server has not been restarted) through the getItem(String pId,
String pDescriptorName) method of the Repository (unless the
GSARepository.storeTransientItems property is set to false). Maintaining profile
RepositoryItems in RAM rather than in the profile database allows anonymous users to be represented
in the same Repository API, but does not hamper performance for handling requests for large sites. It
becomes untenable to attempt to create anonymous user database records for web sites that have a large
volume of users.
addItem
Once an item is created, at some later point you might want to turn it into a persistent repository item. To
do so, use the addItem method. This takes in the repository item that you want to add persistently:
RepositoryItem addItem(MutableRepositoryItem pItem)
removeItem
Removing an item from the repository is very easy. Pass the ID and ItemDescriptor name of the item
you want to remove persistently to the removeItem method. The items property values will be deleted
and will no longer be accessible from the repository:
removeItem(String pId, String pDescriptorName)
updateItem
The MutableRepository updates a repository item in a transactionally aware manner. It does not occur
like standard JavaBeans because we want to make sure the update operation in the backend data store
(such as a relational database) is efficient. Thus, updating an item takes three steps:
1.
Fetch a mutable version of the repository item through the getItemForUpdate and
getItemsForUpdate methods. These methods return instances of
MutableRepositoryItem. This interface extends RepositoryItem and adds one
method:
setPropertyValue(String pPropertyName, Object pPropertyValue)
2.
3.
Save the changes with the updateItem method. This method extracts all the changes
required for the item and updates the item in the data store. Depending on how you
have configured transactional behavior, the update can be committed immediately, or
the update may happen automatically when the associated transaction commits. See
24
2 - Repository API
Repositories and Transactions in the SQL Repository Architecture chapter. If there was
any type of error, a RepositoryException is thrown.
For example:
try {
RepositoryItem user = ... // get a reference to the user you want to update
MutableRepository mutableRepository = (MutableRepository)user.getRepository();
MutableRepositoryItem mutableUser =
mutableRepository.getItemForUpdate(user.getRepositoryId(),
user.getItemDescriptor().getItemDescriptorName());
mutableUser.setPropertyValue("name", "bob");
mutableUser.setPropertyValue("age", new Integer(26));
mutableRepository.updateItem(mutableUser);
}
catch (RepositoryException exc) {
// deal with exception
}
This same methodology should be applied for RAM-based RepositoryItems that you have created
through the createItem method. No database transaction will be performed, but the values will be
updated in the repository.
The Dynamo Application Framework (DAF) includes three classes that provide useful methods for dealing
with repository items:
atg.repository.servlet.RepositoryFormHandler
atg.userprofiling.ProfileForm
atg.userprofiling.ProfileFormHandler
See the Using Profiles and Profile Forms chapter in the ATG Page Developers Guide and the source code for
the ProfileForm and ProfileFormHandler classes, included in the ATG Personalization module
distribution in the <ATG2007.1dir>/DPS/src/Java/atg/userprofiling directory.
atg.repository.PropertiesChangedEvent
When a repository item is modified, its item descriptor broadcasts locally a PropertiesChangedEvent.
This event can be one of the following types:
Event Type
Description
DELETE
UPDATE
INSERT
25
2 - Repository API
CACHE_INVALIDATE
Properties
Description
item
The item that is changed. This is set to null if the item that was modified is
not currently in the cache. In that case, look at the repositoryId
property for the identity of the item that was changed.
repositoryId
itemDescriptorName
properties
If you have a component that you want to be notified when repository item properties change, it can
implement the atg.repository.PropertiesChangedListener interface. You can add your
PropertiesChangedListener implementation to the atg.repository.ItemDescriptorImpl
returned by the repository.getItemDescriptor() method, using the method
ItemDescriptorImpl.addPropertiesChangedListener.
26
2 - Repository API
Parameter
Description
pItem
Item to copy.
pDeepCopy
The mode of the copy. If true, the method creates a deep copy of the
item and its properties. Otherwise, the method creates a shallow copy,
only getting references of child RepositoryItems. Note that shallow
copying will only work if the source and destination repositories are the
same.
pPropExceptions
pExcludedProperties
Properties to exclude from the clone. Keys are item descriptor names
and the values are collections of property names to exclude. Optional.
pDestRepository
Repository to copy the new item into. If the source and destination
repositories are the same, properties that are items will be cloned to the
repository of the source item-property. Optional; if null, the new item is
copied to source repository.
pId
27
2 - Repository API
28
2 - Repository API
3 Repository Queries
All repositories have the ability to execute queries. A query is a request to find all of the items of a
particular item type that fits a certain set of criteria. Those criteria are specified in terms of the item types
properties. For example:
Find all Person items whose age property is greater than 30
The Repository API can express a wide variety of queries, including queries that match patterns in text,
query through collections, or even query through complex values. Queries can also specify the order in
which results should be returned, and can specify which results from a large result set should be returned.
For example, a query can express something like:
Find all Person items whose lastName starts with A,
whose interests includes biking, and
whose addresses property contains an Address with a zipCode of 02139.
Order the results by lastName, and return only items 10-20.
Queries can be built and executed using the Repository API. For complex queries, this can be a tedious
task. In these cases, or in cases where the Repository API should not be used directly, queries can be
represented in a textual form called Repository Query Language (RQL). See the Repository Query
Language section of this chapter for details of this language and other information about repository
queries. In most cases, however, repository queries can be constructed easily using targeting UI
components in the ATG Control Center.
atg.repository.QueryBuilder
The atg.repository.QueryBuilder interface defines the available query operations that repositories
should support. The QueryBuilder interface enables you to build Query objects that can be passed to
the repository. A Query is constructed from QueryExpressions. Each Query relates one or more
QueryExpressions and a query operation. There are standard logical query operations, such as AND, OR,
NOT, EQUALS, GREATER THAN, LESS THAN OR EQUALS, plus more complicated query operations like
collection inclusion, pattern matching, and others. The implementation of QueryBuilder does not have
29
3 - Repository Queries
to support all query operations it depends on what query features the data store supports. For
unsupported query operations, the method should throw a RepositoryException. You can use the
atg.repository.QueryOptions class to limit the size of a querys result set or otherwise modify the query.
2.
3.
4.
atg.repository.QueryOptions
You can use the atg.repository.QueryOptions class to specify ways that a query may be modified.
You can set the QueryOptions properties, and then pass the QueryOptions bean to the following
executeQuery method:
RepositoryItem[] executeQuery(Query pQuery, QueryOptions pQueryOptions);
The QueryOptions properties let you limit the size of the result set, direct how the result set should be
sorted, and pre-cache specified properties:
Property Name
Description
startingIndex
The index of the first element of a query result set that should
actually be returned. By setting startingIndex and endingIndex,
you can limit the size of the querys result set.
endingIndex
sortDirectives
30
3 - Repository Queries
precachedPropertyNames
2.
3.
4.
5.
31
3 - Repository Queries
The following example supposes we have an item descriptor named user with an integer property
named userType. This is how we might perform a simple query to find users whose userType property is
2.
import atg.repository.*;
MutableRepository pRepository =
(MutableRepository)ServletUtil.getCurrentRequest().resolveName
("/atg/userprofiling/ProfileAdapterRepository");
//
//
//
//
Lets expand the preceding example with a slightly more complicated query. The next code fragment
builds on the preceding fragment to create the query userType < 2 AND login STARTS WITH "j":
32
3 - Repository Queries
import atg.repository.*;
MutableRepository pRepository =
(MutableRepository)ServletUtil.getCurrentRequest().resolveName
("/atg/userprofiling/ProfileAdapterRepository");
// reuse the building blocks we have to create
// the "userType < 2" query
Query userTypeLTTwo =
userBuilder.createComparisonQuery(userType, two, QueryBuilder.LESS_THAN);
// create the "login STARTS WITH j" query
QueryExpression login =
userBuilder.createPropertyQueryExpression("login");
QueryExpression j =
userBuilder.createConstantQueryExpression("j");
//Note that we could make this query case-insensitive by adding another
//parameter to the createPatternMatchQuery, with a value of true
Query startsWithJ =
userBuilder.createPatternMatchQuery(login, j, QueryBuilder.STARTS_WITH);
// now AND the two pieces together. You can AND together as many
// Query pieces as you like: we only have two in our example
Query[] pieces = { userTypeLTTwo, startsWithJ };
Query andQuery = userBuilder.createAndQuery(pieces);
// execute the query and get the results
answer = userView.executeQuery(andQuery);
System.out.println("running query: userType < 2 AND login STARTS WITH j");
if (answer == null)
{
System.out.println("no items were found");
}
else
{
for (int i=0; i<answer.length; i++)
{
RepositoryItem item = answer[i];
String id = item.getRepositoryId();
String l = (String)item.getPropertyValue("login");
Integer a = (Integer)item.getPropertyValue("userType");
System.out.println("item: " + id + ", login=" + l + ", userType=" + a);
}
}
33
3 - Repository Queries
Note that the ATG Control Center limits the number of items that can be returned by such queries. This
limit is configurable and is set in the maxQueryCount property of /atg/devtools/RepositoryAgent.
The default value is 1000.
You can use RQL servlet beans to perform RQL queries in a page. DAF includes two
RQL servlet bean classes, atg.repository.servlet.RQLQueryForEach (Nucleus
address /atg/dynamo/droplet/RQLQueryForEach) and
atg.repository.servlet.RQLQueryRange (Nucleus address
/atg/dynamo/droplet/RQLQueryRange). See the descriptions of these servlet
beans in Appendix B: ATG Servlet Beans in the ATG Page Developers Guide.
You can define an RQL filter that is implicitly applied to all queries performed by the
repository. See Repository Filtering in the SQL Repository Queries chapter.
34
3 - Repository Queries
You can include RQL queries in <query-items> tags in the XML repository definition
file. This is useful principally for unit testing queries, but can also be used to pre-load
repository caches. See Querying Items in the Development, Testing and Debugging
with the SQL Repository chapter and Cache Loading in the SQL Repository Caching
chapter.
RQL Overview
RQL is a textual query syntax, similar to SQL. It describes the set of conditions that must be matched by
items of a particular item descriptor. The following is a simple RQL query that matches all items whose
age property is greater than 30.
age > 30
Notice that the name of the item descriptor is not included in the RQL query. The item descriptor is
usually implied by the context of the querys use.
All of the standard comparison operators can be used, as well as logical operators like AND, OR, and NOT:
age > 30 AND (lastName = "jones" OR paymentOverdue = true)
Constants such as 30, true, or jones can represent numbers, boolean, or String values. String values are
represented using Java syntax. They must be enclosed by quotes, and escape sequences for special
characters or UNICODE characters must use Java escape syntax.
Note that properties such as age or lastName must be the property names used in the repository. For
example, a database might have a column named phone, but that might be mapped to the repository
property primaryPhoneNumber. An RQL query must use the repositorys property name
primaryPhoneNumber, not the column name phone.
Also note that all RQL keywords (AND, OR, NOT, etc.) may be specified in either all upper-case or all lowercase. For example:
age > 30 and (lastName = "jones" or paymentOverdue = true)
RQL statements specify the conditions that an item must meet in order to be included in the result set. In
addition, an RQL statement may specify other directives to be applied to the result set, including ordering
of the results, or returning just a portion of the result set.
Comparison Queries
These are the simplest RQL queries, in which a propertys value is compared against another property
value, or against a constant. For example:
35
3 - Repository Queries
age > 30
All of the standard comparison operators can be used: =, !=, <, <=, >, >=. These operators can even be
applied to String properties and arguments, in which case ordering is determined by lexical order of the
Strings.
In general, these operators can only be used on properties that are scalar values. They shouldnt be used
on properties that are array or collection values.
When you use text comparison queries, be sure to enclose the comparison value in double quotes, as in
the examples above; otherwise, the RQL parser assumes the comparison term refers to a property name
rather than a property value.
The optional IGNORECASE directive can be applied to any of these queries to perform a case-insensitive
comparison:
firstName STARTS WITH IGNORECASE "h"
lastName ENDS WITH IGNORECASE "son"
phoneNumber CONTAINS IGNORECASE "33"
state EQUALS "utah"
Be aware, though, that negated pattern match queries have the potential to cause performance
problems. You should consider the queries you want to use and plan your database indexes accordingly
to avoid table scans. STARTS WITH and EQUALS queries can be optimized easily using database indexes.
The other pattern match queries generally cannot be. Likewise case-insensitive pattern matching may
affect the ability of the database to optimize the query.
36
3 - Repository Queries
address.zip = "48322"
This query means find all people whose address property points to an Address item whose zip code is
48322.
RQL allows for multiple levels of property-of-property expressions. For example,
department.manager.address.state.
Logical Operators
Any query expressions may be combined using the AND, OR, and NOT operators. Parentheses may be
used to affect grouping. NOT has the highest precedence, AND the next highest precedence, and OR has
the lowest precedence. For example, this expression:
name = "joe" OR NOT phone ENDS WITH "7" AND age > 30
Collection Queries
The above comparison operators, as well as the MATCH and MATCHES operators described below in the
Full Text Search Queries section, should only be applied to scalar properties. There is another set of
queries that may be applied to arrays or collections of scalar values, for example, properties of type
int[], or Set of Strings. Use the INCLUDES, INCLUDES ANY, or INCLUDES ALL operators only for querying
multi-valued properties.
The INCLUDES query matches items for whom the specified property includes the specified value. For
example:
interests INCLUDES "biking"
The INCLUDES query can also match one of a set of items. To do this, the ANY or ALL keyword must be
used, and the comma-separated set of items must be enclosed in braces. For example:
interests INCLUDES ANY { "biking", "swimming" }
While this:
interests INCLUDES ALL { "biking", "swimming" }
is equivalent to:
(interests INCLUDES "biking") AND (interests INCLUDES "swimming")
37
3 - Repository Queries
Some repositories support properties that are arrays or collections of items belonging to another (or the
same) item descriptor. For example, the addresses property might point to an array of items, which
themselves have address-related properties.
In this case, RQL allows for a subquery to be defined on these properties. For example:
addresses INCLUDES ITEM (zip = "48322" AND state = "MI")
This query means find all people whose list of addresses includes at least one address whose zip code is
48322 and whose state is MI.
Is Null Queries
Seeing whether an expression evaluates to null should be done with an IS NULL query. For example:
phoneNumber IS NULL
Count Expressions
The COUNT operator can be used to query on the size of a collection property. For example:
COUNT (addresses) > 3
This will find all people whose addresses property contains 4 or more elements.
All Query
An RQL query of ALL will return all of the items in a particular item descriptor. This should, of course, be
used with care since the result set could be very large. Usually this is combined with an ORDER BY or
RANGE directive (described below). The RQL query is simply:
ALL
This will return those items whose content matches mars in a full text search. (Content repositories allow
parts of the items data to be designated as content for the purposes of display and searching).
Another form of the query allows the full text search to proceed over a particular property:
38
3 - Repository Queries
Note that MATCH and MATCHES queries apply only to scalar properties.
Both forms of the query allow a special USING directive to pass special instructions to the underlying
search engine. The format of this directive depends on the repository and whatever search engine it is
using.
For example, to use the Sybase Full-Text Search Specialty Data Store, the query would look like this:
firstName MATCHES "abr" USING "SYBASE_SDS"
To use the Oracle ConText full text search engine, the query would look like this:
firstName MATCHES "abr" USING "ORACLE_CONTEXT"
ID-based Queries
RQL offers the ability to query items based on their repository IDs. This ability should be used with care,
since repository IDs are not portable across repository implementations.
The first query searches for items that match a set of IDs. For example:
ID IN { "0002421", "0002219", "0003244" }
The next ID-based query applies only to content repositories, in which items have been organized into
folders. This query restricts the search to only those items in the specified folders. The folders must be
specified by ID:
IN FOLDERS { "10224", "10923", "12332" }
Such a query would return an item with a composite repository ID of dept2:emp345. A query like this
would return items with any of the IDs dept2:emp345, dept2:emp346, or dept2:emp347:
ID IN { ["dept2", "emp345"], ["dept2", "emp346"], ["dept2", "emp347"] }
39
3 - Repository Queries
Order By Directives
Once a query has been defined using the above query elements, the result is a set of items. Using the
ORDER BY directive, the results may be ordered by the items properties. For example:
age > 30 ORDER BY firstName
This will return all people for whom age is greater than 30, with the results ordered by the firstName
property in ascending order. The results may be ordered in descending order by adding SORT DESC to the
end of the directive:
age > 30 ORDER BY firstName SORT DESC
The SORT ASC directive may also be used, but its unnecessary because its already the default.
The results may be ordered by multiple properties, each in ascending or descending order. For example:
age > 30 ORDER BY lastName, firstName SORT DESC
This will order the results by lastName. If multiple results have the same lastName, then within their
group they will be ordered by firstName in descending order.
A further directive, CASE IGNORECASE, may be used for case-insensitive sorting:
age > 30 ORDER BY firstName SORT ASC CASE IGNORECASE
Note that you can omit the tokens SORT and CASE, unless you are using parameters for the ASC/DESC or
USECASE/IGNORECASE tokens.
Range Directives
Many queries have the potential for returning large result sets. Most applications dont want to display
the entire result set - they may want to display just the first 10 results. Or they may want to page through
the results, showing results 0-9, then results 10-19, etc.
The RANGE directive is used to specify this in the RQL query. The RANGE directive must come after the
ORDER BY directive (if any). It has three forms. The first is the most common:
age > 30 RANGE +10
This causes only the first 10 results to be returned. If the result set is already less than 10, then all of the
results are returned.
The next form of the RANGE directive allows the results to start at a specified index:
age > 30 RANGE 10+
This causes the first 10 results to be skipped, and the remaining results to be returned.
The final form of the RANGE directive combines the above two forms, and is often used for paging:
40
3 - Repository Queries
This skips the first 40 results, then returns up to the next 10 results.
Parameters in Queries
In all of the above examples, the queries contain hard-coded constants, such as 30 or joe. Most of the
time, the actual values used in the query will not be known at the time the RQL statement is written. In
these cases, the values may be substituted with parameter expressions. For example:
age > ?0 AND firstName CONTAINS ?1 RANGE ?2+10
Every ?{number} represents a parameterized value that will be filled in when the query is executed. How
those values are supplied depends on the application performing the query. In the case of entity EJBs,
where RQL queries are used to represent finder methods, the parameters are filled in from the arguments
of the finder methods. For example, ?0 is substituted with the value of the first argument, ?1 with the
second, etc.
Parameter expressions can generally be used wherever constant values are used, including in RANGE
expressions. However, parameter expressions may not be used in array expressions, such as ID IN or IN
FOLDERS queries. In addition, parameter expressions may not be used as substitutes for property names all property names must be hard-coded into the RQL query when it is written.
In this example, only one object is passed into the query at runtime. However, this object is expected to
have two public member variables called name and age. The query will extract the values of these
member variables from the object and substitute those values for the ?0.name and ?0.age parameters.
Note that the fields must be public member variables of the object that is passed in, not JavaBean
properties. For example, the following object could be passed in to the query:
public class QuerySpecifier {
public String name;
public int age;
}
Parameterized Field Queries are used most often for entity EJBs, which allow primary key classes to
contain multiple fields. In this case, only one object will be passed to the query (the primary key), but if
the primary key spans multiple database fields, then the primary key object will contain the values of
those fields in its public member variables.
41
3 - Repository Queries
RQL Examples
The following example shows how you might use a parameter expression in Java code. It creates an
RqlStatement and then uses it in executing a query to find person repository items where the value of
the age property is greater than 23.
RqlStatement statement =
RqlStatement.parseRqlStatement("lastName STARTS WITH ?0");
Object params[] ={new String("m")};
items = statement.executeQuery (view, params);
Note how in the text comparison queries the comparison value "m" is enclosed in double quotes;
otherwise, the RQL parser assumes the comparison term refers to a property name rather than a property
value.
RQL Grammar
The following is a formal definition of the RQL grammar:
RQLStatement:: Query OrderByClause RangeClause
Query:: OR | AND | NOT | Comparison | ID IN | IN FOLDERS | ALL |
TextSearch | PropertyTextSearch | INCLUDES ITEM | IS NULL | (Query)
(Query)
NOT
AND
OR
42
3 - Repository Queries
IdIn:: ID IN StringArray
All:: ALL
43
3 - Repository Queries
44
3 - Repository Queries
The ATG SQL repository can be used to connect ATG applications to a SQL database. A SQL database
provides fast, scalable storage and retrieval of persistent information. The SQL repository works with a
SQL database to store objects and make those objects visible inside an ATG application as Dynamic Beans.
The uses of a SQL repository can be as varied as the uses of a relational database. The ATG platform
includes SQL repositories used to store:
User profiles (the Personalization modules SQL Profile Repository). See the SQL Profile
Repositories chapter in the ATG Personalization Programming Guide.
Content to be displayed on a Web site (the SQL content repository). See this chapter
and the SQL Content Repositories chapter.
Security profiles used by the Administrative Security system. See the Managing Access
Control chapter of the ATG Programming Guide.
The store catalog. See the Using and Extending the Default Catalog chapter in the ATG
Commerce Programming Guide.
In process orders. See the Configuring Purchase Process Services chapter in the ATG
Commerce Programming Guide.
Inventory. See the Inventory Framework chapter in the ATG Commerce Programming
Guide.
Gift lists and wish lists. See the Configuring Merchandising Services chapter in the ATG
Commerce Programming Guide.
Pricing and promotions. See the Using and Extending Pricing Services chapter in the ATG
Commerce Programming Guide.
The ATG platform includes a component at /atg/registry/ContentRepositories that lists all SQL
content repositories that have been registered with it.
Create the repository definition file for the SQL repository to use. This template is an
XML file that defines the item descriptors and repository item attributes contained in
your SQL repository and describes the relationship of your SQL repository to the SQL
45
4 - SQL Repository Overview
database. While the SQL repository is flexible enough to represent a variety of different
data models, it cannot easily represent any arbitrary data model. Therefore, it is usually
a good idea to design the SQL repository schema before you design your SQL
database schema.
The SQL Repository Data Models and SQL Repository Item Properties chapters
describe how to design your item descriptors and other SQL repository elements. See
also the SQL Repository Definition Tag Reference for full details of the XML tags used
to create the SQL repository definition file.
2.
3.
form of the SQL needed to create the database schema, and then edit the output to
optimize the database schema. See startSQLRepository Script and the Template Parser
in the Development, Testing and Debugging with the SQL Repository chapter.
46
4 - SQL Repository Overview
The SQL repository is a generalized and flexible implementation of the ATG Repository API that an
application can use to access data stored in a SQL database. See the Repository API chapter for more
information. The SQL repository is implemented through the atg.adapter.gsa package. (GSA stands for
Generic SQL Adapter.) The main ATG component in the SQL repository is an instance of the
atg.adapter.gsa.GSARepository class, which implements the interfaces
atg.repository.MutableRepository and atg.repository.content.ContentRepository and
which extends the class atg.repository.RepositoryImpl. Whenever you need to create a new SQL
repository instance, you should instantiate the atg.adapter.gsa.GSARepository class. The ATG API
Reference does not include Javadoc for this class and it is not intended that you access this class directly
from Java code. Normally, you should access all Repository functionality using the interfaces
atg.repository.Repository and atg.repository.MutableRepository. This will enable your
classes to work with any repository implementation for the greatest flexibility. Certain methods like those
for cache invalidation are defined on the class atg.repository.RepositoryImpl. It is anticipated that
future repositories will extend that class and so again you can make your code more reusable and
maintainable by accessing those methods on this base class rather than the implementation class of
atg.adapter.gsa.GSARepository.
The SQL repository uses an XML file called a repository definition file to describe the item descriptors that
make up a repository. The repository definition file also describes the relationships between the
repositorys item descriptors, repository items, and repository item properties, on the one hand, and the
SQL databases tables, rows, and columns. The XML tags that make up the repository definition file are
described in detail in the SQL Repository Reference chapter. The XML tags are also introduced in the
examples in this chapter and in the SQL Repository Item Properties chapter.
47
5 - SQL Repository Architecture
1.
2.
Call setPropertyValue.
3.
Commit JTA transaction. At this point, SQL is issued and the changes are committed.
2.
Call setPropertyValue.
3.
4.
Generally, you will want to call updateItem explicitly. This ensures that if you perform any queries
between the change made in the setPropertyValue call and the commitment of the transaction, those
queries will have the new property value available to them.
You can configure the ATG platform to send repository item cache invalidation messages to other remote
ATG servers. If you set the cache mode to distributed mode for an item descriptor, then cache
invalidation message will be set. Also, if you set the cache mode to locked mode for an item descriptor,
then when a server gives up ownership of the lock, it also invalidates the cache. See the SQL Repository
Caching chapter.
The SQL repository implements transaction isolation. The first time the item is accessed in a particular
transaction, either through the getItem() call, or the first attempt to call getPropertyValue() in an
item which was retrieved in a different transaction, we guarantee that it is up to date at that time. If the
item is changed by another transaction while this transaction is in progress, we dont see those changes
until a new transaction is started.
By default, a transaction will be created and committed for each method call. This is generally not the
most efficient way to handle repository item updates. It is generally most efficient to ensure that all of the
method calls in creating or updating a repository item are performed in a single transaction. The ATG
platform offers several different techniques for transaction demarcation that you can use to group
repository method calls into a single transaction. These are described in detail in the Transaction
Management chapter in the ATG Programming Guide.
One option is to use the Transaction servlet bean to explicitly create a transaction in a page. This servlet
bean is described in Appendix B: ATG Servlet Beans in the ATG Page Developers Guide. For example, the
following uses the current transaction, if any exists. If there is no current transaction, then one is created
before calling the output open parameter, then committed at the end of the droplet:
<droplet bean="/atg/dynamo/transaction/droplet/Transaction">
<param name="transAttribute" value="required">
<oparam name="output">
... do repository item work ...
</oparam>
</droplet>
48
5 - SQL Repository Architecture
You can also use the JTA (Java Transaction APIs) to explicitly manage the transaction. For example, you
might explicitly create a transaction around a repository item creation or update like this:
TransactionManager tm = ...
TransactionDemarcation td = new TransactionDemarcation ();
try {
try {
td.begin (tm);
... do repository item work ...
}
finally {
td.end ();
}
}
catch (TransactionDemarcationException exc) {
... handle the exception ...
}
If you are writing a FormHandler component, you can simply extend the class
atg.droplet.TransactionalFormHandler. This FormHandler automatically wraps a transaction
around all property get method calls called from a page and another transaction around all property set
or handle method calls made from a page. See the Working with Forms and Form Handlers chapter of the
ATG Programming Guide for more information.
<gsa-template>
<item-descriptor name="user" cache-mode="locked" item-cache-size="500">
<table name="dps_user">
49
5 - SQL Repository Architecture
adds a few options for the enumerated value of the userType property,
%flag; "false"
If your base SQL repository definition file sets the expert attribute of a property to true, and if
supplemental SQL repository definition files modify that property, you must also explicitly set the expert
attribute of a property to true in the supplemental SQL repository definition files; otherwise the
attributes value will revert to the default specified in the DTD.
50
5 - SQL Repository Architecture
DAF lets you represent this in a single repository using two independent types named book and author.
They are independent in that they do not share any properties in their item descriptors. They may happen
to each have properties like name or weight, but their properties are independently defined. Another way
to look at it is that they each have their own item descriptor.
The SQL repository also supports a simplified form of inheritance for item descriptors. See Item Descriptor
Inheritance in the SQL Repository Data Models chapter.
51
5 - SQL Repository Architecture
52
5 - SQL Repository Architecture
Repository items correspond to business objects, like customers, and elements of business objects, like a
customers shipping address. An item descriptor in a SQL repository defines a single type of repository
item. It specifies the properties of its repository items and the database tables and columns that store
these properties. This chapter describes how to define item descriptors and how to represent the
relationships between item descriptors in a SQL repository definition.
Note that a SQL repository cant necessarily work with any arbitrary SQL database model. The basic data
model patterns are described in the Sample SQL Repository Definition Files section of the SQL Repository
Reference chapter.
id Property
In order to retrieve a specific repository item, you need its repository ID. The primary table defines an id
property, which specifies the repository ID of repository items of this item type. You can specify this
property using the id-column-names attribute, and you do not need to further define the id property in
the item descriptor. If you do not define an id property in the item descriptor, then the id property must
use the default data-type, which is string. However, if you do explicitly define the id property in the
item descriptor, using a <property> tag, you can query repository items by their ID and you can set a
different data-type for the id property. The columns specified by the id-column-names attribute do not
have to use the same data-type as each other; you can have a composite repository ID the elements of
which are strings, integers, and longs.
After a repository item is created, but before it is added to the database, you can change its repository ID
by changing the value of the id property. Once the item has become persistent, you can no longer
change the ID.
53
6 - SQL Repository Data Models
A repository ID can be represented by a single column in the database, with a corresponding Java type of
String, Integer, or Long. A repository ID may also be represented by more than one column in the
database, each column of which can be either String, Integer, or Long. This type of repository ID is
referred to as a multi-column ID or composite key ID.
A single property can be used to represent a single column of a multi-column ID or might represent all of
the ID columns. So, both of the following are valid:
In this case, the folder property represents one of the ID columns, while the
document property represents the other:
<table name="doc" type="primary" id-column-names="folder_id,doc_id">
<property name="folder" column-names="folder_id" data-type="string"/>
<property name="document" column-names="doc_id" data-type="int"/>
</table>
Both single-column repository IDs and multi-column repository IDs are encoded as strings. By default, a
multi-column ID is encoded by concatenating the IDs elements, in the order specified by the item
descriptors id-column-names attribute, with each element separated by a separator character. By
default, this separator character is the colon (:). You can specify a different separator character using the
item descriptors id-separator attribute. For example, in an item descriptor defined like this:
<item-descriptor name="user" id-separator="*">
<table name="user" type="primary" id-column-names="dept_id,emp_id">
properties...
</table>
</item-descriptor>
you might have repository IDs that are string-encoded like this:
sales*bbanzai
You should not use brackets or commas for the separator character, since these characters are used by
RQL and the SQL repository when specifying lists of IDs.
54
6 - SQL Repository Data Models
primary-table-name.id-column-names
the default IdSpace would be named user. In an item descriptor with a composite repository ID defined
like this:
<table name="user" type="primary" id-column-names="dept_id,emp_id">
properties...
</table>
the default IdSpaces would be named user.dept_id and user.emp_id. In any case, you can override
the default IdSpace names using the id-space-names attribute in the item descriptor definition:
<table name="user" type="primary" id-column-names="dept_id,emp_id"
id-space-names="DepartmentId,EmployeeId">
properties...
</table>
See the ID Generators section of the Core Dynamo Services chapter in the ATG Programming Guide for more
information about ID space names and how they affect the IDs of newly generated items.
Auxiliary Tables
You can handle some data relationships using auxiliary attribute tables. For example, you could store
users and their addresses in two related database tables, as described in the following piece of an XML
repository definition:
55
6 - SQL Repository Data Models
<item-descriptor name="user">
Each user has a single address. For the purposes of this example, the user information is stored in a
separate table from the users address information.
Note that if you use auxiliary tables, each table definition, whether primary or not, must define an idcolumn-names attribute. This attribute defines the column names in the table that represent the
repository ID. This tells us how to join auxiliary tables to the primary table. The columns in the idcolumn-names attribute must be listed in the same order as they are in the id-column-names attribute
of the primary table.
References Constraints
In general, auxiliary and multi tables should not have REFERENCES constraints that point to each other.
Instead, each of these tables can have a REFERENCES constraint that points to the primary table for the
repository item. This limitation exists because the SQL repository processes insert and delete statements
for auxiliary and multi tables in the same order. As a result, if you specify REFERENCES constraints
between an auxiliary and a multi table or vice versa, a constraint error will result on either the insert or the
delete.
The column-names attribute of a property specifies the database column that stores the property. Only
id properties can have more than one column in their column-names attribute. Each property must also
define its data type, using the data-types attribute. In the case of a multi-column id property, the datatypes attribute is a comma-separated list of data types, each entry of which corresponds to an entry in the
column-names attribute. Each column can have a different data-type. If no data type is specified, the
string data type is used by default. The valid data type names and their Java and SQL representations are
56
6 - SQL Repository Data Models
listed in the Data Type Correspondences section of SQL Repository Definition Tag Reference in the SQL
Repository Reference chapter.
2.
Note that the multi-column-name attribute ensures that the ordering of the multivalues are maintained. The column specified by the multi-column-name attribute is
used for multi-valued properties of data-type array, map, and list and is not used for
sets (which are unordered). For map type properties, the values in the column
specifiedy by the multi-column-name attribute must be a string. For list or array type
properties, these values should be an integer or numeric type, and must be sequential.
3.
The multi-valued property in this table must have a data-type of array, set, map or
list:
<property name="..." column-name="interest" data-type="array" ...
4.
If the property is a collection of primitive data types (string, int, double, etc.), specify
the data type of the members of the collection by using the component-data-type
attribute in the <property> tag for the multi item property:
<property name="interests" column-name="interest" data-type="array"
component-data-type="string"/>
Note that the SQL repository does not support multi-valued collections of binary type
members.
5.
If the property is a collection of repository items defined by other item descriptors (for
example, an array of users), specify the repository item type of the members of the
collection by using the component-item-type attribute to the <property> tag for
the multi item property:
<property name="..." column-name="designers" data-type="array"
component-item-type="user"/>
57
6 - SQL Repository Data Models
The value of the component-item-type attribute is the name of the item descriptor
that defines the item type of the members of the collection of repository items.
6.
As with auxiliary tables, the ordering of the ID column names is important. The
columns in the id-column-names attribute must be listed in the same order as they
are in the id-column-names attribute of the primary table.
7.
The following example shows how a multi-valued attribute named interests can be expressed in the
XML repository definition:
<item-descriptor name="user">
<table name="dps_user" id-column-names="id" type="primary">
<property name="login" data-type="string"/>
</table>
<table name="dps_interest" type="multi" id-column-names="id"
multi-column-name="idx">
<property name="interests" column-name="interest" data-type="array"
component-data-type="string"/>
</table>
</item-descriptor>
See also the Sample SQL Repository Definition Files section in the SQL Repository Reference chapter for
more examples of one-to-many relationships in repository definitions.
Now you can use toUseElsewhere either in a writeObject call or in another setPropertyValue call.
Many-to-Many Relationships
You can also represent many-to-many data relationships in a SQL repository. For example, an author may
have written multiple books, and a book may have multiple authors. Representing this kind of
relationship depends on the type="multi" attribute in a <table> tag. You can represent a many-tomany relationship using two one-to-many relationships that point to the same intermediate table. The
58
6 - SQL Repository Data Models
following example represents a many-to-many relationship between the authors of a book and the books
written by an author. Author items use a primary database table named author, book items use a primary
database table named book, and both author and book items use a multi table named author_book as
an intermediate table that handles the relationship between authors and books.
<item-descriptor name="author">
<table type="primary" name="author">
...
</table
<table type="multi"
name="author_book"
id-column-names="author_id"/>
<property name="booksWritten" column-name="book_id"
data-type="set" component-item-type="book"/>
</table>
</item-descriptor>
<item-descriptor name="book">
<table type="primary" name="book">
...
</table
<table type="multi"
name="author_book"
id-column-names="book_id"/>
<property name="authors" column-name="author_id"
data-type="set" component-item-type="author"/>
</table>
</item-descriptor>
This example uses three tables, author, book, and author_book. The data type of the properties in the
intermediate multi table must be set, not array, map or list. Note also that tables can have columns other
than the ones referenced in the repository definition file, so long as such columns allow null values and so
long as there is no design requirement that the repository recognize the existence of such columns.
59
6 - SQL Repository Data Models
The SQL repository uses the cascade attribute in a <property> tag to better handle hierarchical
properties, which is to say properties with either the item-type or component-item-type attributes.
The cascade attribute can have one or more of the values insert, update, or delete. For example:
<property name="scenarios" item-type="scenario" cascade="update,delete"/>
Cascade Insert
If a repository item has a property with the item-type attribute and the cascade="insert" attribute
set, then when the item is created:
a new item of the type declared by the item-type attribute is also created; and
Cascade Update
If a repository item has a property that refers to other items and that has the cascade="update"
attribute set, then:
when you call addItem(), any new (transient) items referenced by this property are
added automatically to the repository; and
when you call updateItem, any referenced items that have been modified are
automatically updated. Any referenced items that are new (transient) items are added.
Cascade Delete
If a repository item has a property with the cascade="delete" attribute set, then when you remove the
repository item, any items that are referenced by the property will also be removed. Also, when you
remove a reference to this item, the item is automatically removed. You must take special care in using
cascade delete in one-to-many relationships. Do not use cascade="delete" in properties on the many
side of the relationship that refer to items on the one side of the relationship. The item on the one side
of the relationship cannot be deleted safely, since multiple items may be referring to it.
For example, suppose you have an item descriptor named company with an employee property that
references many repository items defined by an employee item descriptor. The employee item descriptor
itself defines a company property. In this one-to-many relationship, the employee property in the
company item descriptor could use cascade="delete". However, the company property in the
employee item descriptor should not use cascade="delete", since deleting one employee item would
then delete the company item that is referenced by the remaining employee items.
60
6 - SQL Repository Data Models
You can use one of the following three values in the cascadeDeleteOrder attribute tag:
Value
Description
first
afterAuxiliaryBeforePrimary
Cascade deletion is performed after deleting auxiliary multitable rows, but before deleting the primary table row. This is
the default behavior.
last
61
6 - SQL Repository Data Models
Cascade Example
For example, consider the following item descriptors, which define an author repository item type and
an address repository item type. The address property of the author repository item type has the
attribute item-type="address", indicating that the value of the address of an author is a repository
item of type address. The address property has the attribute cascade="insert,update,delete"; as
a result, whenever an author type repository item is created, added, updated, or deleted, the
corresponding address repository item will be also.
62
6 - SQL Repository Data Models
<property name="shippingWeight"/>
<property name="size"/>
<property name="season"/>
</table>
</item-descriptor>
<!-- The "shorts" item type -->
<item-descriptor name="shorts">
<table name="shorts" id-column-names="id" type="primary">
<property name="name"/>
<property name="description"/>
<property name="color"/>
<property name="shippingWeight"/>
<property name="size"/>
<property name="pleated" data-type="boolean"/>
</table>
</item-descriptor>
And the database data model would have two unrelated tables:
Coats and shorts have a lot of properties in common, which in the above model means
duplicated database columns and probably duplicated code.
This model does not allow you to easily perform a query like: find all the items of
clothing (shorts and coats) that have Star Wars in their description
An object-oriented approach like that used by the SQL repository allows you to define a base item
descriptor class called clothing to hold the attributes common to coats and shorts. You can use simple
inheritance to make coats and shorts subclasses of clothing. You can then model the data in your
clothing catalog like this:
and the corresponding XML repository definition (with changes in bold type) becomes:
63
6 - SQL Repository Data Models
Created the parent item descriptor, clothing, that holds attributes common to the
coat and shorts types.
Made the coat and shorts types into direct subclasses of the clothing type.
The clothing parent item descriptor has a sub-type-property attribute. This attribute defines the
property of the clothing type that specifies which sub-types can be instantiated. Each item descriptor
that extends the clothing item descriptor must specify a sub-type-value attribute, which is the value
for this property that triggers use of this particular type. The sub-type-value must not be null. This is
the value of the sub-type-property, which automatically implies this item type. If there is no subtype-value, this item-descriptor itself will never be returned from a getItem or query on one of its
super-type item descriptors. Such an item descriptor may have one or more item descriptors that have
it as its super-type and thus can serve as kind of an abstract item-descriptor.
It should be noted that instances of objects are associated with their superclasses by ID. So, in this
example, the ID of a coat always has a matching clothing ID.
64
6 - SQL Repository Data Models
As noted above, the sub-type-value of a sub-type item should never be null. If you have clothing items
that are neither coats nor shorts, you should create a sub-type-value in the clothing item descriptor
with a value of clothing and add a clothing option to the sub-type-property definition. For example:
From the Repository API point of view, each ItemDescriptor maps to a single RepositoryView. When
a SQL repository uses item type inheritance, each parent item type results in a
RepositoryViewContainer that contains its subtype views as children.
Most important, the items returned will be a combination of coat items and shorts items. Without SQL
repository support for inheritance, this query would have to be run against each subtype in turn. At first
blush this may not seem significant, but envision a store with perhaps ten subtypes like this. Running this
query against all of the subtypes would be painful. By building the inheritance knowledge into the SQL
repository, we enable it to optimize the actual SQL queries needed.
Here is what the code for this query would look like:
65
6 - SQL Repository Data Models
Query q = qb.createComparisonQuery(itemWeight,
weightLimit,
QueryBuilder.GREATER_THAN);
// run the query
RepositoryItem[] items = clothingView.executeQuery(q);
// separate the coats and shorts and do whatever with them
for (int i=0; i<items.length; i++)
{
RepositoryItem item = items[i];
// all clothing items have a name and a description
logDebug("clothing: " + item.getPropertyValue("name") +
' ' + item.getPropertyValue("description"));
// the ItemDescriptor defines the "type" of an item
RepositoryItemDescriptor desc = item.getItemDescriptor();
// now we do different things, depending on the
// type of clothing item we have
if (desc.getItemDescriptorName().equals("coat")
{
// coats have a property called "season"
logDebug("\tcoat, season = " + item.getPropertyValue("season"));
// do coat-related things
myCoatProcessor(item);
}
else
{
// shorts have a property called "pleated"
logDebug("\tcoat, season = " + item.getPropertyValue("pleated"));
// do shorts-related things
myShortsProcessor(item);
}
}
In this example, we used the name of the item descriptor to determine the item type. You can also look at
the value of the type property declared in your template. In our example, wed define the enumerated
properties with the useCodeForValue attribute set to true and then the query would go something like:
...
RepositoryItem item = items[i];
Integer itemTypeCode = (Integer)item.getPropertyValue("type");
66
6 - SQL Repository Data Models
if (itemTypeCode.intValue() == 0)
{
... coats ...
}
else
{
... shorts ...
}
Which technique to use is up to you and may be largely a matter of style. The item descriptor approach
uses the actual name like coat or shorts. The type attribute approach uses the type code stored in the
clothing table: typically something like 0 or 1, as in this case.
You can then create queries like these, even though the products item descriptor does not include
waterproofRating, size, or channelCount properties:
products whose waterproofRating is 'JIS-4'
products whose channelCount = 7
products whose waterproofRating is 'JIS-4' OR whose size > 7
67
6 - SQL Repository Data Models
However, since these item descriptors will share the same tables (and thus the same data), it may not be
wise to use both the old and new item descriptors. Instead, it may be preferable to use the super-type
attribute and sub-type-property attribute if you want to use both.
A type can only inherit from one parent. No multiple inheritance is allowed.
A class hierarchy can only have a single sub-type-property value. You can define a
second level of sub-classing -- for example, you might define an item descriptor
named bermuda-shorts that has shorts as its super-type-- but you cannot have
another different sub-type-property.
All parent item descriptors (item descriptors that are used in super-type or copy-from attributes) must
be fully defined by the time they are referenced in the XML repository definition file. They can either be
defined in front of the new XML file in the same file or specified in an XML file that is parsed before this
XML file.
You should avoid using too many levels of inheritance. Queries against items whose properties span
multiple sub-types may require joins of all of the tables in the hierarchy. If you use these kinds of queries,
keep in mind that performance decreases as the number of tables joined increases.
Derived Properties
In a SQL repository, you can use derived properties. This feature enables one repository item to derive
property values from another repository item or from another property in the same repository item. To
illustrate: some data models are organized in a tree structure in which certain property values are passed
down from other properties. For example, an organization might have divisions, departments, and
employees, organized in a tree structure. A repository represents this tree structure with division,
department, and employee item descriptors. Each of these item descriptors might define a property
called spendingLimit. A business rule might specify that an employees spending limit comes from their
department if it isnt set for that employee. If the spending limit is not set for the department then it
should be derived from the spending limit for the division.
This derived property relationship would be represented in a repository definition file like this:
<item-descriptor name="employee">
<property name="department" item-type="department"/>
<property name="empSpendingLimit" data-type="int"/>
<property name="spendingLimit" writable="false">
<derivation>
<expression>empSpendingLimit</expression>
<expression>department.spendingLimit</expression>
</derivation>
</property>
68
6 - SQL Repository Data Models
</item-descriptor>
<item-descriptor name="department">
<property name="division" item-type="division"/>
<property name="deptSpendingLimit" data-type="int"/>
<property name="spendingLimit" writable="false">
<derivation>
<expression>deptSpendingLimit</expression>
<expression>division.divSpendingLimit</expression>
</derivation>
</property>
</item-descriptor>
<item-descriptor name="division">
<property name="division" item-type="division"/>
<property name="divSpendingLimit" data-type="int"/>
</item-descriptor>
Derived properties can use multiple levels of subproperties. So in our spending limit example, we might
derive the employees spending limit this way:
<item-descriptor name="employee">
<property name="department" item-type="department"/>
<property name="spendingLimit" data-type="int" writable="false">
<derivation>
<expression>department.employeeDefaultInfo.spendingLimit</expression>
</derivation>
</property>
</item-descriptor>
<item-descriptor name="department">
<property name="employeeDefaultInfo" item-type="employeeInfo"/>
<property name="deptSpendingLimit" data-type="int"/>
</item-descriptor>
<item-descriptor name="employeeInfo">
<property name="spendingLimit" data-type="int" writable="false"/>
<property name="officeLocation" data-type="string"/>
</item-descriptor>
Bear in mind that using derived properties can affect performance, and that the more complex the
derivation, the greater the impact is likely to be.
The FirstNonNull derivation method must be non-writable, unless you set a writable override property
for the derived property. You can set a property to be not writable like this:
<property name="spendingLimit" data-type="int" writable="false"/>
69
6 - SQL Repository Data Models
The FirstWithAttribute and FirstWithLocale derivation methods can be writable, even if the
property does not define a writable override property. See Override Properties in this section for more
information.
Override Properties
There are times when you might want to explicitly set a property value rather than have the property
derivation logic supply one. This is done by specifying an override-property attribute in the
<derivation> tag:
<item-descriptor name="employee">
<property name="department" item-type="department"/>
<property name="empSpendingLimit" data-type="int"/>
<property name="spendingLimit">
<derivation override-property="empSpendingLimit">
<expression>department.spendingLimit</expression>
</derivation>
</property>
</item-descriptor>
The above change specifies that if the empSpendingLimit property is not null then it is used as the value
of the spendingLimit property, otherwise the spendingLimit property is derived as before. Using the
override-property attribute lets you edit in a Repository Editor a property that would otherwise be
derived from another property.
<item-descriptor name="user">
<property name="shipToAddress" writable="false" item-type="address">
<derivation>
<expression>shippingAddress</expression>
<expression>homeAddress</expression>
</derivation>
</property>
<table name="user" >
<property name="shippingAddress" item-type="address">
<property name="homeAddress" item-type="address">
</table>
</item-descriptor>
70
6 - SQL Repository Data Models
Complex Derivations
The examples given so far show a comparatively simple tree model, but actually a repository item
property can be derived from any other properties, so long as the properties can be represented as
vertices on a path of a directed acyclic graph. That is, the derivation of a property can have multiple paths
that are traversed to derive the value. For example, a shipping address might derive from the users
specified shipping address, the users billing address or home address, or the users companys address:
<item-descriptor name="user">
<property name="shipToAddress" writable="false" item-type="address">
<derivation>
<expression>shippingAddress</expression>
<expression>billingAddress</expression>
<expression>homeAddress</expression>
<expression>company.address</expression>
</derivation>
</property>
<table name="user" >
<property name="shippingAddress" item-type="address"/>
<property name="billingAddress" item-type="address"/>
<property name="homeAddress" item-type="address"/>
<property name="company" item-type="company"/>
</table>
</item-descriptor>
Note that this example assumes youve also defined item descriptors named address and company. To
determine the value of the shipToAddress for a user, the expressions specified in the derivation are
searched in order. Any expression may also refer to properties that are themselves derived.
Derivation Methods
A derived property definition can specify one of several different derivation methods to determine the
appropriate property value. The SQL repository traverses in order each of the expressions in the
<derivation> tag, applying the specified derivation method. There are six derivation methods included
in the ATG platform:
71
6 - SQL Repository Data Models
By default, the SQL repository derives a property by traversing the expressions in order, starting with the
property itself. The first non-null value found is used as the property value. This is the firstNonNull
derivation method.
The firstNonNull method is the default derivation method, and so it is not necessary to specify it in the
XML. However, the derivation method can be specified in the method attribute of a <derivation> tag in
the SQL repository definition file, as in this example:
<item-descriptor name="employee">
<property name="department" item-type="department"/>
<property name="empSpendingLimit" data-type="int"/>
<property name="spendingLimit" writable="false">
<derivation method="firstNonNull">
<expression>empSpendingLimit</expression>
<expression>department.spendingLimit</expression>
</derivation>
</property>
</item-descriptor>
<item-desciptor name="myItem">
<property name="name">
<derivation method="firstWithAttribute">
<expression>englishName</expression>
<expression>icelandicName</expression>
<expression>shonaName</expression>
</derivation>
<attribute name="derivationAttribute" value="language"/>
<attribute name="defaultKey" value="en"/>
</property>
<property name="englishName">
<attribute name="language" value="en"/>
</property>
<property name="icelandicName">
<attribute name="language" value="is"/>
</property>
<property name="shonaName">
72
6 - SQL Repository Data Models
If getKey returns sn (the user is in Zimbabwe, for example) then myItem.name will return the same value
as myItem.shonaName.
Gets the users current locale as the key from the Nucleus component defined by a
keyService attribute.
2.
3.
The locale is searched in a locale-specific way. For example, if locale=fr_FR_EURO, it first looks for a
property where the locale attribute is fr_FR_EURO, then looks for fr_FR, and finally looks for fr.
There is also a defaultKey, which the keyService uses if the value with the real key is null. In other
words, if the real key is de_DE and we are looking for displayName, but displayName_de is null, then
return displayName_en instead (assuming its locale is en and the defaultKey is en or en_US).
Using a defaultKey may slow performance. If no default key is defined, it is not used. If the default key is
the same as the current key, there are no performance implications. In all other cases, there is an extra
clause on all search terms, which can result in a slower search.
Here is an example of a derived property definition using the firstWithLocale derivation method:
73
6 - SQL Repository Data Models
For example, suppose an item descriptor defines a property named firstName. You want some
application code to be able to refer to this property as name1. You can use the Alias derivation method to
define name1 to be the equivalent of firstName, as follows:
<item-descriptor name="user" ...>
<table name="USER" ...>
<property name="firstName" ...>
....
</table>
<property name="name1">
<derivation method="alias">
<expression>firstName</expression>
</derivation>
</property>
In this example, when the name1 property is accessed, the firstName property of the item will be
returned.
The siblings property represents a union of values in the sets brothers and sisters. The data type of
the values in the collections defined in all the expressions of the derived property must be the same.
If two or more of the properties to be combined in the Union derived property include the same element,
then the Union derived property will have duplicate elements if the property is of type list, but will have
unique elements if the property is of type set.
74
6 - SQL Repository Data Models
a catalogs property that is derived from a union of the catalogs properties of the items that make up
the parentProducts property.
<item-descriptor name="sku" ...>
<property name="catalogs">
<derivation method="collectiveUnion">
<expression>parentProducts</expression>
</derivation>
<attribute name="collectionProperties" value="catalogs"/>
</property>
<table name="sku_prod" ... >
<property name="parentProducts"
data-type="set" component-item-type="product">
...
</table>
</item-descriptor>
<item-descriptor name="product" ...>
<table name="prod_com" ... >
<property name="catalogs" ... />
...
</table>
</item-descriptor>
In this example, the union of product.catalogs will be returned for each product in the
parentProducts property of the sku item. The derived property will be accessible using the catalogs
property of the sku item.
75
6 - SQL Repository Data Models
<property ...>
<attribute name="serialize" value="true"/>
</property>
If the item is persistent, its persistent properties will have been updated in the database already, and can
be retrieved from the database if the session needs to be restored.
76
6 - SQL Repository Data Models
An item descriptor in a SQL repository can define various useful special types of properties. The following
section describe some of these special property types, as well as other useful repository property
attributes:
Enumerated Properties
Required Properties
Unique Properties
Date and Timestamp Properties
Null Properties
Property Validation with a Property Editor Class
Maintaining Item Concurrency with the Version Property
Repository Items as Properties
Transient Properties
Assigning FeatureDescriptorValues with the <attribute> Tag
Linking between Repositories
SQL Types and Repository Data Types
User-Defined Property Types
Property Fetching
Handling Large Database Columns
Enumerated Properties
Properties of repository items can have a data type of enumerated. Enumerated properties are string
properties constrained to a predefined list of valid values and stored as integer codes in the database. A
TaggedPropertyEditor is registered for enumerated properties so that components like user interfaces
can access the list of valid values. For these reasons, the list of valid values should be small. For example, it
is not appropriate to use an enumerated attribute for something that could take on more than a hundred
values. Instead, you might make a reference to another item with a single int property.
Here is an example of an item descriptor definition that creates an enumerated property named
transactionType:
77
7 - SQL Repository Item Properties
<item-descriptor name="transaction">
<table name="transaction" id-column-names="xact_id">
<property name="amount" data-type="int"/>
<property name="transactionType" data-type="enumerated">
<option value="credit"/>
<option value="debit"/>
<option value="purchase"/>
</property>
</table>
</item-descriptor>
In this example the list of valid String values is specified explicitly and the corresponding integer codes
are generated by the SQL repository when the template is initialized. It is also possible to specify the
integer codes explicitly, using the code attribute in the <option> tag:
By default, an enumerated property returns its value as an integer code. You can instead configure an
enumerated property so that the repository converts the integer code into a string value. Do this by using
the useCodeForValue attribute in the property definition, as in the following example:
<property name="gender" data-type="enumerated">
<attribute name="useCodeForValue" value="false"/>
<option value="male" code="0"/>
<option value="female" code="1"/>
</property>
In the above property example, with useCodeForValue set to false, then if you get the gender property,
the string male or female is returned. With useCodeForValue set to true, then the integer code 0 or 1
would be returned instead. If your enumerated property returns an integer code, you can get the
property editor for your enumerated property and use that to create a property editor that can convert
between the string value and the integer code value. See the JavaBeans specification for a description of
PropertyEditors.
78
7 - SQL Repository Item Properties
In order to avoid collisions between option codes used by your repository definition and any option
codes used in subsequent ATG products, you should observe the following convention in assigning
option codes to enumerated property values:
Project
customer use
DAF
1000 - 1999
DPS
2000 - 2999
DSS
3000 - 3999
4000 - 4999
5000 - 5999
ATG Portal
6000 - 6999
ATG Outreach
7000 - 7999
8000 - 8999
ATG Ticketing
9000 - 9999
10000 - 10999
11000 - 11999
12000 - 12999
ATG Search
13000 - 13999
Agent
14000 - 14999
ATG Chat
15000 - 15999
16000 +
In the unlikely event that you need more than 1000 separate values for a single enumerated property, you
could also use negative integers as option code values.
79
7 - SQL Repository Item Properties
Required Properties
You can specify that a repository item property is required, using the required attribute in a property
tag:
<property name="lastName" data-type="string" required="true" />
Then, if you also set the enforceRequiredProperties property of the GSARepository component to
true (the default value), the repository checks to make sure all required properties are present when
adding repository items and forbids the setting of a required property to null.
Make sure your repository definition corresponds to your database schema; if your database schema
defines a property as NOT NULL, your repository definition should define the property with the
required="true" attribute.
Note: If a property references an item that is defined in the database as NOT NULL but you cannot mark
the property as required, indicate this by adding the references attribute tag and set its value to true,
as in the following example:
<property name="myProperty">
<attribute name="references" value="true" data-type="boolean"/>
</property>
Unique Properties
You may want a repository item property to require a unique value. For example, you probably want each
user profile to contain a login property that is unique to that profile and not shared by any other. You can
mark a property as unique using the <attribute> tag, as in the following example:
<property name="login" data-type="string" required="true">
<attribute name="unique" value="true"/>
</property>
Repository editors in the ATG Control Center enforce the requirement that the value be unique.
80
7 - SQL Repository Item Properties
For more information about this technique, see the Assigning FeatureDescriptorValues with the
<attribute> Tag section in this chapter.
Last-Modified Properties
In some applications, it is useful to know when a repository item has been modified most recently. You
can set up a special property to store the last-modified time of a repository item. To configure a lastmodified property:
1.
2.
Add the last-modified-property attribute to the item descriptor and set its value
to the name of your last-modified property. For example:
<item-descriptor name="article" last-modified-property="lastActivity">
3.
The item descriptor might then look, in relevant part, like this:
When items of this type are added or updated, the lastActivity property will be updated with the
current time.
Null Properties
If a propertys definition does not set the default value of a property, then its default value is null. For
example, the following <property> tag defines a property named favoriteColor whose default value
is null:
81
7 - SQL Repository Item Properties
If you want to affirmatively set a default value to null, use this form:
<property name="favoriteColor" data-type="string" default="__NULL__"/>
This technique is useful if, for example, you are combining two or more repository definition files into a
single template and need to override a non-null value and restore the default to null.
Sorting Properties
You can use the category attribute in a <property> tag to group similar properties together in a
Repository user interface. All properties with the same category attribute value are listed together. For
example, Login name and Password are listed together under the heading Login. If you examine the
<property> tags that define the login and password properties, you will see that each <property> tag
uses the attribute category="Login".
If you examine the Basics category in the ATG Control Center user profile display, you will see the
properties First name, Middle name, and Last name listed in that order. The order these properties appear
in the ATG Control Center depends on the propertySortPriority attribute. Properties with the same
category attribute are listed in ascending order according to the integer value of their
propertySortPriority attributes. Properties that do not have a propertySortPriority attribute are
listed in alphabetical order.
For example:
<property category="Basics" name="firstName" data-type="string"
display-name="First name">
<attribute name="propertySortPriority" value="-3"/>
</property>
Category Ordering
Categories are listed in the ACC in the following order:
1.
All named categories that contain required properties (properties with the attribute
required="true").
2.
3.
A special anonymous category containing properties that are not assigned to any
category.
4.
Within each of these four sets, categories are listed in alphabetical order.
82
7 - SQL Repository Item Properties
Property Ordering
Properties are listed together with other properties that have the same category attribute. Within a
category, properties are listed in ascending order, according to the value of their
propertySortPriority attribute. The default value for propertySortPriority is 0. Properties that
have the same propertySortPriority value are listed in alphabetical order, according to their
displayName attribute. The propertySortPriority attribute is set in an <attribute> tag that is a
child tag of the <property> tag, as in this example:
<property category="Basics" name="firstName" data-type="string"
display-name="First name">
<attribute name="propertySortPriority" value="-3"/>
</property>
A property editor can be used to validate a property value. Note, however, that a property editor doesnt
have access to the repository item, only the property. Therefore, you cant use a property editor to make
comparisons with other properties.
You can also limit a propertys values to a list of valid choices using an enumerated property. See the
Enumerated Properties section.
83
7 - SQL Repository Item Properties
</table>
</item-descriptor>
The value of the version property is incremented every time the item is updated. Its value starts as 0 when
the item is created, is set to 1 when the item is added, and then is incremented in each subsequent
update.
The version number for a particular item is read and associated with that transaction the first time that
item is referenced in a transaction. If you attempt to update the item from a transaction whose version
does not match the current version number in the database, a ConcurrentUpdateException is thrown
to abort that update. This exception is a subclass of RepositoryException.
Here is a sample scenario that shows how the SQL repository uses the version property to implement
optimistic locking:
1.
Dynamo1 reads a repository item for update. It obtains the items version property,
which has a value, lets say, of 2.
2.
Dynamo2 reads the same repository item for update. Since Dynamo1 has not yet
committed any changes to the item, Dynamo2 gets the same value for the items
version property, 2.
3.
Dynamo1 updates the repository item. In the course of the update, the value for the
version property in the repository item is checked to see whether it is the same as
what is found in the corresponding database column. In this case, both values are still
2. The update to the repository item is committed, with the version property
incremented by 1, so that the value of the version property is now 3.
4.
Dynamo2 attempts to update the repository item. When the value for the version
property in the repository item is checked to see whether it is the same as what is
found in the corresponding database column, the values do not match. Dynamo2 is
holding a value of 2, while the value of the version property in the database is now 3.
Dynamo2 throws a ConcurrentUpdateException and does not apply the changes in
the update.
This can be very useful for simple and distributed cache modes where there is a possibility of overwriting
another Dynamos changes.
You can take advantage of optimistic locking in pages that include forms. Often in a form, you read the
data in one transaction and update the data in another transaction. There is a possibility that another
process might try to update an item in an intermediate transaction. To handle this case, you can place the
version property value as a hidden field in your form. Then, you can either check that it is still the same
yourself after you start the transaction which updates the item, or you just set the version property (along
with the other properties in the item) and deal with the ConcurrentUpdateException when it occurs.
For example, you can include in a page a hidden input tag like this:
<input type="hidden" bean="FormHandler.value.version">
You can also use a RepositoryFormHandler, which can set a version property just like it sets any other
property of a repository item.
84
7 - SQL Repository Item Properties
Without support for objects as properties, the application would have to get an authorId from the book
and then perform another lookup to get the actual author.
You can specify that a repository item property is another repository item, rather than a primitive, using
the item-type attribute, rather than the data-type attribute. The following example shows a portion of
a template that defines two item descriptors, book and author. Repository items of the book item
descriptor have an author property whose value is another repository item, an author. Repository items
of the author item descriptor have a book property whose value is another repository item, a book.
85
7 - SQL Repository Item Properties
You can also specify that a repository item property is a collection of repository items. In the previous
example, an author may have written more than one book. Instead of the book property in the preceding
example, this next example uses a books_written property whose value is a Set of book repository
items. The <property> tag for the books_written property uses the following attributes:
Note that in this example, the book table is defined twice in the XML, first in the book item descriptor and
then in the author item descriptor. The second time, this table is a multi-table where each author item
may have more than one row with the id column. In a multi-table, all of the attributes that we define are
multi-valued types. To define Array, List and Map types, you also must specify a multi-column-name
attribute on the table tag. This specifies which column is to be used as the sorting value to determine the
order of the List and the key for the Map.
Now the properties author and books_written are actually real beans (in this case RepositoryItems)
instead of just simple Java primitives. In the author item descriptor, the books_written property is a
Set of RepositoryItems that correspond to books. The other types supported are List, Map, and
Array.
86
7 - SQL Repository Item Properties
This setting makes it easier for you to keep the book repository items synchronized with the author
repository items that refer to them. See the Cascading Data Relationships section. You can then add a
book item to a set of items in the books_written property like this:
Repository gsa = ...;
RepositoryItem newBook = getRepository().createItem("book");
Set books_written = (Set) author.getPropertyValue("books_written");
books_written.add(newBook);
If the books_written property does not have cascade="update", you will have to add the item using
the addItem() method (thus inserting the row in the database) before you add it to the list:
Repository gsa = ...;
RepositoryItem newBook = getRepository().createItem("book");
getProfileRepository().addItem(newBook);
Set books_written = (Set) author.getPropertyValue("books_written");
books_written.add(newBook);
Remember that in each of these cases, it would be most efficient to ensure that all of these method calls
are performed in a single transaction. See Repositories and Transactions in the SQL Repository Architecture
section.
Querying Subproperties
The SQL repository allows applications to query repository items based on attributes of their attributes,
sometimes referred to as subproperties.
Continuing our book example, subproperty querying means that you can write repository queries like:
Get me all the books that were written by authors living in NY
87
7 - SQL Repository Item Properties
Transient Properties
The SQL repository allows you to define properties of a repository item that are transient. Transient
properties are never stored or read from the persistent data store. They are readable and writable, but not
queryable. Transient properties are a convenience that give applications a hook on which to hang their
own custom objects which are not persisted by the repository.
You can specify a transient property by defining a <property> tag that is not associated with any
database table, but which is instead a direct child of an <item-descriptor> tag. For example, in the
following example, the user item descriptor has a transient property that specifies whether the user is
logged in at that time:
You can also define an entire item descriptor to be transient. Such an item descriptor has no <table>
tags and no properties that are direct children of a <table> tag. The properties of transient item
descriptor are queryable by default, unlike a transient property of an item descriptor with other properties
that are persistent properties. In the case of a transient item descriptor, no indexing is used, so queries
against large repositories will be slow. Using transient repositories is sometimes a useful testing tool
during application development.
88
7 - SQL Repository Item Properties
<property name="author">
<attribute name="maxLength" value="30"/>
</property>
</table>
</item-descriptor>
It is also useful to be able to refer to values of Nucleus components as attributes. You can do this with the
bean attribute of the <attribute> tag. For example:
<attribute name="documentRootPath"
bean="/atg/demo/QuincyFunds/repositories/
FeaturesDataStore.relativePathPrefix" />
If you use a relative Nucleus address for the bean attribute, it refers to a component relative to the
Repository component.
You can access property attributes programmatically using the
RepositoryPropertyDescriptor.getValue method. For example:
RepositoryPropertyDescriptor.getValue("maxLength");
Attribute
Description
uiqueryable
uiwritable
If this attribute is set to false, the property cannot be modified or set in the
ACC. Unless the property tag also has the writable=false attribute, the
property can still be modified or set programmatically or through a form.
89
7 - SQL Repository Item Properties
A property value can refer not just to another type of repository item, but also to a repository item in
another repository. When you define a property that refers to an item in a different repository, use the
repository attribute as part of the property tag. For example, if you had a workAddress item type in an
LDAP repository, you might refer to it in a SQL repository like this:
<table name="employees" id-column-names="id">
...
<property name="work_address"
item-type="workAddress"
repository="/atg/userprofiling/LDAPRepository"/>
</table>
The repository attribute can be used with either the item-type or the component-item-type
attribute to indicate that this item is in a different repository, not the current repository. The value of the
linked property in the database is the repository ID of the item in the other repository.
The repository attribute specifies a Nucleus component name relative to the location of the current
repository. This enables you to create a composite repository that combines the item descriptors from
more than one repository. Note, however, that a single repository query cant span repositories with
different data sources.
When you use composite repositories, make sure that your affiliated repositories do not use the same
item descriptor names. In this way, your application can go to the composite repository, get an item
descriptor named products, and not have to know that it is dealing with a separate repository.
If you set the sql-type or sql-types attribute for every non-transient property in your repository
template, then the SQL repository does not attempt to check the JDBC type of your properties.
90
7 - SQL Repository Item Properties
One drawback of this approach is that the sql-type can vary depending on your database software
vendor. If you set the sql-type attribute explicitly, you will need to review and update the value if you
move your repository implementation to a different vendors database software.
The SQL type in this case is the integer type code specified in the class java.sql.Types. For example,
setting the updateSchemaInfoCache to true in the ProfileAdapterRepository component might
generate a file that begins like this:
These files can be generated against one database and copied to another as long as the schemas are
compatible. If you change the schema and this property is enabled, you need to remove the
/data/schemaInfoCache directory so it is regenerated. The /data/schemaInfoCache directory does
get regenerated automatically if you add a property that refers to a new column or a new table. If those
files exist, they are used even if the updateSchemaInfoCache property is set to false. This is so that on
a live server, you can use the schema info cache generated by a different staging server. This will, for
example, help you in the event that one server in your cluster has some problem that prevents it from
generating the schema information.
91
7 - SQL Repository Item Properties
In addition to the standard data types of properties of repository items, described in the Data Type
Correspondences section of SQL Repository Definition Tag Reference, the SQL repository allows you to
add your own types of properties. These new property types will be able to implement both getX and
setX functionality in one of two ways. You can either implement your property from scratch, extending
atg.repository.RepositoryPropertyDescriptor, or you can modify the SQL repositorys default
property implementation by extending atg.adapter.gsa.GSAPropertyDescriptor (which itself
extends RepositoryPropertyDescriptor). If your property is to be stored in the database and needs
to appear inside a <table> tag, it must extend GSAPropertyDescriptor. If your property is transient
and does not correspond to a database table, it should extend RepositoryPropertyDescriptor.
In either case, you can add additional configuration information to the XML file unique to your type. Also,
the get and set methods of your PropertyDescriptor can set and get other property values in this
same item.
indirectly, by specifying a type name of your type and registering it using the method
atg.repository.RepositoryPropertyDescriptor.
registerPropertyDescriptorClass(String typeName,
Class pPropertyDescriptorClass)
Note that if you use the indirect method you must set the userPropertyDescriptors property of the
GSARepository component to include your type (or, in the alternative, call the
registerPropertyDescriptorClass() method at some point before your XML repository definition
file gets loaded).
Most users will find the direct Java class approach simpler. In either case, you use the property-type
attribute in the <property> tag to cause the SQL repository to use a property of your type.
Generally, if you specify a type with the property-type attribute, you do not need to use the data-type
or item-type attribute to specify the type. However, in some cases, you might create user-defined
properties that can represent more than one data type. In such cases, you can use the data-type
attribute to further constrain the property.
92
7 - SQL Repository Item Properties
If this property refers to another items type, this method sets the item descriptor for that type.
RepositoryPropertyDescriptor.setComponentItemDescriptor()
depending on which attributes are set in the <property> tag (data-type, item-type, componentdata-type, or component-item-type). They define the propertys Java class, the components property
class (if the property is a multi-valued property), the RepositoryItemDescriptor for either a scalar or a
multi-valued property which refers to other item types.
If your property type can accept any of these values, you do not need to override these methods. If your
property is constrained in what data types it supports (which is generally the case), you should put error
checking logic into these methods to throw errors if an invalid type is specified. Your property descriptor
should throw the unchecked IllegalArgumentException to provide details about what type is
required. If your property type is very restrictive, you can implement these methods to return the
appropriate values:
RepositoryPropertyDescriptor.getPropertyType()
GSAPropertyDescriptor.getComponentPropertyType()
RepositoryPropertyDescriptor.getPropertyItemDescriptor()
RepositoryPropertyDescriptor.getComponentItemDescriptor()
This will prevent developers of repository definitions from having to set the data-type, componentdata-type, item-type, and component-item-type attributes. You may still want to put error checking
in these methods to signal errors if they do provide invalid values.
When you attempt to get or set one of these property values, your getPropertyValue or
setPropertyValue method is called. The SQL repository provides the RepositoryItemImpl class as an
argument to these methods so that you can call the methods setPropertyValue and
getPropertyValue of the underlying item. If you call these methods with the same name as your
property (this is available in the getName() method on your RepositoryPropertyDescriptor class), it
will result in an infinite loop, so be careful.
The getPropertyValue method receives an extra Object pValue argument. This is set to any value
weve found in the cache for this property name, if any, or null if no value is in the cache. In your
setPropertyValue method, you can call setPropertyValueInCache(this, yourvalue) if youd like
to cache this property value for subsequent method calls.
93
7 - SQL Repository Item Properties
If your property is not set, you may choose to return the value of the getDefault() method on the
RepositoryPropertyDescriptor. This will allow the user to set the default value for this property
using the default attribute in the XML tag. This method calls setDefaultValueString, which converts
the default value based on the class returned by getPropertyType, which then calls setDefaultValue.
You may choose to modify this behavior by overriding these methods though typically this functionality
is sufficient.
Note that user defined properties must be serializable. The getPropertyValue and setPropertyValue
methods do not need to work on an unserialized version, but the getPropertyType,
getComponentType, getPropertyItemDescriptor, and getComponentItemDescriptor methods in
particular do need to work. This is important so that the ATG Control Center can understand the type of
property it is editing.
To make your user-defined property queryable, it should represent a database column. Unless your userdefined property extends GSAPropertyDescriptor, then the property is not queryable and you should
implement the method isQueryable to return false. If you want your user-defined property to be
queryable, make sure it extends GSAPropertyDescriptor. You may also override the methods
isWritable and isReadable to turn off write access or read access to your property respectively. Other
methods such as isHidden, isExpert can also be overridden if you want to set additional Bean
attributes. The method setValue(String pName, Object pValue) is called if any feature descriptor
attributes are supplied with this property.
This method converts from the database version of the property value to the property value as it would
be returned by RepositoryItem.getPropertyValue. For example, for you might convert the ID of an
item into the item itself. You do not need to implement this method if the default behavior of the SQL
repository is what you want.
realToRaw
This does the opposite of rawToReal. It converts from the version of the value given to
setPropertyValue into the value given to the setObject call in JDBC. For example, if you have a
property that specifies a reference to another item, you would convert from the RepositoryItem to its
ID.
For examples of these methods, see the source code for the
atg.adapter.gsa.EnumPropertyDescriptor class at:
<ATG2007.1dir>/DAS/src/Java/atg/adapter/gsa/EnumPropertyDescriptor
createDBPropertyEditor()
This method is used by some reporting UIs or other tools which need to get the database value directly
from SQL, but then want to convert that value to or from a String representation. For example, you might
perform a query against the Profile Repository, and receive the code value for an enumerated property.
You can then use the createDBPropertyEditor to convert the code value to its String representation
94
7 - SQL Repository Item Properties
for display in a UI, for instance. This method is like the method createPropertyEditor, but the property
editor returned from createDBPropertyEditor should operate on the raw value (the value returned
from the JDBC getObject call), not the real value (as returned by the
RepositoryItem.getPropertyValue call).
If pValue = null, that indicates that there was no value in the cache for this
property.
<item-descriptor name="images">
<table name="book" id-column-names="book_id" type="primary">
<property name="title"/>
<property name="author"/>
<property name="lastModifiedTime"/>
<property name="contentFileName" data-type="string"/>
</table>
<property name="contentFile"
property-type="atg.repository.FilePropertyDescriptor">
<attribute name="pathNameProperty" value="contentFileName"/>
</property>
</item-descriptor>
95
7 - SQL Repository Item Properties
If you extend the GSAPropertyDescriptor class, you have two additional methods that you can
override. These methods convert data between the type that is stored in the database and the type that is
stored in the cache. These methods are called only when the data is loaded from or stored to the
database. If a cached value is found, it will be returned without calling these methods. Thus it is slightly
more efficient to do conversion here than in the getPropertyValue or setPropertyValue methods.
//------------------------------------/**
* Translate a raw property value to a real value. The real value is what
* applications use. The raw value is what is stored in the DB.
* @param pRawValue for a property
* @return real value to use in applications for the property
**/
public Object rawToReal(Object pRawValue)
//------------------------------------/**
* Translate a property value to a raw value. The real value is what
* applications use. The raw value is what is stored in the DB.
* @param pRealValue for a property
* @return raw value for storing this property in the DB
**/
public Object realToRaw(Object pRealValue)
The following example is taken from the productCatalog.xml file in ATG Commerce. We define two
user defined property descriptors. The first property descriptor is named data. It returns a java.io.File
object when you call getPropertyValue("data") on one of the media-external items. The path
name for this File object is computed by concatenating the value of the url property of the mediaexternal item with the value of the pathPrefix attribute below, ./docs. Thus the path is of the form:
./docs/<value of the url property>
The second user defined property computes a MIME type from the url property. It returns a MIME type
string like "text/html" from the value of the url property. It uses a MimeTyper component to convert
the suffix to a MIME type.
96
7 - SQL Repository Item Properties
writable="false" queryable="false">
<attribute name="pathNameProperty" value="url"/>
<attribute name="pathPrefix" value="./docs"/>
</property>
<property name="mimeType"
property-type="atg.repository.MimeTyperPropertyDescriptor"
data-type="String" writable="false" queryable="false">
<attribute name="identifier" value="url"/>
</property>
</item-descriptor>
You can register user defined property types in a static registry so that they can be defined with a simple
name, like tag converters. List your user defined properties in the userPropertyDescriptors property
of the GSARepository component.
97
7 - SQL Repository Item Properties
Property Fetching
Normally, when a repository item is loaded from the database, properties in each table are loaded at the
same time. By default, all of the primary table properties of a repository item are loaded when getItem is
called for the first time on the item.
You may need to modify this default property fetching behavior. For some applications, the database
activity required in loading all of the primary table properties may result in performance suffering
unnecessarily. For example, an application may want the SQL repository to load a large GIF property only
if it is specifically asked for. This is referred to as lazy evaluation. On the other end of the spectrum, you
may need to load properties from different tables immediately. For example, an application may want to
always load a users last name whenever a profile is read from the database. For lack of a better term, we
will refer to this as prefetching. Finally, some applications want to group properties so that when one
value is requested, all values in this group are loaded. An example of caching groups would be loading a
zip code and state code whenever a street address is loaded.
You can achieve a finer level of control over property loading using cache groups in your repository
definition. By default, the cache group of a property is the same name as the table that the property is
defined in. You can set the cache group of a property with the group attribute in the propertys definition
in the repository definition file. All properties with the same group attribute are fetched whenever any
member of the group is fetched. Only those properties that are in the same cache group as the repository
ID (or, if there is no ID property, then all the properties in the primary table) are loaded when getItem is
called for the first time on an item. While generally you would define a cache group using the group
attribute in the propertys definition in the repository definition file, you can also define a cache group
using the setGroup method of atg.adapter.gsa.GSAPropertyDescriptor.
For example, an address might be composed of several properties, like this:
<property
<property
<property
<property
name="address1" group="address"/>
name="city" group="address"/>
name="state" group="address"/>
name="zip" group="address"/>
Using the group="address" attribute assures that the whole address is loaded whenever one element of
the address is accessed, even if the properties are stored on different database tables. So, if you call
getPropertyValue for the city property, the address1, state, and zip properties would be loaded as
well.
If you want to assure that only the repository ID is returned and none of the repository items other
properties, you can isolate the repository ID in its own cache group:
98
7 - SQL Repository Item Properties
useSetBinaryStream
useSetUnicodeStream
useSetAsciiStream
useSetObject
99
7 - SQL Repository Item Properties
100
7 - SQL Repository Item Properties
The SQL repository adds a number of features to the basic query architecture of the Repository API
described in the Repository Queries chapter. This chapter discusses the following topics:
Repository Filtering
Overriding RQL-Generated SQL
Parameterized Queries
Named Queries
Text Search Queries
Wildcards in Queries
Not Queries and Null Values
Outer Joins
Table Ownership Issues
Unsupported Queries in the SQL Repository
Repository Filtering
The SQL repository allows you to filter database read operations. For example, you might want a database
lookup always to return only items whose activeFlag property is true. You can filter the repository by
defining a repository query that specifies the filter criteria you want and associating it with the
appropriate item descriptor. The filter is then automatically applied to all of the following operations:
Repository.getItem()
Repository.getItems()
MutableRepository.getItemForUpdate()
MutableRepository.getItemsForUpdate()
RepositoryView.executeQuery()
RepositoryView.executeCountQuery()
Use the <rql-filter> tag in the definition file for an item descriptor.
101
8 - SQL Repository Queries
Set the rqlFilterString property of the item descriptor to an RQL string, which will
be compiled into the Query object that defines the filter.
In most cases, the first method, using the <rql-filter> tag, is easiest and preferable.
<item-descriptor name="article">
<rql-filter>
<rql>name starts with "n"</rql>
</rql-filter>
<table name="article" id-column-names="article_id">
<property name="name"/>
<property name="date"/>
</table>
</item-descriptor>
This setting causes queries and item lookups for this item descriptor to return only items whose name
property starts with n. The SQL repository issues SQL in the form of an extra WHERE clause condition to
implement filtering so any given query or item lookup should be no slower with a reasonable filter tacked
on.
You can also use RQL substitution parameters in your filter query. For example:
<item-descriptor name="article">
<rql-filter>
<rql>name starts with ?0 or availabilityDate < ?1</rql>
<param value="n"></param>
<param bean="/myApp/IssueDate"></param>
</rql-filter>
<table name="article" id-column-names="article_id">
<property name="name"/>
<property name="availabilityDate" data-type="timestamp"/>
</table>
</item-descriptor>
In this second example, the RQL parameters are substituted into the query. The first parameter is a simple
constant value. Typically it is not necessarily to substitute constant values as they can be inlined in the
RQL query string. The second parameter is a Nucleus component. If an object of type
atg.service.util.CurrentDate is used as a parameter (as in this example), the filtering logic will call
getTimeAsTimeStamp() on that object and use that as the value of the parameter. This allows you to
have a CurrentDate service used in a filter. Also note, as in this example, that the less than symbol (<) is
a markup character and must be escaped in your XML file as <. For information about using RQL, see
the Repository Query Language section in the Repository Queries chapter.
102
8 - SQL Repository Queries
potentially costly joins will be required for every item access. This could dramatically degrade repository
performance. It is suggested that the filter not be changed too often. Each query executed by the SQL
repository is AND-ed to the filter query before being executed (or looked up in the cache). If the filter or
the RQL filter parameters are changed too often, the effectiveness of the query cache will be diminished.
Changing the filter once or twice a day should not cause a problem.
GSARepository repo =
(GSARepository)request.resolveName("/examples/TestRepository");
RepositoryView view = repo.getView("canard");
Object params[] = new Object[4];
params[0] = new Integer (25);
params[1] = new Integer (75);
params[2] = "french";
params[3] = "greek";
Builder builder = (Builder)view.getQueryBuilder();
String str = "SELECT * FROM usr_tbl WHERE (age_col > ?0 AND age_col < ?1
AND EXISTS (SELECT * from subjects_tbl where id = usr_tbl.id AND subject
IN (?2, ?3)))";
RepositoryItem[] items =
view.executeQuery (builder.createSqlPassthroughQuery(str, params));
103
8 - SQL Repository Queries
if (items == null)
out.println(" Is null.");
else{
out.println(" Is not null: " + items.length + "<p>");
for (int i = 0; i < items.length; i++){
out.println(items[i].toString() + "<br>");
}
}
Parameterized Queries
A parameterized query is a Repository Query that is incomplete (that is, missing some data) when it is
created, and then is supplied with that data when the query is executed. This is very similar to a
PreparedStatement in JDBC. Parameterized queries are supported only in the SQL repository. You can
substitute a parameter only for constant values, and not column specifications in a Repository Query. The
use of parameters in a Query enables developers to reuse a single instance of that Query over and over
again, supplying different parameter values at execution time. For example, if your goal is to create a
Query like this:
select id from dps_user where first_name = 'keith'
only the value 'keith' can be parameterized; the column name first_name cannot. Sorting and range
information also cannot be parameterized, so only constraints can use parameters. Furthermore,
parameterized queries are used only in queries against the database; you cannot use parameterized
queries against transient properties or in cases where the SQL repository components
useDatabaseQueries property is set to false.
Each element in the pParameterValues array corresponds to a parameter in the given Query.
pParameterValues[0] corresponds to the first parameter in the Query, pParameterValues[1]
corresponds to the second parameter in the Query, and so on. When you create a Query, you need to
remember how many parameters the Query has, and what order they come in, especially when making
compound queries such as AND and OR. You can use the Query.getQueryRepresentation() method
to obtain a string of the query representation including all parameter locations.
104
8 - SQL Repository Queries
Method
Parameter Locations
createComparisonQuery
createPatternMatchQuery
createTextSearchQuery
createIncludesQuery
createIncludesAnyQuery
= 'Jerry'
105
8 - SQL Repository Queries
106
8 - SQL Repository Queries
In the first example, we have to create a new constant QueryExpression if we want to change the name
from Jerry to Phil. This means we must also create a new instance of a comparison Query to use the
new QueryExpression. In the second example, we can increase efficiency by creating just one Query
object, and changing the value of the name we want in an Object array that is passed to the
executeQuery method. This also allows you to cache a Query in your internal application (above the
query cache layer), and pass in varying parameter values at execution time.
Named Queries
Named queries are a useful way to create reusable queries in the Repository Query API, so you dont have
to build a new query object each time. In addition, named queries allow you to specify precisely the SQL
statements to be used by a query, rather than using the generalized SQL generated by the SQL repository.
A named query is a Repository Query or SQL statement that can be defined by the developer and be
invoked by some String name.
The basic Query API requires you to create a Query by calling several methods to create
QueryExpressions, and then combine those QueryExpressions into Query objects. When you use
named queries, once you have defined the query, you need to call only one method to execute the
desired Query. This can reduce object creation and simplify custom code.
Currently, named queries are supported in the SQL repository and the Integration Repository.
Stored Procedures
107
8 - SQL Repository Queries
You can also create a named query using Java code, as described in the next section.
import atg.repository.*;
...
public void sampleQueryTest() {
// Get a hold of your repository somehow
Repository rep = getRepository();
RepositoryItemDescriptor desc = rep.getItemDescriptor("user");
RepositoryView view = desc.getRepositoryView();
QueryBuilder builder = view.getQueryBuilder();
try {
// Create your query
QueryExpression prop = builder.createPropertyQueryExpression("firstName");
QueryExpression constant = builder.createConstantQueryExpression("Adam");
Query query = builder.createComparisonQuery(prop, constant,
QueryBuilder.EQUALS);
//See if we have a NamedQueryView to work with...
if(view instanceof NamedQueryView) {
NamedQueryView nameView = (NamedQueryView)view;
String queryName = "firstNameQuery";
108
8 - SQL Repository Queries
<item-descriptor name=...>
...
<named-query>
<sql-query>
<query-name>myQuery</query-name>
<sql>
select id,first_name,last_name from dps_user WHERE login=?
</sql>
<returns>id,firstName,lastName</returns>
<input-parameter-types>java.lang.String</input-parameter-types>
<dependencies>login</dependencies>
109
8 - SQL Repository Queries
</sql-query>
</named-query>
...
</item-descriptor>
There are several things to note here. First, the SQL string contained in the <sql> element must contain
the actual SQL column and table names, not the property names defined by the repository. Second, the
SQL string must include the ID from the primary table in the item-descriptor. If the ID for an item
descriptor is defined as a composite ID (using two or more columns), then all of the columns that make up
the composite ID must be selected.
The <returns> element is a comma-separated list of Repository property names that are returned by this
query. These are property names so we can know the type of the column when were reading values from
the returned ResultSet.
The <dependencies> element indicates which properties this query depends on. If any of the properties
in the <dependencies> element are changed, then this query should be removed from the query cache.
This is typically, if not always, the properties referenced in the WHERE clause of the SQL statement, if any.
The <input-parameter-types> element is a comma-separated list of class names that any parameters
in the Query must be an instance of. There should be one value for each parameter, even if the classes
used are similar; thus, if there are three String parameters in your Query, the <input-parameter-types>
element must contain "java.lang.String, java.lang.String, java.lang.String". This tag
serves two purposes. It is used primarily for type checking when a Query is executed. It is also used to
indicate how many parameters there are in the Query.
The properties that are used in the <returns> element must be:
single-valued and not multi-valued properties (unless they are composite IDs).
The property columns are returned in the order they appear in the <returns> tag. Since users do not
have to define an explicit RepositoryPropertyDescriptor for the ID property, the ID property can be
omitted from the <returns> element, but it still must exist in the SQL statement as the first column(s)
selected.
The <returns> element is optional. Select statements do not have to return any other properties besides
the ID. Again, the ID property should only appear in the <returns> element if that item descriptor
explicitly defines it with the <property> element. Otherwise, the value in the id-column attribute of the
<item-descriptor> tag is used as the name of the ID column.
Stored Procedures
Stored procedures can also be used as a mechanism to return complex query results. Using a named
query to invoke a stored procedure is very similar to a direct SQL named query, with one minor addition.
The <sql> tag for a stored procedure uses an attribute named stored-procedure. When using a storedprocedure, this attribute must be set to true. Here is an example of how you might define a stored
110
8 - SQL Repository Queries
procedure in a repository definition file. Note that the values used in the <returns> and
<dependencies> tags in this example are dependent upon what the stored procedure returns and
defines constraints for.
<item-descriptor name=...>
...
<named-query>
<sql-query query-name="myQuery">
<sql stored-procedure="true">
{ call myStoredProcedure (?, ?) }
</sql>
<returns>id,firstName,lastName<returns>
<input-parameter-types>java.lang.String,java.lang.String
</input-parameter-types>
<dependencies>login</dependencies>
</sql-query>
</named-query>
</item-descriptor>
The syntax for calling a stored procedure is different from executing a SQL statement. The body of the
<sql> tag for a stored procedure must use the syntax required by java.sql.CallableStatement. There are
two formats typically used in a CallableStatement, one for procedures, which dont have an explicit
return value, and one for functions, which do. Unless you are using Oracle, stored procedures in the SQL
repository must use the procedure form, as in the above example. The enclosing curly braces are
required. The question marks indicate parameters. In stored procedures, parameters can be IN, OUT and
INOUT, indicating that the values go in the database, come out, or both go in and come out. The SQL
repository supports only IN parameters. Any OUT parameters or INOUT parameters will not function
correctly, except in the case of Oracle stored procedures.
Any stored procedure used must return a java.sql.ResultSet. Most JDBC drivers will do so by default, but
there are exceptions. Oracle, for example, requires some special tuning, as described in the next section.
Note the leading ? =, which indicates that a value is returned. You might consider wrapping existing
stored procedures with procedures that can format the results in a manner that the ATG SQL repository
requires.
111
8 - SQL Repository Queries
Item descriptor subtypes inherit the named queries defined for their parent item types. For example,
suppose, as in the Quincy Funds demo, you have defined a user item type with investor and broker
subtypes. If the user item descriptor defines a named query called getUsersByLogin, then the
investor and broker subtypes will also have access to that same named query.
Returns the names of all Named Queries that this RepositoryView knows about, or null if there are none.
public QueryDescriptor getQueryDescriptor(String pQueryName)
Returns a QueryDescriptor object that describes aspects of the requested NamedQuery. If there is no
Returns the name of the given Query, if any exists. Otherwise, null is returned.
QueryDescriptor
The atg.repository.QueryDescriptor interface defines methods for an object that describes a
Repository Query and associates a Repository Query object with a user-defined String name. It defines the
following methods:
public Query getQuery()
public String getQueryName()
RepositoryViewImpl
The atg.repository.RepositoryViewImpl class implements NamedQueryView and supports the use
of named queries.
112
8 - SQL Repository Queries
current directory.
package atg.adapter.gsa.sample;
import atg.repository.*;
public class FullTextQuery
extends Harness
{
//------------------------------------/** Class version string */
public static final String CLASS_VERSION =
"$Id: FullTextQuery.java,v 1.1 2000/03/20 19:39:31 mstewart Exp $";
//------------------------------------/**
* Run our sample.
* @param pRepository repository to use
* @exception RepositoryException if there is repository trouble
**/
public void go(Repository pRepository)
throws RepositoryException
{
// print header
pln("### Running Sample Full-Text Query ###");
pln(CLASS_VERSION);
pln("");
/*
**
**
**
**
**
**
**
**
**
*/
113
8 - SQL Repository Queries
// now we build our query: comment contains 'dog' within 10 words of 'cat'
Query dogTenWordsNearCat =
userBuilder.createTextSearchQuery(comment, dogNearCat, format, minScore);
// finally, execute the query and get the results
RepositoryItem[] answer = userView.executeQuery(dogTenWordsNearCat);
pln("running query: story contains 'dog' within 10 words of 'cat' ");
if (answer == null)
{
pln("no items were found");
}
else
{
for (int i=0; i<answer.length; i++)
pln("id: " + answer[i].getRepositoryId());
}
114
8 - SQL Repository Queries
}
//------------------------------------/**
* Main routine. This example uses no command line arguments
**/
public static void main(String[] pArgs)
throws Exception
{
runParser(FullTextQuery.class.getName(), pArgs);
}
} // end of class FullTextQuery
You can specify what properties a text search query should search, using the text-search-properties
attribute in the <item-descriptor> tag that defines an item type. For example, the following value
indicates that a text search should examine the keywords and content properties for matches:
<item-descriptor name="newsItems
text-search-properties="keywords,content">
...
The SQL repository will then convert text search queries into CONTAINS pattern match queries, which are
implemented using the SQL LIKE operator.
Simulated text search queries are useful for demos and standalone development when one wants to put
in place the createTextSearchQuery() API calls without having to set up a text search engine.
However, simulated text queries are extremely inefficient and are not supported for production systems.
A simulated text search query using LIKE will typically cause a table scan, so you wouldnt want to use
simulated queries in production.
Wildcards in Queries
Databases often treat % and _ as wildcard characters. In pattern-match queries in the SQL repository (such
as CONTAINS, STARTS WITH, or ENDS WITH), we assume that a query that includes % or _ was intended as
a literal search including those characters, and was not intended to include wildcard characters. The query
generated therefore uses an escape character in front of the characters % and _ in pattern-match queries.
115
8 - SQL Repository Queries
An exception is where we use a pattern-match query to simulate a text search query, since in that case we
want to allow wildcards to be passed through.
You can disable this behavior by setting the escapeWildcards property of the SQL repository
component to false.
The escape character is \ (backslash) by default. You can set a different escape character using the
wildcardEscapeCharacter property of the SQL repository component.
For example, if your query is balance = 101 or balance < 101, the query will (unsurprisingly) not
return an item whose balance property is null. However, if your query is balance != 101, the query will
still not return an item whose balance property is null.
If you wish your query to return items whose queried property is null, you may use an IS NULL query, or an
IS NULL clause as part of an OR query, for example:
(balance != 101) OR (balance IS NULL)
Outer Joins
By default, the SQL repository uses outer joins in queries that involve auxiliary tables. Different database
vendors use different syntax to create outer joins. The ATG platform automatically sets the
outerJoinSupport property of the GSARepository component to specify the appropriate type of outer
join to be used by the SQL repository. You can also configure this property manually, using the following
values:
Value
Database
Vendor
Description
sql92
jdbc
Similar to sql92 but uses JDBC escape syntax {oj ... } to tell the
JDBC driver to convert to native join syntax.
plus-equals
Oracle
x += y
116
8 - SQL Repository Queries
star-equals
Microsoft
Sybase
x =* y
informix
Informix
none
The metaDataSchemaPattern attribute should be set to the name of the database account that was
used to create the tables that underlie the repository.
The metaDataCatalogName attribute represents a catalog name. If the user does not own the table to be
used by the repository, this attribute can be used once during initialization of the repository in a call to
determine the column types.
If the user does not own the table used by the repository, you can use the tablePrefix attribute to
construct a qualified table name. This attribute is not used during the initial metadata query, but if
present will be prepended to the table name when inserts or updates are made. For example:
<attribute name="tablePrefix" value="myPrefix."/>
For instance, the following snippet sets dps_user to use testing2 as the schema name for the metadata
call. The string testing2. will be prepended to the table name for all other queries.
<gsa-template>
...
<table name="dps_user" type="primary" id-column-name="id">
<attribute name="tablePrefix" value="testing2."/>
<attribute name="metaDataSchemaPattern" value="testing2"/>
...
117
8 - SQL Repository Queries
In using these attributes, make sure you use the exact case (upper, lower, mixed) that your database uses
to store object identifiers. For example, Oracle stores its identifiers in uppercase. So, you would set
metaDataSchemaPattern to DYNAMO instead of Dynamo. See the Javadoc for
java.sql.DatabaseMetaData.getColumns() for more information. See also Database Loggers in the
Logging and Data Collection chapter of the ATG Programming Guide for more information about these
attributes.
In addition, you can set the metaDataSchemaPattern and metaDataCatalogName as properties in the
SQL repository component. If they are set as a property, then that setting will be used as a default for
every table in the repository that does not have the corresponding attribute explicitly set in the repository
definition file. The attribute in the table tag takes precedence, in other words.
118
8 - SQL Repository Queries
You can use Java resource bundles to make it easier to localize a SQL repository definition. By using
resources for repository item properties used in the ATG Control Centers Repository Editors, you can
display labels and values suitable for the Repository Editor users locale.
The SQL repository localization feature enables you to use resource files to localize:
the values of the display-name and description of both item descriptors and
properties
You can use the <attribute> tag to set the resource bundle at either the property, table, or item
descriptor level. A property will use its own resourceBundle attribute if it is set. If not, it looks for a
resourceBundle attribute set in its <table> tag, and then for a resourceBundle attribute set in its
<item-descriptor> tag. For example:
<item-descriptor name="user" ....>
<attribute name="resourceBundle"
value="atg.userprofiling.ProfileResources"/>
...
If you are using xml-combine="append" to add properties to an item descriptor defined in a different
configuration layer, do not set the resourceBundle attribute in the item descriptor , as it will overwrite
the setting of resourceBundle in the other configuration level. Set the resourceBundle at either the
table or property level instead.
119
9 - Localizing SQL Repository Definitions
Localizing Properties
To localize labels used in a Repository Editor, use the localizable attributes, as follows:
display-name
display-name-resource
description
description-resource
category
category-resource
For example, to localize the display-name, use the display-name-resource attribute instead of the
display-name attribute:
<item-descriptor name="user" ...
display-name-resource="itemDescriptorUser">
<attribute name="resourceBundle"
value="atg.userprofiling.UserProfileTemplateResources"/>
Then, for each locale you want to support, create resource bundle properties files for each repository
definition. Each resource bundle consists of a list of keys defined in the resource label attributes, with the
localized value.
The UserProfileTemplateResources.properties resource bundle referred to in the preceding
example would contain this entry:
itemDescriptorUser=User
When you specify a default, use the resource name as the value, such as:
120
9 - Localizing SQL Repository Definitions
Use caution when localizing the strings used for enumerated types. Remember that if you have
useCodeForValue set to false, calling getPropertyValue will not return the localized property value.
If you want to display the localized value on a page, you need to include the localized string in your page,
using a Switch servlet bean to choose the proper value.
For more information about resource bundles and localization, see the Internationalizing a Dynamo Web
Site chapter in the ATG Programming Guide.
121
9 - Localizing SQL Repository Definitions
122
9 - Localizing SQL Repository Definitions
Thoughtful design of database access is a key to achieving acceptable performance in many ATG
applications. You want to minimize how often your application needs to access the database, while still
maintaining data integrity. An intelligent caching strategy is central to achieving these goals. This chapter
includes the following topics that describe how to use caches with a SQL repository:
Item Caches and Query Caches
Cache Modes
No Caching
Locked Caching
Distributed Caching via TCP
Distributed Caching via JMS
Cache Configuration
Cache Statistics
Cache Loading
Cache Flushing
Cache Invalidation Service
Caching and Item Descriptor Inheritance
If query caching is enabled, the first time this query is issued, the result set is retrieved from the database
and stored in the query cache. Then, the next time this same query is issued, the SQL repository can
retrieve the result set from the cache, rather than needing to access the database.
123
10 - SQL Repository Caching
The item caches hold the values of repository items. Repository queries are performed in two passes,
using two separate SELECT statements. The first statement gathers the IDs of the repository items that
match that query. The SQL repository then examines the result set from the first SELECT statement and
finds any items that already exist in the item cache. A second SELECT statement retrieves from the
database any items that are not in the item cache.
Query caching is turned off by default. If items in your repository are updated frequently, or if repeated
queries are not common, you may not receive enough benefit from a query cache to justify the minor
amount of overhead in maintaining the cache. See Cache Configuration for more information about
enabling the query cache and setting the sizes of the item and query caches.
Cache Invalidation
The SQL repository computes which properties are used in each query. When any of these properties are
modified, the affected queries are invalidated. In addition, whenever an item of a given item type is added
to or removed from the repository, all queries are invalidated in the query cache for that item descriptor.
Item cache entries are invalidated when that item is updated.
When a set of properties of a repository item belonging to an item descriptor have been modified, the
item descriptor fires a PropertiesChangedEvent (atg.repository.PropertiesChangedEvent). If
you need specific notification when an items cache entry is invalidated, you can listen for these events.
The PropertyChangedEvent class has a type property that indicates the cause of the event. The value of
type can be one of the following:
int Code
Value
Description
UPDATE
DELETE
INSERT
CACHE_INVALIDATE
Property Name
Property Type
Description
item
RepositoryItem
124
10 - SQL Repository Caching
repositoryId
String
itemDescriptorName
String
properties
Map
How and when cache invalidation is communicated across multiple ATG servers depends on the cache
mode you select, as described in the Cache Modes section. You can also flush caches yourself, as
described in the Cache Flushing section.
Cache Expiration
In addition to invalidating cache entries actively when repository items change, you can assure data
integrity by configuring your caches to expire after the lapse of a specified time. See the Cache
Configuration: Cache Timeout section.
Cache Modes
The SQL repository implements several different modes of caching. Which mode you choose depends on
the nature of your application. You can set the default caching mode on each item descriptor in a
repository. Each propertys definition can override the cache mode of its item-descriptor. The caching
modes implemented in the SQL repository are:
125
10 - SQL Repository Caching
No Caching (disabled)
Simple Caching (caching is handled separately in each VM, with no invalidation events
to synchronize updates from different server instances)
Locked Caching (read and write locks are used to synchronize the caches)
Distributed Caching via TCP (caching with cache invalidation events; events are
communicated using TCP)
Distributed Caching via JMS (caching with cache invalidation events; events are
communicated using JMS)
By default, the SQL repository uses simple caching. To enable a different cache mode for an item
descriptor, set the cache-mode attribute in the <item-descriptor> tag of the repository definition file:
<item-descriptor name="role" cache-mode="distributed">
You can also set the property cache mode to inherit. This specifies that the default cache mode should
be used. This value might be useful if a propertys cache mode has been set to disabled at one point in the
CONFIGPATH and you want to enable the default cache mode for the property at a later point in the
CONFIGPATH.
126
10 - SQL Repository Caching
The SQL Repository then will not store the repository items referred to, but will instead store only the
repository IDs of the items referred to in the item cache.
No Caching
Disabled mode disables caching of items across transactions. Values for this item will be cached within
the current transaction so that you see a consistent view of the item in any given transaction. When
properties are not cached, they are loaded directly into a store that is visible for each transaction. If you
call getPropertyValue() on the same property twice in the same transaction, the property value will
not be re-queried.
You can also disable caching for an individual property within an item-descriptor by setting the cachemode="disabled" attribute on the property itself. All other cache modes must be set at the item
descriptor level, however, and cannot be set on a per-property basis.
Disabled caching should be used with great caution, because it will result in database access for every
page that accesses an item of this type. This potentially has a severe impact on performance.
Caching should generally be disabled when there is a possibility that the underlying data will be changed
by a non-ATG repository application. For instance, if you have an on-line banking application, and the
same data is accessed by other applications in addition to ATG, you may want to turn off caching for
displaying the users account balances. You could, in some circumstances, configure your integration so
that when the data is modified by an external application, the repository cache is invalidated.
Simple Caching
In the simple caching mode, each VM maintains its own cache of the information. Any changes made by
other servers will not be recognized in this server until the cache entry is manually or automatically
invalidated. This mode is suitable for read-only repositories (which includes repositories like product
catalogs where all changes are made on a staging server and not a live server) or architectures where you
can guarantee that only one server will handle a particular repository item.
127
10 - SQL Repository Caching
You can use the item-cache-timeout attribute together with simple cache mode for caching user data
associated with a particular session. If only one user at a time is accessing an item type, this approach can
generally prevent access to stale data from a cache without the overhead imposed by locked mode
caching. See Cache Timeout in this section for more information.
You can also use simple cache mode in cases where it is practical to invalidate caches manually by calling
these methods:
Class
Method
atg.repository.RepositoryImpl
invalidateCaches
atg.repository.ItemDescriptorImpl
removeItemFromCache
invalidateItemCache
invalidateCaches
atg.repository.RepositoryView
invalidateQueryCache
See the Cache Flushing section of this chapter, which also describes the Cache Invalidator service, which
can be used to control cache invalidation manually.
Locked Caching
For some item descriptors, it may be most efficient to use read and write locks to manage access to your
data. At any given time, you can have one server that is caching or modifying an item (or no servers
modifying the item and zero or more servers caching the item, for read-only access). This avoids the need
for you to broadcast cache invalidation events to all servers each time you make a change to the item.
Locked mode may be the best choice for item types that are modified frequently during runtime. It also
does the best job of guaranteeing that only up-to-date data is seen by your application.
If an item uses locked mode caching, then the repository consults a central lock server to obtain read or
write access for the item. The lock server allows multiple simultaneous readers, but only a single writer.
Lock ownership (either read or write) is cached on the server that last requested that type of access to that
item. As soon as another server requests a write lock for that item, all servers which currently have
ownership are asked to give up lock ownership as soon as all transactions which own that lock release it.
When you set the cache-mode attribute to locked in a repository, ATG uses read and write locks to
control which threads can access an item. (Read locks are also called shared locks and write locks are
called exclusive locks.) The exact behavior depends on how you set the isolation level for the item
descriptor.
Isolation Levels
To minimize deadlocks when you use locked cache mode, you can configure the isolation level of an item
descriptor. You can use one of three isolation levels:
128
10 - SQL Repository Caching
Isolation Level
Description
readCommitted
This is the default value. ATG gets a read lock on the item when you first call
getItem or getPropertyValue for an item in a transaction. If you then try to
change the item, you release the read lock and try to acquire a write lock. New
property values are visible only once they have been committed. You are not
guaranteed that subsequent getPropertyValue calls made in the same
transaction return the same value.
repeatableRead
ATG acquires a read lock the first time you call getItem or
getPropertyValue for an item in a transaction. When you call
getItemForUpdate or setPropertyValue in a transaction, ATG attempts to
convert the read lock into a write lock.
serializable
Prevents different transactions from reading an item at the same time either
from the same server, or from different servers.
Set the isolationLevel attribute in an <attribute> tag that is a sub-tag of the <item-descriptor>
tag. For example:
<item-descriptor name="myItem" cache-mode="locked">
<attribute name="isolationLevel" value="readCommitted"/>
...
</item-descriptor>
Locking Exceptions
When you attempt to get a write lock on an item, the following exceptions may arise:
If you are calling a method, such as setPropertyValue, which does not throw a
RepositoryException, you get an IllegalArgumentException instead.
129
10 - SQL Repository Caching
time expiration setting (see the Cache Configuration: Cache Timeout section) or by sending cache
invalidation events programmatically (see Cache Flushing).
ClientLockManager
For each SQL repository that contains any item descriptors with cache-mode="locked", you must set
the lockManager property of the Repository component to refer to a ClientLockManager. ATG comes
configured with a default client lock manager, which you can use for most purposes:
lockManager=/atg/dynamo/service/ClientLockManager
When you first install the ATG platform, the ClientLockManager component has its useLockServer
property set to false, which disables use of the lock server. In order to use locked mode repository
caching, you must set this property to true. This setting is included in the ATG platform liveconfig
configuration layer, so you can set the useLockServer property by adding the liveconfig
configuration layer to the environment for all your ATG servers. You must also set the lockServerPort
and lockServerAddress properties to match the port and host of your ServerLockManagers
components. For example, suppose you have two ServerLockManagers, one running on host tartini
and port 9010 and the other running on host corelli and port 9010. You would configure the
ClientLockManager like this:
$class=atg.service.lockmanager.ClientLockManager
lockServerAddress=tartini,corelli
lockServerPort=9010,9010
useLockServer=true
ServerLockManager
You should also configure one or more ATG servers to start the
/atg/dynamo/service/ServerLockManager on application startup. To do this, add the
ServerLockManager to the initialServices property of /atg/dynamo/service/Initial in the
server-specific configuration layer for the server in which youve chosen to run a ServerLockManager.
For example, if you wanted to run the ServerLockManager in a ATG server instance named derrida,
you could add this properties file at
<ATG2007.1dir>/home/servers/derrida/localconfig/atg/dynamo/service/Initial.propert
ies:
#server/derrida
#/localconfig/atg/dynamo/service/Initial.properties:
initialServices+=ServerLockManager
130
10 - SQL Repository Caching
ServerLockManager Failover
You can configure more than one ServerLockManager. One ServerLockManager acts as the primary
lock server while the other acts as backup. If the primary ServerLockManager fails, then the backup
ServerLockManager takes over and clients will begin to send lock requests to the backup. If both
ServerLockManagers fail, caching is simply disabled. Under that condition, the site still functions, but
just slower since it must access the database more frequently rather than using the cache. The cache
mode also switches into disabled mode for all transactions that are unable to obtain the lock. Once a
ServerLockManager is restored, caching resumes.
A ServerLockManager component is configured with the following properties:
Property
Description
port
The number of the port this server runs on. This should
match the lockServerPort property of the
ClientLockManager.
otherLockServerAddress
otherLockServerPort
otherServerPollInterval
waitTimeBeforeSwitchingFromBackup
If this server detects that the other lock server has failed, it
waits the time in milliseconds specified by this property
before becoming the primary lock server.
For example, if you have two ServerLockManager components named tartini and corelli, each
running on port 9010, they could be configured like this:
# tartini:9010
$class=atg.service.lockmanager.ServerLockManager
handlerCount=0
port=9010
otherLockServerAddress=corelli
otherLockServerPort=9010
otherServerPollInterval=2000
waitTimeBeforeSwitchingFromBackup=10000
# corelli:9010
$class=atg.service.lockmanager.ServerLockManager
handlerCount=0
131
10 - SQL Repository Caching
port=9010
otherLockServerAddress=tartini
otherLockServerPort=9010
otherServerPollInterval=2000
waitTimeBeforeSwitchingFromBackup=10000
It is best if the primary ServerLockManager runs in a ATG instance that does not also handle user
sessions by running a DrpServer. Not only does this prevent the load on the ServerLockManager from
affecting user sessions, but it also lets you stop and restart the DrpServer without restarting the
ServerLockManager. If you find that there is enough lock contention on your site that the lock server
itself becomes a bottleneck, then you might choose to create separate lock servers for different
repositories to distribute the load. Note that in this situation, the lock server will be unable to detect
deadlocks that span lock servers. In this situation, you will need a separate ClientLockManager instance
in each ATG instance to refer to each ServerLockManager.
Key
Read owned
Does this process have a read lock on that key? Note that multiple
processes can simultaneously have the read lock but when one process has
a read lock, no other process will have the write lock.
Write owned
Read/write requested
Has another process requested this read/write lock? Each process first goes
to the ServerLockManager. The ServerLockManager sends out the
read/write request to each client that owns the lock for that key when
necessary. This is only set when read or write owned are set to true and
there are active lock owners that conflict with the lock requested (i.e. a
write lock is requested when this process has a thread with active read
ownership).
Globally owned
This is true if there is an active read or write lock and if the lock was
acquired via the ServerLockManager. If the ServerLockManager
becomes unavailable, each process will still give out locks but will return
the Globally owned flag as false, indicating that this lock is only good for
this process. The SQL repository uses this flag to disable caching.
Write owner
Set to the thread that owns this lock in this process. There can be only one.
Read owners
Set to the threads that own this lock in this process. There can be more
than one.
132
10 - SQL Repository Caching
Read/write waiters
The lock table for a ServerLockManager is similar but in that case the lock owners are actually the
processes, not the threads within a process.
EventServer
Distributed cache mode uses EventServer components (of class
atg.adapter.gsa.event.GSAEventServer) to communicate distributed cache mode events with each
other. All SQL repository instances in a given JVM use the same EventServer instance. The Nucleus
address of the EventServer component is specified by the eventServer property of the
133
10 - SQL Repository Caching
When an item is modified by one ATG server, a cache invalidation event is sent to all other servers that
might cache that item type. At this time, a persistent connection is created to all other EventServer
components in the cluster as listed in the das_gsa_subscriber table. Events are written into this
connection.
When the EventServer receives a GSAEvent, which contains an item descriptor name, it looks up the
EventListener for that item descriptor name and sends the event to the matching repository.
GSARepository implements the GSAEventListener interface. When a GSARepository component
receives a GSAEvent, it removes the item identified by the GSAEvent from its cache.
When an EventServer attempts to connect to an ATG server, it may not be able to establish a
connection immediately. For example, if the servers JVM is performing garbage collection, the
connection will not be established until garbage collection is completed. Therefore, the EventServer
waits a certain amount of time before timing out the connection. By default, it waits 60 seconds, but you
can specify a different value with the connectTimeout property of the EventServer component. You
specify this property value in milliseconds. For example, to set the timeout to 2 minutes:
connectTimeout=120000
If an ATG server fails to connect to another server, it prints a warning message and removes the
das_gsa_subscriber row for that item type for that server. When the ATG server sends an event, it also
checks to make sure the current EventServer instance is still in the table. If not, it prints a warning and
adds itself back in.
When an ATG application starts up, the EventServer broadcasts a dummy invalidation event. This
dummy event is used to form all of the connections that may be required and to ensure that the
das_gsa_subscriber table is valid and properly updated for all item types. Also note that distributed
cache mode is used by the internally defined item descriptor used to access the das_gsa_subscriber
table. This is necessary so that other servers can be made aware dynamically of changes made to this
table while the server is running.
checkSubscriptions() Method
The GSARepository class has a checkSubscriptions() method that can be called periodically (e.g., by
a scheduled service) to ensure that the repository is still subscribed to events for items with cachemode="distributed". When another server fails to connect to the EventServer, the EventServer
removes the server from the das_gsa_subscriber table. The checkSubscriptions() method issues
one query on this table for each item descriptor in the repository with cache-mode="distributed"; if
the method determines that a server has been removed from the table, it issues a warning and flushes the
caches of the affected item types to ensure that no invalid data is maintained in the caches.
134
10 - SQL Repository Caching
135
10 - SQL Repository Caching
When you use the TCP-based distributed cache mode, it is possible for a server to occasionally miss cache
invalidation events. This can happen if the server temporarily fails to connect to another server. If a server
does fail to connect, there are various safeguards in place so that it receives the event once the
connection is re-established. However, even with these safeguards, there can sometimes be missed
events.
To ensure that invalidation events are not missed, you can instead use cache-mode=distributedJMS.
This mode ensures that servers always receive the cache invalidation events, but it should be used only
for items that are updated infrequently, because performance is considerably slower than for
distributed mode.
When cache-mode=distributedJMS, communication of invalidation events is done using JMS. Each
Repository component refers to a GSAInvalidatorService component that is configured as a message
source in Patch Bay. When a repository item is updated, the GSAInvalidatorService component sends
a JMS message for this event. A GSAInvalidationReceiver component is configured as a message sink
in Patch Bay, and is a durable subscriber to the topic that receives these messages.
To use cache-mode=distributedJMS, you must enable the GSA Cache Invalidator on all servers, and
make sure that this service is configured properly. See Cache Invalidation Service for more information.
Cache Configuration
The SQL repository maintains separate item caches and query caches for each item descriptor. This lets
you size and flush caches separately for separate item types. You can configure these caches using the
following attributes in an <item-descriptor> tag:
item-cache-size
query-cache-size
item-expire-timeout
136
10 - SQL Repository Caching
query-expire-timeout
item-cache-timeout
Note that the query cache and item cache sizes are set by default is 0, effectively disabling the caches. This
is an appropriate setting for evaluation and development; for testing and deployment, you need to set
these caches to more appropriate values. Assuming you have realistic testing scripts, you may be able to
estimate reasonable cache sizes during testing, using the feature described in the Cache Statistics section
of this chapter.
137
10 - SQL Repository Caching
Cache Timeout
The SQL repository offers two different approaches to setting a maximum lifetime for items in the cache.
Each of these approaches uses a timeout attribute in the item descriptor definition. The first approach
uses the item-expire-timeout and query-expire-timeout attributes. Each of these attributes
specifies the number of milliseconds that an entry can stay in the item cache or query cache before it
needs to be refreshed. For example, if you set item-expire-timeout to 1000 milliseconds, the SQL
repository will reload those properties after they have been in the cache for 1 second. If you set queryexpire-timeout to 1000 milliseconds, the SQL repository re-executes the query after that entry has
been in the query cache for 1 second.
To use the item-expire-timeout and query-expire-timeout attributes, set the value (in
milliseconds) in the item-descriptor tag. For example:
<item-descriptor name="order" cache-mode="simple"
item-expire-timeout="180000"
query-expire-timeout="180000">
...
</item-descriptor>
Under the second approach, the SQL repository expires entries in the item cache when they have been
unused for the specified amount of time. In this case, an item remains in the item cache as long as it is
being accessed in this particular ATG instance before the time specified by the item-cache-timeout
attribute for that item descriptor has elapsed. If an item has not been accessed for the duration of the
item-cache-timeout value, then its entry in the item cache is invalidated. The default, 0, lets the item
remain in the cache indefinitely until otherwise invalidated.
To use the item-cache-timeout attribute, set its value (in milliseconds) in the item-descriptor tag
for item types that are associated with a particular user session. For example:
<item-descriptor name="order" cache-mode="simple"
item-cache-timeout="180000">
...
</item-descriptor>
In most cases, using the item-expire-timeout attribute will be preferable to using the item-cachetimeout attribute, since the item-cache-timeout approach is not suitable for certain types of data
access.
You can use either of these cache timeout approaches as an alternative to using the locked cache mode.
You can use cache timeout attributes together with simple cache mode to control the behavior of the
repository caches. The cache timeout attributes are useful for caching user data associated with a
particular session. A users session is typically handled by a single ATG server for as long as the session
lasts. If a users session expires, and then the user moves to another ATG server, the cached data will
expire before the user has a chance to log back in to a server that might have previously cached stale data
for that user. This attribute is also a means to reduce a systems memory requirements by removing
cached items that are no longer being accessed.
138
10 - SQL Repository Caching
Cache Statistics
You can view statistics on caches in a repositorys page in the Administrative Interface Component
Browser, under the heading Cache usage statistics. For example, the Profile Repositorys page in the
Component Browser can be found at:
http://hostname:port/nucleus/atg/userprofiling/ProfileAdapterRepository/
In this URL, hostname is the name of the machine running the application server and port is the port that
your application server uses to listen for HTTP requests. To find the default port, see the ATG Installation
and Configuration Guide.
Monitoring cache statistics during testing and after deployment can help you improve performance by
setting cache sizes to appropriate levels. If you have a high quantity of misses and no hits, you are gaining
no benefit from caching, and you can probably just turn it off, by setting the cache size to 0. If you have a
mix of hits and misses, you might want to increase the cache size. If you have all hits and no misses, your
cache size is big enough and perhaps too big. There is no harm in setting a cache to be too big unless it
will fill up eventually and consume more memory than is necessary.
The cache usage statistics table presents the following statistics:
Property
Description
entryCount
cacheSize
usedRatio
The percent of the maximum cache size now in use. If this ratio is
close to 100, you probably want to increase the cacheSize.
accessCount
hitCount
missCount
The total number of times an access attempt did not find an item
or query result from the cache.
hitRatio
cacheInvalidations
entryInvalidations
You can also examine the contents of a cache using the <dump-caches> tag. See dump-caches Tag in the
SQL Repository Definition Tag Reference section of this chapter.
139
10 - SQL Repository Caching
Cache Loading
You can achieve better performance in a SQL repository-based application by pre-loading the caches.
Note that the improvement in performance from pre-loading caches comes at the expense of slower
application startups. You can configure the SQL repository to automatically save and restore its caches, or
you can load the caches manually by using the startSQLRepository script with an appropriate XML file.
It is probably a good practice to put cache-loading tags in a separate XML file with the same name as your
repository definition file and rely on XML file combination to invoke the queries.
Automatic Reloading
You can configure a GSARepository component to automatically save the state of its item caches when
it is stopped and reload the same items into its caches when it is restarted. Set the following property:
restoreCacheOnRestart=true
With this setting, the repository saves an XML file at the location specified by the repositorys
cacheRestoreFile property. This XML file contains a set of tags that reload each of the items that
existed in the cache when the repository component was stopped.
The PCDATA for this element is a comma-delimited list of repository IDs that should be pre-cache during
start-up of the repository. The properties attribute, which is optional, specifies a list of property names
to cache. For example, the following caches the displayName and description properties of four
product items:
<load-items item-descriptor="product">
properties="displayName,description">
prod10001,prod100001,prod100002,prod100003
</load-items>
The <query-items> tag produces a large number of log messages when you use it to load many items.
You can use the quiet="true" attribute to eliminate these messages:
140
10 - SQL Repository Caching
You can use the id-only="true" attribute to output just the repository IDs of the items returned by the
query:
<query-items item-descriptor="product" id-only="true">ALL</query-items>
For more information, see the Querying Items topic in the Development, Testing and Debugging with the
SQL Repository section of this chapter.
might cause the following to be logged, if there were four product items in the cache:
============== START BUFFER PRECACHE ==============
<load-items item-descriptor="product">
prod100003,prod100002,prod100001,prod10001
</load-items>
=============== END BUFFER PRECACHE===============
Cache Flushing
You can flush (invalidate) the caches for an item descriptor or an entire SQL repository, using the
following methods. Note that first, you must cast your atg.repository.RepositoryItemDescriptor
to an atg.repository.ItemDescriptorImpl. If you are using distributed cache mode, use the Cache
Invalidator, as described in the Cache Invalidation Service section below.
The methods in atg.repository.ItemDescriptorImpl are:
invalidateItemCache()
invalidateCaches()
removeItemFromCache(String pId)
141
10 - SQL Repository Caching
These methods also have versions that accept a boolean parameter that indicates whether the cache
should be changed globally, or just for the local cache. These methods are:
removeItemFromCache(id, boolean pGlobal)
invalidateCaches(boolean pGlobal)
invalidateItemCache(boolean pGlobal)
If this global parameter is true, the invalidation occurs across the cluster. Otherwise, the invalidation
occurs only in the local ATG instance.
The removeItemFromCache method, when given a true value, will use one of two mechanisms to
distribute the invalidation event:
If the item descriptor uses distributed cache mode, it uses the event server to send the
invalidation event.
The invalidateCaches and invalidateItemCache methods, when given true for the global
parameter, will always use the GSAInvalidatorService. If this service is not enabled, a warning is
logged and the cache is only invalidated locally.
This method in atg.repository.RepositoryImpl affects all caches in the repository:
invalidateCaches()
You can cast your repository to these classes and call these methods from there. You can both flush items
of a specific kind, items and queries of a specific kind or a specific item with these methods.
For example, here is how you might use the invalidateItemCache() method to invalidate the item
caches for every item descriptor in a repository:
142
10 - SQL Repository Caching
You can invoke it from the command line for repository items that you specify. See
Invoking the Cache Invalidator Manually.
If your installation includes the DPS module, the SQL-JMS system is preconfigured to work with the Cache
Invalidator, so you only need to set the gsaInvalidatorEnabled property to true.
Note that you must make sure that all ATG instances participating in the Cache Invalidator scheme are
configured to access the same SQL-JMS database with the appropriate JDBC Datasource configurations.
The SQL repository or repositories that will be invalidated may or may not be part of the same datasource.
The {repositoryPath} required parameter is the Nucleus address of the GSARepository component.
You can optionally specify the [itemDescriptorName] parameter to invalidate only cached items of that
143
10 - SQL Repository Caching
item type. You can optionally specify the [repositoryItemId] parameter to invalidate only the specified
repository item.
The Cache Invalidator action is initiated by performing a RMI call to the
GSAInvalidatorService.invalidate() method. The RMI call is made by executing the GSA
repositoryPath
repositoryName
repositoryId
item ID (optional)
You can enable additional debugging messages by setting the property loggingDebug=true in the
following components:
debugging)
debugging)
144
10 - SQL Repository Caching
145
10 - SQL Repository Caching
146
10 - SQL Repository Caching
The XML document type definition for the SQL repository includes operation tags whose primary purpose
is to help you develop, test, and debug your SQL repository template. You can use these tags to modify
your repositorys database by:
Adding Items
Updating Items
Removing Items
Querying Items
These tags are used by a script named startSQLRepository. The use of this script is described in the
startSQLRepository Script and the Template Parser section in this chapter.
To use these developmental tags:
1.
Go to the repositorys page in the Administration Interface. For example, for the SQL
Profile Repository, go to:
hostname:8830/nucleus/atg/userprofiling/ProfileAdapterRepository
2.
In the Run XML Operation Tags on the Repository text area, enter the
developmental tags and click Enter.
You can instead use the startSQLRepository script from a command line. Create an XML repository
definition file and pass it to the startSQLRepository script with appropriate arguments. See the
startSQLRepository Script and the Template Parser section in this chapter for more information.
Note that if you add or remove an item descriptor in your repository definition file, you must close and
reassemble, redeploy, and restart your application, which restarts the ACC and DAF. Otherwise, errors may
result. For example, if you remove an item descriptor, the item descriptor will still appear as an option in
the ACC query editor (List items of type... ) and may cause errors if selected. For instructions on
assembling applications, see the ATG Programming Guide.
147
11 - Development, Testing and Debugging with the SQL Repository
Adding Items
You can use an XML template to add repository items. Use an <add-item> tag for each repository item
you want to add. Each <add-item> tag must include an item-descriptor attribute to specify the name
of the item descriptor to which this repository item should belong. You can nest <set-property> tags
within the <add-item> tag to set property values of the new repository item. Any properties you do not
set will have the default property value for that item descriptor.
For example, the following tags add to the database an instance of users with id = 1. It sets the
username property to Marty.
<add-item item-descriptor="users" id="1">
<set-property name="username" value="Marty"/>
</add-item>
Note that <add-item> tags are processed one at a time. They cannot make forward references to other
items and no attempt is made to satisfy database integrity constraints (beyond that automatically done
with the cascade operator). Use the <import-items> tag if you want to load in items with forward
references.
Note also that if you specify the ID of an existing repository item, you will update that item, overwriting
the values of the existing item with the values you specify in the <add-item> tag. Any add or remove
attributes in a <set-property> tag within an <add-item> tag will be ignored.
148
11 - Development, Testing and Debugging with the SQL Repository
The template parser substitutes the ID of the item you created using that tag.
For example, suppose you want to add an item, one of whose properties is another repository item. Lets
say we are adding a book item, and the book item has a author property, which is itself a repository item.
If you dont want to supply the repository ID for the author repository item, you can use a tag attribute
placeholder like this:
<add-item item-descriptor="author" tag="AUTHORID_TAG">
<set-property name="authorName" value="Arthur Ransome"/>
...
</add-item>
<add-item item-descriptor="book">
<set-property name="title" value="Swallows & Amazons"/>
<set-property name="author" value="$tag:AUTHORID_TAG$"/>
</add-item>
149
11 - Development, Testing and Debugging with the SQL Repository
Updating Items
You can update repository items using the <update-item> tag. The <update-item> tag encloses one or
more <set-property> tags that specify the properties and values being set. Each <update-item> tag
must include an item-descriptor attribute to specify the name of the item descriptor of the repository
item being removed. You can also use the skip-update attribute to set properties in the item, but avoid
the update item call until the transaction is committed.
For example, the following element changes the value of the dependents property of the user with id of
1:
<update-item item-descriptor="user" id="1" skip-update="true">
<set-property name="dependents" value="1414,1732"/>
</update-item>
You can use the add or remove attributes to add or remove values from multi-item properties without
overwriting the whole property value. For example, to add another value to the dependents property:
<update-item item-descriptor="user" id="1" skip-update="true">
<set-property name="dependents" value="1799" add="true"/>
</update-item>
Removing Items
You can remove items from the repository using the <remove-item> tag. Each <remove-item> tag must
include an item-descriptor attribute to specify the name of the item descriptor of the repository item
being removed.
For example, the following tag removes any repository item whose item descriptor is named users and
whose repository ID is 1.
<remove-item item-descriptor="users" id="1"/>
150
11 - Development, Testing and Debugging with the SQL Repository
Data in an auxiliary table will always be deleted by a <remove-item> tag regardless of this attribute
because it is not considered a reference.
The anyReferencesToItem method queries whether any cross-references to a repository item exist
within the repository that contains that item. It uses the same logic as removeReferencesToItem to
determine whether references exist. This method can only detect references through queryable
properties.
Calling these methods generates multiple repository queries per call, one for each property descriptor
that might refer to the item. For example, if the items type is contact-info, one query is performed for
each property descriptor whose type is either contact-info or any multi-valued type that might contain
a list of items of type contact-info. The queries each fetch at most one item from the repository, so the
effect on the repositorys cache should be minimal.
Querying Items
You can perform queries against the repository using the <query-items> tag. The query itself may be
specified as a query attribute of the <query-items> tag or as PCDATA between the opening and closing
tags. The query uses the Repository Query Language (RQL) described in the Repository Query Language
section of the Repository Queries chapter.
For example, the following tag queries the database for any repository items whose item descriptor is
named users and whose username property is Marty:
<query-items item-descriptor="users">username="Marty"</query-items>
Queries can be used in this way to pre-load repository caches. See Cache Loading in the SQL Repository
Caching chapter.
151
11 - Development, Testing and Debugging with the SQL Repository
Other operations tags let you import items from another repository or export items. You can also print out
the DDLs used in setting up the tables corresponding to the repository template. See the descriptions of
the following tags in the SQL Repository Definition Tag Reference:
remove-all-items Tag
export-items Tag
import-items Tag
print-ddl Tag
Note that you can also use the startSQLRepository script to export, import, and print repository items,
as described in the next section.
Verifies your XML is correctly formed and compliant with the DTD.
Parses and processes optional operation tags like <add-item>, <remove-item>, and
<query-items>. These tags provide a means for adding, removing, updating items in
your SQL repository.
Returns results of <query-items> and <print-item> requests in the form of <additem> tags. This allows you to easily copy and paste the results into another XML
template, so you can add the items to another repository.
you have appropriate database access authorization to perform any import or create
database operations
152
11 - Development, Testing and Debugging with the SQL Repository
In addition, you need to make sure that repository IDs in the source repository do not collide with
repository IDs in the target repository. Make sure that both the source database and target database
already contain IdSpaces. The IdSpaces in the source database and target database must have the
same names. Furthermore, the name of the IdSpaces used by each item descriptor should be the same
in the source repository for the export and the target repository for the import. If you do this, then the
import operation reserves all the IDs it encounters for repository items it creates in the target database.
See ID Generators in the Core Dynamo Services chapter of the ATG Programming Guide for more
information about IdSpaces.
The startSQLRepository scripts use the following syntax:
startSQLRepository <arguments> [file name.xml]
(Note that in Windows, you must run the script from the <ATG2007.1dir>\home directory.)
For example, to load an XML template whose CONFIGPATH name is /atg/test.xml in a repository with
a Nucleus address of /atg/userprofiling/ProfileAdapterRepository:
startSQLRepository -m DPS
repository /atg/userprofiling/ProfileAdapterRepository /atg/test.xml
Note that the repository template file name you provide is a reference to a CONFIGPATH location, and not
a pathname in your file system. For example, you can put a file in your localconfig directory and refer
to it as /<your-file>. You can also use a file with the same name as your repositorys existing definition
file and then omit the file name argument from the startSQLRepository command. The
startSQLRepository script uses XML file combination to combine all files with the same name into a
single repository definition. See XML File Combination in the Nucleus: Organizing JavaBean Components
chapter of the ATG Programming Guide. For example, if you include the following file at
<ATG2007.1dir>/home/localconfig/atg/userprofiling/userProfile.xml and run the
startSQLRepository script, it will print all profiles in your Profile Repository:
<gsa-template>
<print-item item-descriptor="user"/>
</gsa-template>
You can use the startSQLRepository script together with the test operation tags described earlier in
this chapter to quickly test a query, or add, update, remove, or print an item.
153
11 - Development, Testing and Debugging with the SQL Repository
Argument
Purpose
-m <module name>
-s <server name>
component.
-outputSQL
-outputSQLFile <file>
-database <vendor>
-output <file>
-verboseSQL
154
11 - Development, Testing and Debugging with the SQL Repository
-export <item-type1,item-type2>
<file>
-exportRepositories <path-torepository1,path-to-repository2>
<file>
-skipReferences
-noTransaction
-import <file>
-debug
-encoding <encoding>
If the content you are exporting contains nonASCII characters, use this option to specify an
encoding such as 8859_1 or SJIS in which to save
your content.
<file names>
For information on options for using the startSQLRepository script with ATG Content Administrations
versioned repository, see the ATG Content Administration Programming Guide.
155
11 - Development, Testing and Debugging with the SQL Repository
You can pass more than one XML file to the startSQLRepository script, and the Template Parser
handles them in the order you specify them. This enables you, for example, to pass your full repository
definition file together with a test file that uses the test operation tags to manipulate repository items. See
the following SQL Repository Test Example.
Server Conflicts
Note that if you start the startSQLRepository script on a server that is already running, errors can
result. Use the -s argument to specify a server that is not currently running.
156
11 - Development, Testing and Debugging with the SQL Repository
<query-items item-descriptor="users">
prop1="Marty"
</query-items>
</gsa-template>
2.
3.
Migrate the data from the SOLID database to an XML file. You can choose specific
repositories to migrate, or you can migrate all the data in all the repositories.
Exporting All Repositories
To export the data from all SQL repositories registered in the
/atg/registry/ContentRepositories component, issue the following command
(type the following on one line, with no line breaks):
bin/startSQLRepository -m module -exportRepositories all all.xml
For example, to export the Product Catalog from the Pioneer Cycling store database
into a file called products.xml, issue the following command:
bin/startSQLRepository -m PioneerCycling -export all products.xml
-repository /atg/commerce/catalog/ProductCatalog
Add a JDBC driver for your database and configure the JDBC connection pool. For
more information, see the ATG Installation and Configuration Guide.
5.
157
11 - Development, Testing and Debugging with the SQL Repository
6.
For example, to import the content from the Pioneer Cycling products.xml file, issue
the following command:
bin/startSQLRepository -m PioneerCycling -import products.xml
-repository /atg/commerce/catalog/ProductCatalog
Note that when binary data in repository items exported, it is represented using base64 encoding.
In this URL, hostname represents the name of the machine that runs your application server and port
represents the port number that your application server uses to listen for HTTP requests. To find your
default port, see the ATG Installation and Configuration Guide.
In the text field, you can enter any XML operations tags against the current repository as if they were
commands. Click the Enter button and the page displays the output you would obtain by running the
startSQLRepository script against the repository.
Debug Levels
The SQL Repository component has a debugLevel property you can use to adjust the debug log entries.
This property is an integer from 0 to 15, with 15 resulting in the greatest frequency of debug log entries.
158
11 - Development, Testing and Debugging with the SQL Repository
The default level is 5; this typically is the level desired when contacting ATG support to diagnose
problems. Level 3 is the lowest level at which SQL debugging statements are issued. At level 3 and above
when loggingDebug is true on the SQL Repository component, it also turns on loggingDebug on its
Transaction Manager, since Transaction Manager debugging is often needed in conjunction with SQL
debugging. Even at level 0, some debug statements are issued.
You can also get debugging messages for an individual item descriptor or property. Turn on logging
debug in the Dynamo Administration Interface page for the Repository component.
You can also turn on debug messages by including a loggingDebug attribute tag in the repository
definition for that item descriptor or property. For example:
<item-descriptor name="user" ...>
<attribute name="loggingDebug" value="true" />
...<property ... />
</item-descriptor>
Modify your database, running the appropriate DDLs to make any necessary changes.
2.
3.
159
11 - Development, Testing and Debugging with the SQL Repository
160
11 - Development, Testing and Debugging with the SQL Repository
161
12 - SQL Repository Reference
input-parameter-types Tag
returns Tag
dependencies Tag
Development Operation Tags:
transaction Tag
rollback-transaction Tag
import-items Tag
add-item Tag
update-item Tag
query-items Tag
print-item Tag
set-property Tag
remove-item Tag
remove-all-items Tag
export-items Tag
load-items Tag
dump-caches Tag
print-ddl Tag
DOCTYPE Tag
All XML documents declare their doctype, referencing a document type definition file. The DTD for SQL
repository templates is installed within the <ATG2007.1dir>/DAS/lib/classes.jar archive, but can
be referred to with this URL:
http://www.atg.com/dtds/gsa/gsa_1.0.dtd
You should not need to change the reference to the DTD in the following DOCTYPE tag:
<!DOCTYPE gsa-template
PUBLIC "-//Art Technology Group, Inc.//DTD General SQL Adapter//EN"
"http://www.atg.com/dtds/gsa/gsa_1.0.dtd">
Note that if your SQL repository definition is composed from more than one file, using XML file
combination, only the file that appears first in the CONFIGPATH should include the DOCTYPE declaration.
See XML File Combination in the Nucleus: Organizing JavaBean Components chapter of the ATG
Programming Guide.
162
12 - SQL Repository Reference
gsa-template Tag
The entire template is enclosed in a <gsa-template> tag.
Child Tag
How many?
header
zero or one
item-descriptor
zero or more
add-item
zero or more
dump-caches
zero or more
export-items
zero or more
import-items
zero or more
load-items
zero or more
print-ddl
zero or more
print-item
zero or more
query-items
zero or more
remove-all-items
zero or more
remove-item
zero or more
transaction
zero or more
update-item
zero or more
header Tag
The <header> tag provides information that can help you manage the creation and modification of
repository definition files.
Child Tag
How many?
Description
name
zero or one
163
12 - SQL Repository Reference
author
zero or more
version
zero or one
description
zero or one
For example, the header of your template might look like this:
<header>
<name>Catalog Template</name>
<author>Herman Melville</author>
<author>Emily Dickinson</author>
<version>$Id: catalog.xml,v 1.10 2000/12/24 03:34:26 hm Exp $</version>
<description>Template for the store catalog</description>
</header>
item-descriptor Tag
The SQL repository template contains one <item-descriptor> tag for each set of repository items that
have common attributes.
Attribute
Description
Value
name
Required
display-name
String
display-nameresource
String
164
12 - SQL Repository Reference
description
description-resource
String
default
super-type
sub-type-property
String
sub-type-value
String
String
display-property
String
165
12 - SQL Repository Reference
version-property
String
id-separator
id-space-names
String
hidden
expert
text-searchproperties
A comma-separated list of
property names.
cache-mode
166
12 - SQL Repository Reference
item-cache-size
item-expire-timeout
query-expire-timeout
item-cache-timeout
query-cache-size
Integer. Default is 0,
effectively disabling the query
cache.
167
12 - SQL Repository Reference
content
folder
content-nameproperty
String
content-pathproperty
String
use-id-for-path
folder-id-property
String.
168
12 - SQL Repository Reference
content-property
content-lengthproperty
String
last-modifiedproperty
String
Child Tag
How many?
property
zero or more
table
zero or more
attribute
zero or more
rql-filter
zero or one
named-query
zero or more
169
12 - SQL Repository Reference
If a <property> tag appears as a direct child of an <item-descriptor> tag (rather than a child of a
<table>tag), then the property is a transient property of the repository item and is not stored
persistently in the database. Transient properties are readable and writable, but cannot be made
queryable. See the Transient Properties section of this chapter.
table Tag
The <table> tag defines the SQL database tables that store properties of repository items.
Child Tag
How many?
property
one or more
Attribute
Description
Value
name
Required.
id-column-names
Required.
multi-column-name
String
type
Default is auxiliary.
property Tag
A <property> tag that is a child of a <table> tag defines one of the persistent characteristics of a
repository item. A <property> tag that is a direct child of an <item-descriptor> tag defines a transient
characteristic of a repository item. Since such a transient property is not associated with any database
table, it is not stored when the repository item is updated in the database.
170
12 - SQL Repository Reference
Child Tag
How many?
option
zero or more
attribute
zero or more
derivation
zero or one
Valid Values
Purpose
name
String
Required.
column-names
String
display-name
String
display-name-resource
String
description
String
description-resource
String
171
12 - SQL Repository Reference
category
String
category-resource
String
data-types
string
big string
enumerated
boolean
int
short
long
float
double
byte
binary
date
timestamp
array
set
list
map
item-type
sql-type
property-type
Java type
172
12 - SQL Repository Reference
component-item-type
component-data-type
Data-type name
default
A string
group
A string
Zero or more of
cascade
insert, update or
delete, separated by
commas
repository
cache-mode
disabled
inherit
173
12 - SQL Repository Reference
editor-class
queryable
Boolean
readable
Boolean
Defaults to true
writable
Boolean
Defaults to true
required
Boolean
hidden
Boolean
expert
Boolean
Defaults to false
Note that the values of name attributes are case-insensitive. Therefore, you cannot have, for instance, two
different properties named gender and GENDER.
174
12 - SQL Repository Reference
string
String
VARCHAR
VARCHAR or CLOB (Oracle)
big string
String
date
java.util.Date
timestamp
java.sql.Timestamp
enumerated
String
INTEGER
boolean
Boolean
NUMERIC(1)
int
Integer
INTEGER
byte
Byte
INTEGER
binary
byte[]
short
Short
INTEGER
SMALLINT (DB2)
float
Float
double
Double
long
Long
NUMERIC(19)
BIGINT (DB2)
array
xxx[]
none
set
java.util.Set
none
list
java.util.List
none
map
java.util.Map
none
175
12 - SQL Repository Reference
option Tag
For properties whose data type is enumerated, use <option> tags to indicate the possible values of the
enumerated properties. For example:
<property name="gender" data-type="enumerated">
<option value="male" code="0"/>
<option value="female" code="1"/>
</property>
Attribute
Description
Value
value
Required.
code
resource
Optional.
attribute Tag
A Java Beans PropertyDescriptor can store an arbitrary set of name value pairs called feature
descriptor attributes. You can use the <attribute> tag in the SQL repository as a child of a <property>
or an <item-descriptor> tag to supply parameters that affect the behavior of properties or item types
in your repository definition. A <property> tag or an <item-descriptor> tag can contain zero or more
<attribute> tags. The <attribute> tag is an empty tag that defines the parents feature descriptor
value or values. This tag allows you to associate arbitrary name/string value pairs with any property or
item type. The name/value pairs are added to the property descriptor via the setValue method of
java.beans.FeatureDescriptor, and can later be used by the application. Here is an example:
<property name="employeeNumber" data-type="string">
<attribute name="PCCExpert" value="true" data-type="boolean"/>
</property>
176
12 - SQL Repository Reference
You can refer to values of Nucleus components with the bean attribute of the <attribute> tag. For
example:
<attribute name="documentRootPath"
bean="/atg/demo/QuincyFunds/repositories/FeaturesDataStore.
relativePathPrefix" />
Attribute
Description
Value
Name
Required.
Value
Required.
data-type
int
short
byte
long
float
double
string
date
timestamp
Default is string.
Bean
177
12 - SQL Repository Reference
derivation Tag
The <derivation> tag is used for derived properties. See the Derived Properties section in the SQL
Repository Properties chapter. The <derivation> tag can have one or more <expression> tags as
children.
Attribute
Description
Value
method
overrideproperty
A property name.
expression Tag
The <expression> tag is a child of a <derivation> tag. It encloses a repository item property name.
One or more <expression> tags define how the value of a derived property is determined. The
<expression> tag has no attributes or child tags. See the Derived Properties section in this chapter.
rql-filter Tag
The <rql-filter> tag can be used to define a filter for database read operations. The <rql-filter>
tag has no attributes. It encloses a Repository Query Language (RQL) string that defines the filter query.
See the Repository Filtering section in this chapter.
named-query Tag
rql-query Tag
rql Tag
sql-query Tag
sql Tag
input-parameter-types Tag
returns Tag
dependencies Tag
178
12 - SQL Repository Reference
named-query Tag
Child of: item-descriptor Tag
An <item-descriptor> tag can contain any number of <named-query> elements.
Child Tag
How many?
rql-query
zero or one
sql-query
zero or one
A <named-query> tag can have either an <rql-query> child element or a <sql-query> child element,
not both.
rql-query Tag
Child of: named-query Tag
Child Tag
How many?
rql
one
rql Tag
Child of: rql-query Tag
The body of this tag specifies the RQL string to be used in the named query.
sql-query Tag
Child of: named-query Tag
This element defines a specific SQL statement to be used in the named query.
Child Tag
How many?
sql
one
179
12 - SQL Repository Reference
returns
zero or one
input-parameter-types
one
dependencies
one
sql Tag
Child of: sql-query Tag
The body of this tag specifies the SQL string to be used in the named query.
For stored procedures, use the appropriate stored procedure invocation syntax along with the storedprocedure attribute in the <sql> tag:
<sql stored-procedure="true">
input-parameter-types Tag
Child of: sql-query Tag
The <input-parameter-types> element is a comma-separated list of class names that any parameters
in the Query must be an instance of. There must be as many class names as parameters.
returns Tag
Child of: sql-query Tag
The body of this optional tag specifies a comma-separated list of Repository property names that are
returned by this query.
dependencies Tag
Child of: sql-query Tag
If any of the properties specified by the body of the <dependencies> tag are changed, then this query
will be flushed from the query cache.
180
12 - SQL Repository Reference
transaction Tag
You can use a <transaction> tag to group a set of test operation tags. A <transaction> tag takes no
attributes. If a <transaction> tag appears inside of another transaction, the outer transaction is
suspended while the inner transaction occurs, and is resumed again at the end of the inner transaction.
Note that <add-item> tags in this element are processed one at a time. They cannot make forward
references to other items and no attempt is made to satisfy database integrity constraints (beyond that
automatically done with the cascade operator). Use the <import-items> tag if you want to load in items
with forward references.
All test operation tags are enclosed in a single transaction by default, so you do not need a
<transaction> tag. However, to avoid possible database deadlocks, you should place all test operation
tags inside <transaction> tags. For example:
For example, suppose you have this pattern:
The <print-item> tag will not find item 1 because that item has not been committed yet. In addition,
you can run into deadlocks with this pattern if you try to access or modify items that may be locked by
operations in the outer tag. Instead, you should typically use a pattern like:
<transaction>
<add-item item-descriptor="foo" id="1"/>
</transaction>
<transaction>
<print-item item-descriptor="foo" id="1"/>
</transaction>
Child Tag
How many?
add-item
zero or more
export-items
zero or more
print-item
zero or more
query-items
zero or more
remove-all-items
zero or more
181
12 - SQL Repository Reference
remove-item
zero or more
rollbacktransaction
zero or more
transaction
zero or more
update-item
zero or more
rollback-transaction Tag
The <rollback-transaction> tag is used only in the <transaction> test operation tag, to mark the
transaction as rollback only. It must be empty and has no child tags or attributes.
import-items Tag
The <import-items> tag is a procedural tag that can be used to add items to a repository in a more
complex way than is possible using <add-item> tags in a <transaction> tag.
add-item Tag
add-item Child Tag
Child Tag
How many?
set-property
zero or more
Attribute
Description
Value
itemdescriptor
Required
182
12 - SQL Repository Reference
id
Optional
repository
Optional
on-commit
Boolean
skip-add
Boolean
tag
String
update-item Tag
See Updating Items in the Development, Testing and Debugging with the SQL Repository section of this
chapter.
Child Tag
How many?
set-property
zero or more
Attribute
Description
Value
item-descriptor
Required.
183
12 - SQL Repository Reference
id
tag
skip-update
query-items Tag
The <query-items> tag has no child tags. See Querying Items in the Development, Testing and
Debugging with the SQL Repository chapter.
This tag can also be used for loading caches. See Cache Loading in the SQL Repository Caching chapter.
Attribute
Description
Value
item-descriptor
Required.
query
The query to issue against the itemdescriptor. Instead of using this attribute,
it is usually more convenient to specify
the query in the body of the query-items
tag in a PCDATA section.
An RQL query.
id-only
quiet
print-content
print-item Tag
The <print-item> tag is always empty and has no child tags.
184
12 - SQL Repository Reference
Attribute
Description
Value
folder
id
Optional. RepositoryId of
item.
item-descriptor
Required.
path
print-content
tag
set-property Tag
The <set-property> tag is used only in the <add-item> and <update-item> test operation tags. It has
no child tags.
To set the value of an Array, List, or Set property, use a comma-separated list of values:
<set-property name="interests" value="fishing,fussing,wrassling"/>
To set the value of a Map property, use a comma-separated list of key=value pairs:
<set-property name="homes"
value="Jefferson=Monticello,Jackson=Hermitage,Madison=Montpelier"/>
To add or remove a value to a multi-valued property, use the Boolean add or remove attributes. For
example, to add a value to the preceding example:
<set-property name="homes" value="Buchanan=Wheatland" add="true"/>
185
12 - SQL Repository Reference
To set the value of a property that refers to another repository item, use the ID of the other repository
item:
<set-property name="bestBuddy" value="10022349_5"/>
Attribute
Description
Value
Name
Required
Value
Required
Add
Optional
Remove
Optional
remove-item Tag
The <remove-item> tag is a procedural tag for removing items from the repository. It has no child tags
and the following attributes:
Attribute
Description
Value
item-descriptor
Required.
id
RepositoryId of item.
tag
Optional.
removereferences-to
remove-all-items Tag
The <remove-all-items> tag is a procedural tag for removing all items in the repository. This tag is
enabled only if the system property atg.allowRemoveAllItems is set on application startup. You can
186
12 - SQL Repository Reference
export-items Tag
The <export-items> tag is a procedural tag for exporting the data required to recreate one or more
item descriptors. The data is exported as XML to standard output. Using this tag is similar to running the
startSQLRepository script with the -export argument. See the startSQLRepository Script and the
Template Parser section in the Development, Testing and Debugging with the SQL Repository chapter.
item-descriptors
load-items Tag
The <load-items> tag can be used to load the item cache. Loading an item cache can improve
performance, since otherwise it can take some time for the normal run of queries to fill the caches.
Attribute
Description
Value
item-descriptor
Required
properties
Optional
load-all-items
Optional. Default is
false.
quiet
Optional. Default is
false.
187
12 - SQL Repository Reference
The <load-items> tag has no child tags. Its body is a comma-separated list of the repository IDs of the
items that should be loaded into the item cache.
dump-caches Tag
The <dump-caches> tag can be used to print out the contents of the item cache for one or more item
descriptors. It has no child tags.
Attribute
Description
Value
dump-type
might cause the following to be logged, if there were four product items in the cache:
============== START BUFFER PRECACHE ==============
<load-items item-descriptor="product">
prod100003,prod100002,prod100001,prod10001
</load-items>
=============== END BUFFER PRECACHE===============
print-ddl Tag
The <print-ddl> tag has no child tags. You can use it to print the DDLs used. The data is exported to
standard output. Using this tag is similar to running the startSQLRepository script with the outputSQL or -outputSQLFile <file> argument. See the startSQLRepository Script and the Template
Parser section in the Development, Testing and Debugging with the SQL Repository chapter.
188
12 - SQL Repository Reference
Attribute
Description
Value
database-name
<!-====================================================================
gsa_1.0.dtd - document type for GSA templates
====================================================================
-->
<!-- Flag datatype, and values -->
<!ENTITY % flag "(true | false)">
<!-- The whole template -->
<!ELEMENT gsa-template (header?,
(item-descriptor | add-item | update-item | print-item | remove-item |
transaction | query-items | remove-all-items | export-items | import-items |
print-ddl | dump-caches | load-items)*)>
<!-- The header -->
<!ELEMENT header (name?, author*, version?, description?)>
<!-- Name of template -->
<!ELEMENT name (#PCDATA)>
<!-- The author(s) -->
<!ELEMENT author (#PCDATA)>
189
12 - SQL Repository Reference
190
12 - SQL Repository Reference
query-expire-timeout CDATA
#IMPLIED
id-separator CDATA ":"
versionable %flag;
#IMPLIED
>
<!-- Property tag - defines one property descriptor for an item descriptor -->
<!ELEMENT property (derivation?, (option | attribute)*)>
<!ATTLIST property
name
CDATA
#REQUIRED
column-name
CDATA
#IMPLIED
column-names
CDATA
#IMPLIED
property-type
CDATA
#IMPLIED
data-type
CDATA
#IMPLIED
data-types
CDATA
#IMPLIED
item-type
CDATA
#IMPLIED
sql-type
CDATA
#IMPLIED
sql-types
CDATA
#IMPLIED
component-item-type CDATA #IMPLIED
component-data-type CDATA #IMPLIED
display-name
CDATA
#IMPLIED
display-name-resource CDATA #IMPLIED
description
CDATA
#IMPLIED
description-resource
CDATA
#IMPLIED
required
%flag;
"false"
readable
%flag;
"true"
writable
%flag;
"true"
queryable
%flag;
"true"
default
CDATA #IMPLIED
hidden
%flag;
"false"
expert
%flag;
"false"
editor-class
CDATA
#IMPLIED
category
CDATA
#IMPLIED
category-resource
CDATA
#IMPLIED
cascade
CDATA
#IMPLIED
repository
CDATA
#IMPLIED
cache-mode
%property-cache-mode;
"inherit"
group
CDATA
#IMPLIED
>
<!-- Derived properties have an associated derivation which
specifies how the dervied property values are derived -->
<!ELEMENT derivation (expression+)>
<!ATTLIST derivation
method
CDATA
#IMPLIED
user-method
CDATA
#IMPLIED
override-property
CDATA
#IMPLIED
>
<!-- A derived property expression, when evaluated
specifies a value used in deriving a derived
property value -->
191
12 - SQL Repository Reference
"auxiliary"
#IMPLIED
#IMPLIED
#IMPLIED
#IMPLIED>
192
12 - SQL Repository Reference
the corresponding repository view that this tag is found under -->
<!ELEMENT rql-query (query-name, rql)>
<!ELEMENT sql-query (query-name,
sql,
returns?,
input-parameter-types?,
dependencies?)>
<!ELEMENT sql (#PCDATA)>
<!ATTLIST sql
stored-procedure %flag; #IMPLIED
>
<!ELEMENT returns (#PCDATA)>
<!ELEMENT input-parameter-types (#PCDATA)>
<!ELEMENT dependencies (#PCDATA)>
<!-- The query-name element, which indicates the user-defined name of a named
query instance -->
<!ELEMENT query-name (#PCDATA)>
<!-- The transaction element. It surround the operation elements
add-item, print-item etc. Note that add-item tags in this element
are processed one at a time. They cannot make forward references
to other items and no attempt is made to satisfy database integrity
constraints (beyond that automatically done with the cascade operator)
Use the import-items tag if you want to load in items with forward
references. -->
<!ELEMENT transaction (add-item | update-item | print-item | remove-item |
transaction | query-items | remove-all-items |
export-items | load-items | rollback-transaction)*>
<!-The import-items element. This tag only contains add-item tags. These
tags can contain forward references. The tags are processed in three
passes - pass one creates all items. Pass two, sets required properties
and optional properties which do not refer to other items. Pass three
sets the remaining properties and updates the item -->
<!ELEMENT import-items (add-item)*>
<!-- Procedural tags for adding and modifying items -->
<!ELEMENT add-item (set-property*)>
<!ATTLIST add-item
item-descriptor CDATA
#REQUIRED
id
CDATA
#IMPLIED
tag
CDATA
#IMPLIED
on-commit
CDATA
#IMPLIED
skip-add
CDATA
#IMPLIED
repository
CDATA
#IMPLIED
no-checkin
%flag;
"false"
>
193
12 - SQL Repository Reference
<!-Procedural tag for removing all items. Only enabled if the system
property atg.allowRemoveAllItems is set on startup
-->
<!ELEMENT remove-all-items EMPTY>
<!ATTLIST remove-all-items
item-descriptor CDATA
#REQUIRED
>
<!-- Procedural tag for exporting the data required to recreate one or more
item-descriptors. The item-descriptors attribute specifies a comma
separated list of one or more item descriptor names. If none are
specified, all item-descriptors are exported -->
<!ELEMENT export-items EMPTY>
<!ATTLIST export-items
item-descriptors CDATA
#IMPLIED
skip-references %flag;
"false"
>
<!-- Procedural tag for querying and printing an item -->
<!ELEMENT query-items (#PCDATA)>
<!ATTLIST query-items
item-descriptor CDATA
#REQUIRED
query
CDATA
#IMPLIED
print-content
CDATA
#IMPLIED
quiet
%flag;
"false"
id-only
%flag;
"false"
>
<!-- Procedural tag for caching a list of items -->
<!ELEMENT load-items (#PCDATA)>
<!ATTLIST load-items
item-descriptor CDATA
#REQUIRED
194
12 - SQL Repository Reference
properties
load-all-items
quiet
CDATA
%flag;
%flag;
#IMPLIED
"false"
"false"
>
<!-- Procedural tag for printing an item -->
<!ELEMENT print-item EMPTY>
<!ATTLIST print-item
item-descriptor CDATA
#IMPLIED
path
CDATA
#IMPLIED
folder
CDATA
#IMPLIED
id
CDATA
#IMPLIED
tag
CDATA
#IMPLIED
print-content
CDATA
#IMPLIED
>
<!-- Sets a property value. Used only in the add-item and update-item tags -->
<!ELEMENT set-property (#PCDATA)>
<!ATTLIST set-property
name
CDATA
#REQUIRED
value
CDATA
#IMPLIED
add
%flag;
"false"
remove
%flag;
"false"
>
<!-- Sets a property value. Used only in the add-item and update-item tags -->
<!ELEMENT rollback-transaction EMPTY>
<!-- Procedural tag for printing the DDL needed -->
<!ELEMENT print-ddl EMPTY>
<!ATTLIST print-ddl
database-name CDATA
#IMPLIED
>
<!-- Procedural tag for dumping the caches of one or more
item-descriptors. The item-descriptors attribute specifies a comma
separated list of one or more item descriptor names. If none are
specified, all of the caches for the repository are dumped. The
dump-type attribute specifies if the output should be formatted as
a list of item ids or as XML that can be later used to pre-cache
the items -->
<!ELEMENT dump-caches EMPTY>
<!ATTLIST dump-caches
item-descriptors CDATA
#IMPLIED
dump-type (debug|queries|both)
"debug"
>
195
12 - SQL Repository Reference
This section includes a number of simple examples of SQL repository definition files and the
corresponding SQL statements to create the tables described by the definition files. These examples
demonstrate a variety of data relationship mappings:
Repository Example F: One-to-Many Mapping to Other Repository Items maps a oneto-many relationship. It defines two item types, user and address. Each user can
have many addresses.
196
12 - SQL Repository Reference
VARCHAR(32)
VARCHAR(32)
INTEGER null,
not null,
null,
197
12 - SQL Repository Reference
<header>
<name>Repository Example Version B</name>
<author>Pat Durante</author>
<description>
This template maps a repository item to a
primary table and an auxiliary table (a one-to-one
relationship.) Each user has a job title and
function.
</description>
</header>
<item-descriptor name="user" default="true">
<table name="usr_tbl" type="primary" id-column-names="id">
<property name="id" data-type="string"/>
<property name="name" column-names="nam_col" data-type="string"/>
<property name="age" column-names="age_col" data-type="int"/>
</table>
<table name="job_tbl" type="auxiliary" id-column-names="id">
<property name="function"/>
<property name="title"/>
</table>
</item-descriptor>
</gsa-template>
VARCHAR(32)
VARCHAR(32)
INTEGER null,
not null,
null,
VARCHAR(32)
VARCHAR(32)
VARCHAR(32)
198
12 - SQL Repository Reference
VARCHAR(32)
VARCHAR(32)
INTEGER null,
not null,
null,
199
12 - SQL Repository Reference
200
12 - SQL Repository Reference
VARCHAR(32)
VARCHAR(32)
INTEGER null,
not null,
null,
201
12 - SQL Repository Reference
202
12 - SQL Repository Reference
<header>
<name>Repository Mapping Example Version F</name>
<author>Ben Erwin</author>
<description>
This template maps out a one-to-many relationship
between user items and address items. It
demonstrates the use of the component-item-type
attribute (which allows one repository item
to contain other repository items.) Each user
item will contain many address items (home address,
business address, etc.)
</description>
</header>
<item-descriptor name="address">
<table name="addr_tbl" type="primary" id-column-name="addr_id">
<property name="user" column-name="user_id" item-type="user"/>
<property name="street" data-type="string"/>
<property name="city" data-type="string"/>
</table>
</item-descriptor>
<item-descriptor name="user" default="true">
<table name="usr_tbl" type="primary" id-column-name="id">
<property name="id" data-type="string"/>
<property name="name" column-name="nam_col" data-type="string"/>
<property name="age" column-name="age_col" data-type="string"/>
</table>
<table name="addr_tbl" type="multi" id-column-name="user_id">
<property name="addresses" column-name="addr_id" data-type="set"
component-item-type="address"/>
</table>
</item-descriptor>
</gsa-template>
203
12 - SQL Repository Reference
primary key(addr_id)
);
<item-descriptor name="author">
<table name="author" type="primary" id-column-name="author_id">
</table>
<table name="book" type="multi" id-column-name="author_id"
multi-column-name="sequence_num">
<property name="books" data-type="list" component-item-type="book"
column-name="book_id"/>
</table>
</item-descriptor>
<item-descriptor name="book">
<table name="book" type="primary" id-column-name="book_id">
<property name="author" item-type="author" column-name="author_id"/>
<property name="seq" data-type="int" column-name="sequence_num"/>
</table>
</item-descriptor>
You need to use the List data type to represent the ordered many side of the
relationship.
The sequence_num and author_id columns in the book table cant be specified as
not null, since the SQL Repository will try to set these fields to null when items in the
List are removed.
The book item descriptor needs to define a property to point to the sequence_num
field, like this:
<property name="seq" data-type="int" column-name="sequence_num"/>
204
12 - SQL Repository Reference
205
12 - SQL Repository Reference
not null,
null,
null,
VARCHAR(32)
not null,
VARCHAR(32)
null,
INTEGER
null,
206
12 - SQL Repository Reference
207
12 - SQL Repository Reference
not null,
null,
not null,
null,
not null,
null,
208
12 - SQL Repository Reference
component, or create your own. An ATG instance can have any number of repository components
running at the same time.
The repository is displayed in the ATG Control Center under the name specified by the <display-name>
tag. The repositorys repositoryName property must match the value specified by the <repositoryname> tag in the /atg/devtools/admins.xml file. For example:
Property
Description
$class
class name
atg.adapter.gsa.
GSARepository
209
12 - SQL Repository Reference
autoCommitInitialization
If setAutoCommit and
boolean
localTransactionModeInit
ialization are both true,
true
boolean
true
das_gsa_subscriber
database table?
cacheRestoreFile
If restoreCacheOnRestart is
true, then an XML file used to
reload item caches on restart is
written to this location.
File
cacheSwitchHot
boolean
false
SwitchingDataSourceEvent
databaseName
boolean
false
210
12 - SQL Repository Reference
databaseTableInfo
dataSource
javax.sql.DataSource
int
debugLevel
/atg/dynamo/service/
jdbc/<pool name>
disableItemCaches
AtStartup
XMLFile[]
false
No default
211
12 - SQL Repository Reference
disableQueryCaches
AtStartup
false
enforceRequiredProperties
boolean
true
escapeWildcards
true
wildcardEscapeCharacter
groupContainer
The GSAEventServer
component that handles cache
invalidation messages for item
descriptors using distributed
cache mode.
/atg/dynamo/server/
/atg/registry/Repository
Groups
SQLRepositoryEventServer
212
12 - SQL Repository Reference
idGenerator
atg.service.idgen.
IdGenerator
/atg/dynamo/service/
IdGenerator
itemDescriptorAliases
Properties
itemDescriptorAliases=
All Profiles=user
localeSensitiveSorting
int
boolean
200
false
213
12 - SQL Repository Reference
localTransactionModeIniti
alization
boolean
true
TransactionDemarcation
A ClientLockManager to use
for locked mode caching. See
the SQL Repository Caching
chapter.
ClientLockManager
metaDataSchemaPattern
String
metaDataCatalogName
String
outerJoinSupport
sql92
jdbc
plus-equals (Oracle)
star-equals (Microsoft or
DYNAMO
Sybase)
informix
none
pathSeparator
repositoryName
String
Name of repository
String
SQLRepository
214
12 - SQL Repository Reference
boolean
safeSQLTypes
null
setAutoCommit
boolean
Connection.setAutoCommit
() as needed. If false, the
false
restoreCacheOnRestart
false
boolean
false
215
12 - SQL Repository Reference
skipNullProperties
SQLLowerFunction
storeTransientItems
subscriberRepository
boolean
String
boolean
GSARepository
false
lower
true
/atg/dynamo/service/jdbc
/SQLRepository
/atg/dynamo/service/jdbc
/SQLRepository repository. If
boolean
false
216
12 - SQL Repository Reference
transactionManager
updateSchemaInfoCache
useCacheForDelete
userPropertyDescriptors
A TransactionManager to use
for all transactions. All code in
the same server should
typically use the same
TransactionManager.
javax.transaction.
TransactionManager
Boolean
boolean
Class[]
/atg/dynamo/transaction/
TransactionManager
false
false
217
12 - SQL Repository Reference
useSetAsciiStream
useSetBinaryStream
useSetObject
useSetUnicodeStream
If useSetAsciiStream is set to
true, then the SQL repository
always uses
setAsciiStream() instead of
setString() in prepared
statements. You can
useSetAsciiStream instead
of useSetUnicodeStream, but
then you lose the ability to
handle internationalized values
in the database.
boolean
If useSetBinaryStream is set
to true, then the SQL
repository always uses
setBinaryStream() instead
of setBytes() in prepared
statements. The
setBinaryStream() is
required for large byte arrays in
some JDBC drivers.
boolean
boolean
If useSetUnicodeStream is set
to true, then the SQL repository
always uses
setUnicodeStream() instead
of setString() in prepared
statements. The
setUnicodeStream() method
is required for large Strings in
some JDBC drivers. Setting
useSetUnicodeStream to
true is recommended if you
are using Oracle with
internationalized content, but is
not recommended if you do
not have internationalized
content in your database. Note
that if you are using MS SQL
Server, you must set
useSetUnicodeStream to
false.
boolean
false
false
false
true
218
12 - SQL Repository Reference
boolean
wildcardEscapeCharacter
XMLToDomParser
atg.xml.tools.
XMLToDomParser
XMLToolsFactory
An XMLToolsFactory to use in
parsing XML templates.
atg.xml.tools.
useTransactionsForCached
Reads
false
XMLToolsFactory
/atg/dynamo/service/xml/
XMLToolsFactory
219
12 - SQL Repository Reference
220
12 - SQL Repository Reference
221
13 - SQL Content Repositories
storys byline, dateline, length, and keywords, while the content would include the text of the story itself.
You can adopt one of two basic architectural styles when you set up a SQL content repository:
You can store both the content and the metadata in your SQL database. A content
repository item would include a property whose value was the content.
You can store the metadata in the database, and the content in your file system. In this
case, the metadata would include properties that indicate how to look up the content
in the file system. A content repository item would include a property whose value
was a pointer to the content in the file system.
As with other repositories, setting up a SQL content repository involves the following steps:
1.
Design the item types you want to include in your content repository. For each type of
repository item, decide what sorts of properties you want to have available to you for
searching and targeting content in the repository.
2.
Set up a SQL database containing content repository items, to act as the data store of
the repository.
3.
Create a repository definition. This is an XML file that describes the repositorys item
descriptors and property descriptors, and defines the relationship among these items
and the rows and tables of the database. See Creating a SQL Content Repository
Definition.
4.
Configure a SQL Repository component that interacts with the data store you set up in
step 2 to create, modify, and retrieve repository items. See Configuring a SQL Content
Repository.
A repository that contains content items must include item descriptors flagged with the folder and
content attributes of the <item-descriptor> tag in the SQL repository definition.
A repository that contains content items must include one item descriptor flagged
with the folder and one or more item descriptors flagged with the content
attributes of the <item-descriptor> tag in the SQL repository definition. See Folder
and Content Item Descriptors.
The folder item descriptor and the content item descriptors must define properties
that define the pathname of each folder and content item. These properties are used
to retrieve the content item and identify the place of each folder or content item in the
content repositorys hierarchical namespace. See Path and Item ID Attributes.
222
13 - SQL Content Repositories
one and only one item descriptor that defines repository items that act as folders (the
folder item descriptor) and
one or more item descriptors that define content repository items (the content item
descriptors).
use-id-for-path
content-name-property
content-path-property
Regardless of how you store path information in the database, you can get the path of an item with this
method in the atg.repository.content.FolderItem interface (which is extended by the
ContentRepositoryItem interfaces):
public String getItemPath()
This method returns the path of this item, represented as a relative path from the repositorys root folder
use-id-for-path
This is the simplest mechanism. In this mode, the relative path name of the item is used as the ID of the
repository item. Your database must then include an ID column that is a string large enough to hold the
entire relative path name of each folder item and content item. Put this ID column in your primary table
and set the id-column-name attribute of the primary table to point to this column. You then set useid-for-path="true" for that item descriptor. For example:
<item-descriptor name="folder" folder="true"
use-id-for-path="true" folder-id-property="folder-id">
<table name="folder" id-column-name="id">
<property name="id" column-names="id"/>
<property name="folder-id" column-names="folder_id"/>
...
</table>
</item-descriptor>
223
13 - SQL Content Repositories
The use-id-for-path mode may not work if you have an existing database schema that does not follow
this format. This approach also might not work if path names in your repository could be longer than the
size of varchar you can efficiently store and query against in your database. Some databases impose a
255-character limit on the size of queryable columns. This may be too small to hold the entire path for
some content repositories.
Note that even though you put the entire path name in the property designated by the id-columnnames attribute, you still need to use the folder-id-property attribute to designate a property that
holds the name of the parent folder of the item. In the preceding example, the folder-id property holds
the name of the folder.
content-name-property
You can set the item descriptors content-name-property attribute. In this case, you can store just the
name of the repository item itself (rather than the entire path name) in one property of the repository
item and use the content-name-property to designate the name of this property. The content-nameproperty specifies the property representing the name of the folder item or content item, while the
folder-id-property specifies the property representing the parent folder of the folder item or content
item. From these two pieces of information, we can compute the path for a given item by walking up the
content hierarchy.
The operation of computing the path for this item will be more expensive in this mode, since we have to
query up the folder hierarchy to compute the path for an item rather than getting the path from a single
property. However, this mode can overcome the problem of the size limitation on queryable columns.
Now your column size for the content name limits the size of each individual component of the file name,
not the size of the entire path.
A folder-id-property is required for all content repositories, whichever method they use to specify
how path information is stored in the database. The data-type of the folder-id-property can be either
data-type="string" (or whatever type you define your repository IDs to be), or you can specify that its
item-type is the name of the folder item descriptor. This enables you to conveniently access folder
information from the item itself. For example:
<item-descriptor name="folder" folder="true"
content-name-property="filename" folder-id-property="folder-id">
<table name="folder" id-column-names="id">
<property name="filename" data-type="string"/>
<property name="folder-id" item-type="folder"/>
...
</table>
</item-descriptor>
224
13 - SQL Content Repositories
Since this content-name property is not guaranteed to be unique across the repository, you have a
separate column for the ID of this repository
content-path-property
You might not be able to use the repository items path as its repository ID. If that is the case, perhaps due
to references to these rows from other tables, and if you can store the entire path name of the item as a
column in your table, then you can use a third alternative. In this mode, you can set the content-pathproperty to refer to a property that holds the path name of the item. You would then use a separate
property and column in your table to refer to the ID for this item. For example:
<item-descriptor name="folder" folder="true"
content-path-property="pathname">
<table name="folder" id-column-names="id">
<property name="id" data-type="long"/>
<property name="pathname" data-type="string"/>
...
</table>
</item-descriptor>
225
13 - SQL Content Repositories
You configure your content item descriptors by naming the properties that are to be used to retrieve each
of these values. This is done using the following attributes in the <item-descriptor> tag:
Attribute Name
Description
content-path-property
content-name-property
Refers to the name of this folder or content item (not the full path
name)
content-property
For content item descriptors, this is the name of the property that
holds the content itself.
last-modifiedproperty
226
13 - SQL Content Repositories
<item-descriptor name="book"
display-property="title"
content="true"
content-property="bookcover_image"
content-path-property="filePath"
folder-id-property="parentFolder">
<table name="book_info" id-column-name="id" type="primary">
<property name="filePath" data-type="big string"/>
<property name="parentFolder" item-type="book_folder"/>
<property name="title" data-type="big string"/>
<property name="author" data-type="big string"/>
</table>
<property name="bookcover_image"
property-type="atg.repository.FilePropertyDescriptor"/>
</item-descriptor>
227
13 - SQL Content Repositories
property-type="atg.repository.FilePropertyDescriptor"
As described in the Storing Content on a File System section, this property type indicates that the
repository should use the propertys path name and convert it to a java.io.File object.
<item-descriptor name="book_folder"
display-property="folderPath"
folder="true"
content-path-property="folderPath"
folder-id-property="parentFolder">
<table name="book_folder" id-column-name="id" type="primary">
<property name="parentFolder" item-type="book_folder"/>
<property name="folderPath" data-type="big string"/>
</table>
</item-descriptor>
This item type is specified to be a folder, using the attribute folder="true". The folder-id-property
attribute in the item-descriptor tag indicates that this item stores its parent folder ID in the database using
a property named parentFolder.
Both the book and the book_folder item type stores their paths in the database using the contentpath-property attribute. The content-path-property attribute indicates the property of this item
that defines the absolute path name of this item in the folder hierarchy. In this example, the path is stored
in the property named folderPath. If our example had especially deep hierarchies, resulting in
excessively long path names, we might instead store just the name of this item using the content-nameproperty attribute, and have the repository calculate the items absolute path by determining its parent
folders, using the property indicated by the folder-id-property attribute.
228
13 - SQL Content Repositories
folder-id-property="parentFolder">
not null,
null references book_folder(id),
null,
not null,
null,
null references book_folder(id),
null,
229
13 - SQL Content Repositories
author
primary key(id)
long varchar
null,
);
2.
3.
230
13 - SQL Content Repositories
231
13 - SQL Content Repositories
232
13 - SQL Content Repositories
14 Repository Loader
In many applications, it makes sense from a workflow and production point of view to create the content
of repository items outside of ATG, on a file system, and subsequently load the items into the repository.
The ATG Repository Loader provides a flexible way to take files that are stored in a file system, convert
them into repository items, and load the items into the repository.
In the ATG platform, the Repository Loader has been implemented to handle the loading of HTML files,
XML files, and binary media items (like image or audio files) into a SQL repository. The Repository Loader
uses ATGs xml2repository system to transform files into XML files that use the xml2repository schema,
which are then transformed into repository items. The xml2repository system is described in full in the
Repository to XML Data Binding chapter of the ATG Web Services and Integration Framework Guide. Make
sure you are familiar with that chapter, as it is essential to understanding how to use the Repository
Loader.
The Repository Loader has been designed to be flexible. When you configure the Repository Loader, you
can choose to let the Repository Loader scan the file system, identify the files that need to be loaded, and
load the files, either on command or on a specified schedule. Alternatively, you can submit to the
Repository Loader a manifest file that specifies the files to be loaded. The first alternative requires less
effort on the part of the Web site administrators; the second alternative loads faster and puts less of a
burden on your system. Which to choose depends in part on the size of your repository. The larger the
number of files that need to be scanned, the longer it will take to load them, and the greater the burden
that file system scanning will place on your system.
This chapter includes the following sections:
Repository Loader Architecture
Using the Repository Loader
Configuring the Repository Loader Components
Repository Loader Example
FileSystemMonitorService
233
14 - Repository Loader
FileSystemMonitorScheduler
LoaderManager
FileSystemMonitorScheduler
FileSystemMonitorService
RLClient
FormHandlers:
CreateJob
Delete Job
TypeMapper
LoaderManager
ContentHandler
TypeMappings
Add
Update
Remove
xml2repository services
Repository
Data
base
234
14 - Repository Loader
FileSystemMonitorService
The FileSystemMonitorService implements the interfaces
atg.repository.loader.FileSystemMonitor and
atg.repository.loader.FileSystemManager. A component of the FileSystemMonitorService
class can be configured to scan a specified file directory and identify the files that need to be uploaded.
You can limit the files to be uploaded in three ways, so that the files include:
only files modified since the time the file system was last scanned;
Once the files have been identified by the FileSystemMonitorService component, the list of files is
passed to a component of the LoaderManager class.
See Configuring the FileSystemMonitorService for details about this components properties.
FileSystemMonitorScheduler
The FileSystemMonitorService can be set to scan the file system on a periodic schedule. The schedule
is set and controlled in a FileSystemMonitorScheduler component. Note that the
FileSystemMonitorService can also be set to scan the file system programmatically, using its add,
delete, and update methods, or through the Repository Loader administration application, which
provides JSP pages and form handlers that invoke FileSystemMonitorService methods. See
Repository Loader Form Handlers and Administration Pages.
See Configuring the FileSystemMonitorScheduler for details about this components properties.
LoaderManager
The LoaderManagerImpl implements the interface atg.repository.loader.LoaderManager. A
component of the LoaderManagerImpl class accepts Repository Loader jobs, initiated either by the
FileSystemMonitorService or, if you are supplying a manifest of files to be loaded, from the
Repository Loaders RMI client (see Repository Loader RMI Client). The LoaderManger component passes
the files to a TypeMapper component, which determines how the file should be processed. Or, the
TypeMapping can be specified directly or in a manifest file.
See Configuring the Loader Manager for details about this components properties.
235
14 - Repository Loader
Files are routed to ContentHandlers, based on the mapping of file types and item descriptors
established by the TypeMappings. The ContentHandler is responsible for carrying out the process of
transforming the file into a repository item and invoking a back end system to perform add, update, and
remove operations on the repository. In the ATG platform, a single class of ContentHandler is included
named Xml2RepositoryContentHandler. This class uses ATGs xml2repository system to transform files
into XML files that use the xml2repository schema, which are then transformed into repository items.
The xml2repository system is described in full in the Repository to XML Data Binding chapter of the ATG
Web Services and Integration Framework Guide. Make sure you are familiar with that chapter, as it is
essential to understanding how to use the Repository Loader.
For example, if you use JBoss (and the default JBoss port) and your browser runs on the same machine as
your Web server, you would use the following URL:
http://localhost:8080/rl
To find the default port, see the ATG Installation and Configuration Guide. For information about
assembling applications, see the ATG Programming Guide.
The Repository Loader Administration consists of a set of JSP pages that use Repository Loader form
handlers. On these pages, you can create, delete, monitor, and view the results of Repository Loader jobs.
The Repository Loader Administration reports on the status of Repository Loader jobs, including the
success or failure, number of files, and errors encountered. Click the Get Detail link for a job to get more
information.
236
14 - Repository Loader
For more details about using the Repository Loader Administration, see the Repository Loader Example
section.
Error Policies
The LoaderManager uses a configurable error handling policy, defined by the Repository Loaders
ErrorPolicy component. Each job processed by the LoaderManager might contain files that cause
exceptions to be thrown. The LoaderManager consults the ErrorPolicy component to determine how
to handle exceptions. All exceptions are logged. The Repository Loader sets a success state for each job
and each batch. In addition, the following ErrorPolicy methods are used to determine how to proceed
after encountering a problem while processing a job:
checkIsExceptionFatal
checkRequiresNewTransaction
checkEndTransactionWithBatchState
Make sure you have set your DYNAMO_HOME variable before you run the script. The RLClient
arguments are as follows:
237
14 - Repository Loader
Argument
Description
-m <manifestFilePath>
-mp <manifestFilePath>
-p <propertiesFilePath>
-h <hostname>
-r <RMIPort>
-auth <username:password>
-s <servicename>
RLClient Hints
You can optionally pass the address of a properties file as an argument to the RLClient containing
additional parameters and hints to be used by the LoaderManager. This hints file can include the
following:
batchSize
numElementsInManifest
For example:
238
14 - Repository Loader
atg.repository.loader.batchSize=2
atg.repository.loader.numElementsInManifest=725
If you dont specify a batch size, the RLClient uses the LoaderManagers defaultBatchSize property.
<manifest>
<add>/main/Dynamo/RL/sample-data/user001.xml</add>
<add>/main/Dynamo/RL/sample-data/user002.xml</add>
<add>/main/Dynamo/RL/sample-data/user003.xml</add>
<add>/main/Dynamo/RL/sample-data/user004.xml</add>
<add>/main/Dynamo/RL/sample-data/user005.xml</add>
</manifest>
The elements in a Repository Loader manifest file are handled in the order that they appear. As a result,
you need to take care that an element does not depend on a subsequent element. For example,
remember that a content repository requires a folder hierarchy; do not attempt to add a content item
before you add the folder that is to contain it.
The next two sections, Repository Loader Manifest File Document Type Definition and Repository Loader
Manifest File Tags, provide more details about the XML format used in creating a Repository Loader
manifest file.
<!-====================================================================
rl-manifest_1.0.dtd - document type for Repository Loader manifests
Version: $Id: //product/DAS/main/Java/atg/dtds/rl/rl-manifest_1.0.dtd#1 $
$Change: 286550 $
====================================================================
-->
239
14 - Repository Loader
<!-- A single manifest composed of any number of add, update, remove tags -->
<!ELEMENT manifest (add | update | remove)*>
<!ATTLIST manifest num-elements CDATA #IMPLIED>
<!ELEMENT add (#PCDATA)>
<!ATTLIST add type-mapping CDATA #IMPLIED>
<!ELEMENT update (#PCDATA)>
<!ATTLIST update type-mapping CDATA #IMPLIED>
<!ELEMENT remove (#PCDATA)>
<!ATTLIST remove type-mapping CDATA #IMPLIED>
manifest Tag
A <manifest> tag wraps the entire manifest. It contains one or more of the following child tags:
<add>
<remove>
<update>
The add, remove, and update tags simply contain in their bodies the pathname of the file or folder to be
added, removed, or updated:
<add>/home/Dynamo/RL/sample-data/user001.xml</add>
<update>/home/Dynamo/RL/sample-data/user002.xml</update>
type-mapping Attribute
An add, remove, or update tag can include an attribute named type-mapping. The value of the typemapping attribute is assumed to be the absolute Nucleus path of a component that implements
atg.repository.loader.TypeMapping. If the component can be found and is of the correct class, it
will be used to process the file specified in the tag.
num-elements Attribute
A manifest tag can include an attribute named num-elements. This is an optional attribute that indicates
the total number of add, remove, and update elements in the manifest file.
240
14 - Repository Loader
Configuring a TypeMapping
Automatic Mode
Manifest Mode
Property
Description
Type, Default
createManifestMode
boolean
false
filters
String[]
filters=.html,.htm,.wml
includeFolders
boolean
true
lastScanned
timestamp
0 (all files will load)
FileSystemMonitorService last
executed.
241
14 - Repository Loader
loaderManager
manifestFile
File
RLxxxx.xml
(where xxxx is the unique
createManifestMode="true". This
typeMapper
Property
Description
Type, Default
cancelledJobsFifoSize
int
0
completedJobsFifoSize
int
20
defaultBatchSize
int
-1
defaultTypeMapper
atg.repository.loader.
TypeMapper
/atg/dynamo/service/loader/
FileExtensionTypeMapper
242
14 - Repository Loader
jobEventListeners
atg.repository.loader.
JobEventListener[]
jobIdGenerator
atg.service.idgen.
IdGenerator
/atg/dynamo/service/
FileIdGenerator
jobIdPrefix
String
RLJob
jobIdSpace
String
RLModuleJobIdSpace
jobQueueSize
int
1
loaderErrorEventListeners
atg.repository.loader.
LoaderErrorEventListener[]
null
loaderEventListeners
atg.repository.loader.
LoaderEventListener[]
suspendedJobsFifoSize
int
10
suspendFailedJobs
boolean
true
Property
Description
enabled
lastScannedStorage
The location of a file that stores the time of the last scan
recursiveScan
schedule
scheduler
243
14 - Repository Loader
Property
Description
folderTypeMapping
typeMappings
The ExtFilterTypeMapper class uses the extension of the file name to locate TypeMappings. It includes
the following additional property:
extensions
An array of strings that correspond to the file extensions used by the TypeMapper. For
example:
extensions=.xml,.html
The DirFilterTypeMapper class matches regular expressions (which correspond to directory names) to
TypeMappings. It includes the following additional property:
directories
Configuring a TypeMapping
The Repository Loader includes an implementation of the TypeMapping interface named
TypeMappingImpl. You can configure an instance of TypeMappingImpl for a particular item descriptor
using the following properties:
Property
Description
Property Type
contentIsXML
boolean
parseContent
boolean
244
14 - Repository Loader
itemDescriptorName
contentHandler
String
ContentHandler
ContentRootPathProvider
ContentRootPathProvider
PageEncodingTyper
pathPropertyName
String
repository
MutableRepository
updatePropertyConfiguration
245
14 - Repository Loader
Value
Description
CONTENT_ITEM_DESCRIPTOR_ID_AND_PATH_PROP
CONTENT_DEFINED_ID_AND_NO_PATH_PROP
CONTENT_DEFINED_ID_AND_NAMED_PATH_PROP
GENERATED_ID_AND_NO_PATH_PROP
GENERATED_ID_AND_NAMED_PATH_PROP
ID_EQUALS_FILE_PATH
Note that if a file includes a tag that corresponds to the ID property defined in an item descriptor, then the
value in this tag will be used by the xml2repository function in setting the repository item ID, rather than
the generated ID or the file path ID. Therefore, if you are using any of the following values for the
updatePropertyConfiguration property, you need either to make sure your documents do not
include an ID tag, or else set the parseContent property of the TypeMapping for these documents to
false:
246
14 - Repository Loader
CONTENT_ITEM_DESCRIPTOR_ID_AND_PATH_PROP
ID_EQUALS_FILE_PATH
Property
Description
addService
removeService
updateService
enableTransforms
transformFilter
exceptionOnZeroRemove
idPropertyNameForQueries
For more information about these components, see Repository to XML Data Binding: Repository Operations
in the ATG Web Services and Integration Framework Guide.
The XMLTransformer is a component that can define a set of style sheets to be applied to an XML source.
You can set its styleSheets property to an array of style sheet files to use in transforming XML files. An
instance of the XMLTransformer can be found in Nucleus at
/atg/dynamo/service/loader/typemapping/SampleXMLTransformer. This lets us transform an
XML file at load time into the format required by the xml2repository system. Depending on the number
and complexity of the stylesheets, this might be an expensive operation.
247
14 - Repository Loader
Using automatic mode may require less effort on the part of the Web site administrators. Using manifest
mode results in faster loading and puts less of a burden on your system. If you use a content management
system that can identify for you the files that have changed since your last Web site update, you should
be able to generate a manifest of files to be loaded, updated, or removed fairly easily.
Note that you will need to configure more than just the few properties described in this section. This
section is intended only to bring out the most significant differences between these two modes of
operation.
Automatic Mode
If the file system handled by the Repository Loader is comparatively small, you can configure the
Repository Loader for more automatic operation. You can set the Repository Loader to scan the file
system on command or according to a prescribed schedule and automatically identify the files that need
to be loaded into the repository. As mentioned earlier, what constitutes a comparatively small set of files
will vary, but in any case, a set of 1000 or fewer files may be considered small. To configure the Repository
Loader for automatic operation, set the following properties:
FileSystemMonitorScheduler:
enabled="true"
Set the schedule property to the schedule on which you want the file system to be scanned. For
example:
schedule=every 2 hours in 15 minutes
Manifest Mode
Use the RLClient to load large numbers of files. Use this configuration setting:
FileSystemMonitorScheduler:
enabled="false"
You also need to create a manifest file that lists all the folders and files to be loaded into the repository by
the Repository Loader. See Using a Repository Loader Manifest for information about the manifest file
format.
248
14 - Repository Loader
following item types in its item descriptors. Note how the content item types use item descriptor
inheritance:
Item Descriptor
Name
Description
fileFolder
fileAsset
A content item type. This is the super-type for a series of item types that
inherit from the fileAsset item type. The type property specifies which
subtype (textFileAsset, binaryFileAsset, htmlArticle,
txtPressRelease, xmlPressRelease, gifImage, or jpgImage) an item
belongs to.
A fileAsset item also defines lastModified, size, and parentFolder
properties.
textFileAsset
A content item type that inherits from fileAsset. It is designed for text
files. The text content is stored in the content big string property. It has
subtypes named htmlArticle, txtPressRelease, and
xmlPressRelease.
binaryFileAsset
A content item type that inherits from fileAsset. The content is stored in
the content binary property. It has subtypes named gifImage and
jpgImage.
htmlArticle
txtPressRelease
xmlPressRelease
gifImage
jpgImage
address
A simple non-content item type. Used by the user item types addresses
property.
contact
A non-content item type. Used by the user item types contacts property.
phone
A simple non-content item type. Used by the user item types numbers
property.
user
249
14 - Repository Loader
The user item type demonstrates a variety of data relationships. It shows how an item type can use
properties that nest other item types. The user item descriptor is defined as follows:
250
14 - Repository Loader
multi-column-name="seq_num">
<property name="worstSubjects"
column-names="subject" data-type="list"
component-data-type="string"/>
</table>
<!-- a multivalue property (map) -->
<table name="rlex_credit_card" type="multi" id-column-names="id"
multi-column-name="card_key">
<property name="card_num"
column-names="card_num" data-type="map"
component-data-type="string"/>
</table>
</item-descriptor>
The user item type defines three string properties, id, name, and age, in its primary table. Contact
information about the user is defined in three multi-valued properties, addresses, contacts, and
numbers. The values of each of these multi-valued properties are collections of other repository items
(address, contact, and phone, respectively).
In addition, the user item type defines the following properties:
251
14 - Repository Loader
Item Type
TypeMapping Component
ContentHandler Component
fileFolder
FolderTypeMapping
ContentHandler
textFileAsset
UnparsedContentTypeMapping
ContentHandler
htmlArticle
HtmlArticleTypeMapping
HtmlArticleContentHandler
txtPressRelease
PressReleaseTXTTypeMapping
ContentHandler
xmlPressRelease
PressReleaseXMLTypeMapping
ContentHandler
gifImage
GifImageTypeMapping
ContentHandler
jpgImage
JpgImageTypeMapping
ContentHandler
user
UserTypeMapping
UserContentHandler
The ContentHandler component is used for all the type mappings that dont require XSL preprocessing.
Example TypeMapper
The Repository Loader Example is configured with an ExtFilterTypeMapper component that maps file
extensions to TypeMappings, using the following property configuration:
extensions=.xml,.txt,.gif,.jpg,.html,.eml
typeMappings=pressReleaseXMLTypeMapping,pressReleaseTXTTypeMapping,\
gifImageTypeMapping,jpgImageTypeMapping,htmlArticleTypeMapping,\
userTypeMapping
File Extension
TypeMapping Component
.xml
pressReleaseXMLTypeMapping
.txt
pressReleaseTXTTypeMapping
.gif
gifImageTypeMapping
.jpg
jpgImageTypeMapping
.html
htmlArticleTypeMapping
.eml
userTypeMapping
252
14 - Repository Loader
xml2repository Schemas
The xml2repository system uses XML schemas to represent repository items as XML files. This system lets
you load the repository using XML files that conform to the schema, or export existing repository items in
the form of XML files that can later be loaded. The Repository Loader Example includes an XML schema
for each of the following repository item types. The schemas can be found in the
<ATG2007.1dir>/RL/Example/repository2xml/schemas directory.
Item Type
Schema File
fileFolder
RL-ExampleRepository+fileFolder.xsd
htmlArticle
RL-ExampleRepository+htmlArticle.xsd
txtPressRelease
RL-ExampleRepository+txtPressRelease.xsd
xmlPressRelease
RL-ExampleRepository+xmlPressRelease.xsd
gifImage
RL-ExampleRepository+gifImage.xsd
jpgImage
RL-ExampleRepository+jpgImage.xsd
Dont include any other ATG demo, example, or reference application modules in the application that
includes the Repository Loader because the database connection configurations may conflict.
Once the application is started, you can navigate to this page for the Repository Loader Administration
user interface:
http://hostname:port/rl
where hostname is the machine running your application server and port is the port your application
server uses to listen for HTTP requests. To find the default port, see the ATG Installation and Configuration
Guide.
Then, use the Repository Loader Administration user interface to load the sample files included in the
FileSystemMonitorService root path: <ATG2007.1dir>/RL/Example/j2ee-apps/example/webapp\public :
1.
If you are running on Windows, set the pathSeparator property of the /atg/rlexample/ExampleRepository component to \:
pathSeparator=\
253
14 - Repository Loader
Remember that \ is an escape character in properties files, so if you are setting this
property in a properties file rather than in the ACC, you need to use this format:
pathSeparator=\\
2.
3.
4.
This loads the files from the root path into the ExampleRepository as repository items. The Repository
Loader Administration should show the job as completed.
In addition, the Repository Loader example shows an example of how you can use the xml2repository
feature to export a repository item in the form of an XML file. Go to the following page in your browser:
http://hostname:port/rl-example/itemAsXml.jsp
Note: hostname is the machine running your application server and port is the port your application
server uses to listen for HTTP requests. To find the default port, see the ATG Installation and Configuration
Guide.
This page takes a hard-coded repository item (user001) and outputs it as an XML file. Depending on your
browser settings, you will likely have to view the resulting page as source in order to see the generated
XML. This demonstrates an example of the format used by the xml2repository system. Examining the XML
format generated for a particular item descriptor might help you if you want to generate compliant XML
or write an XSL stylesheet for the Repository Loader to apply before processing a files contents.
254
14 - Repository Loader
The ATG platform provides infrastructure and tools for accessing ATG repositories with Web services. The
Creating Repository Web Services chapter in the ATG Web Services and Integration Framework Guide
describes how to create a Web service that exposes a particular repository item descriptor, or an
individual repository item property. Repository items can be passed via a Web service in the form of an
XML file created with the ATG xml2repository feature, described in the Repository to XML Data Binding
chapter of the ATG Web Services and Integration Framework Guide.
This chapter describes three generalized Web services you can use to provide access to ATG repositories:
These Web services are packaged in the <ATG2007.1dir>/DAS/WebServices directory. For information
about how to deploy these and other Web services, see the ATG Web Services and Integration Framework
Guide.
Note that the descriptions of these Web services include URLs that begin http://hostname:port where
hostname machine running your application server and port is the port your application server uses to
listen for HTTP requests. To find the default port, see the ATG Installation and Configuration Guide.
In addition to these repository Web services, ATG Personalization and ATG Commerce contain several
Web services that provide access to specific ATG repositories and repository item types. A complete list of
the Web services included in the ATG platform can be found in the ATG Web Services and Integration
Framework Guide.
255
15 - Repository Web Services
http://hostname:port/repository/generic
/getRepositoryItem/getRepositoryItem
webservice.GetRepositoryItemSEIImpl
Nucleus Component
/atg/repository/RepositoryServices
Method Name
getRepositoryItem
Input Parameters
String pRepositoryPath
the path of the repository component from which to retrieve the item
String pItemDescriptorName
String
GetRepositoryItem Exceptions
Exceptions generated by the GetRepositoryItem Web service may occur for the following reasons:
RepositoryException
GetException
Parameter
Value
NucleusPath
/atg/repository/RepositoryServices
256
15 - Repository Web Services
MethodName
getRepositoryItem
EarFileName
repositoryWebServices.ear
AppName
RepositoryWebServices
AppDescription
ServletName
getRepositoryItem
ServletDisplayName
GetRepositoryItem
ServletDescription
URLPattern
GetRepositoryItem
WebURI
generic.war
ContextPath
repository/generic
WebAppDisplayName
RepositoryServices
WebAppDescription
Host
Port
ParameterNames
UseSession
true
FunctionalName: - GenericRepositoryUser
2.
SecurityPolicy: - StandardSecurityPolicy
3.
ACL: - Administrators
257
15 - Repository Web Services
atg.repository.RepositoryServices class, which handles all logic, error checking, and result
transformation.
http://hostname:port/repository/generic
/performRQLQuery/performRQLQuery
webservice.PerformRQLQuerySEIImpl
Nucleus Component
/atg/repository/RepositoryServices
Method Name
performRqlQuery
Input Parameters
String pRepositoryPath
The RQL string to execute. Note that this string cannot contain
parameters.
Output
String[]
PerformRQLQuery Exceptions
Exceptions generated by the PerformRQLQuery Web service may occur for the following reasons:
RepositoryException
GetException
258
15 - Repository Web Services
Parameter
Value
NucleusPath
/atg/repository/RepositoryServices
MethodName
performRQLQuery
EarFileName
repositoryWebServices.ear
AppName
RepositoryWebServices
AppDescription
ServletName
performRQLQuery
ServletDisplayName
PerformRQLQuery
ServletDescription
When called, executes the given query against the specified repository
URLPattern
PerformRQLQuery
WebURI
generic.war
ContextPath
repository/generic
WebAppDisplayName
RepositoryServices
WebAppDescription
Host
Port
ParameterNames
UseSession
true
259
15 - Repository Web Services
http://hostname:port/repository/generic
/performRQLCountQuery/performRQLCountQuery
webservice.PerformRQLCountQuerySEIImpl
Nucleus Component
/atg/repository/RepositoryServices
Method Name
performRqlCountQuery
Input Parameters
String pRepositoryPath
The RQL string to execute. Note that this string cannot contain
parameters.
Output
int
PerformRQLCountQuery Exceptions
Exceptions generated by the PerformRQLCountQuery Web service may occur for the following reasons:
RepositoryException
260
15 - Repository Web Services
Parameter
Value
NucleusPath
/atg/repository/RepositoryServices
MethodName
performRQLCountQuery
EarFileName
repositoryWebServices.ear
AppName
RepositoryWebServices
AppDescription
ServletName
performRQLCountQuery
ServletDisplayName
PerformRQLCountQuery
ServletDescription
Performs a repository count query using the given repository path, item
descriptor name and RQL string
WebURI
generic.war
ContextPath
repository/generic
WebAppDisplayName
RepositoryServices
WebAppDescription
Host
Port
ParameterNames
UseSession
true
261
15 - Repository Web Services
262
15 - Repository Web Services
16 Composite Repositories
All ATG repositories provide a means for representing information in a data store as Java objects. The
composite repository provides a means for using more than one data store as the source for a single
repository. The composite repository consolidates all data sources in a single data model, making the data
model flexible enough to support the addition of new data sources. Additionally, the composite
repository allows all properties in each composite repository item to be queryable. Thus, from the point of
view of your ATG application, the composite repository presents a consistent view of your data, regardless
of which underlying data store the data may reside in.
The composite repository is a repository that unifies multiple data sources. Its purpose is to make any
number of repositories appear in an ATG application as a single repository. The composite repository
defines a mapping between item descriptors and properties as they appear to facilities that use the
composite repository and item descriptors and properties of the data models that comprise the
composite data model. A composite repository is composed of any number of composite item
descriptors. Each item descriptor can draw on different data models from different repositories, and map
underlying data model attributes in different ways.
Use Example
Suppose you maintain profile data both in a SQL database and an LDAP directory. ATGs profile repository
ships with a user composite item descriptor comprised of just one primary item descriptor and no
contributing item descriptors. The primary item descriptor is the user item descriptor. You can add to the
composite item descriptor the user item descriptor from the LDAP repository as a contributing item
descriptor. If there are any property name collisions between the SQL repository and the LDAP repository,
you can resolve them by mapping the properties explicitly to different names in the composite repository
configuration. Once youve done this, your ATG applications can view both LDAP profile information and
SQL database profile information as properties of composite items in the composite user item descriptor.
263
16 - Composite Repositories
its unique ID attribute matches the value of the primary items unique ID attribute. If multiple
relationships are defined, then they are AND-ed together.
For example, suppose you have a contributing item descriptor that defines two relationships to the
primary item descriptor. One says that a primary items firstName property must match the contributing
items userFirstName property and the other says that the primary items lastName property must
match the contributing items userLastName. These two relationships together mean that a users first
names and last names must each match for two items to be related. This is useful in situations where no
one property uniquely identifies a user. See link-via-property Tag for an example of defining a relationship
with two or more properties.
Property Derivation
The properties in a composite item descriptor are determined as follows:
1.
If configured to do so, all properties from the primary and contributing item
descriptors are combined into the composite item descriptor, with each property
retaining its property name and property type.
2.
Any properties marked as excluded are removed from the composite item descriptor.
See Excluding Properties.
264
16 - Composite Repositories
3.
All property mappings are performed. This means that a primary or contributing
property that is to be mapped gets renamed in the composite item descriptor. See
Property Mappings.
4.
If there are any two properties in the composite item descriptor that have the same
name, an error results. The composite repository requires that all composite property
names map explicitly to only one primary or contributing property.
Design the composite repository. Pick what item types you want to represent in your
composite repositorys composite item descriptors
2.
Specify the primary item descriptor. This is where the composite repository items
repository item IDs will come from
3.
Specify any contributing item descriptors you need to supplement the primary item
descriptor.
4.
Resolve any property name collisions between properties in the primary item
descriptor and the contributing item descriptors. See Property Mappings.
5.
Determine whether you want to use static or dynamic linking for properties whose
types are repository items. See Link Methods.
6.
Determine what item creation policy you want the composite repository to
implement. See Creating Composite and Contributing Items.
7.
Determine whether there are any properties in your primary or contributing item
descriptors that you want to exclude from the composite item descriptor. See
Excluding Properties.
8.
Property Mappings
The composite repository requires that all composite property names map explicitly to only one primary
or contributing property. If your primary or contributing item descriptors contain one or more properties
that have the same name, you need either to exclude one of the properties (see Excluding Properties) or
map it to a new name.
You can map a property using the mapped-property-name attribute in a property tag in an item
descriptor. For example, suppose you have two contributing item descriptors, each of which has a
property named login. You can map one of the properties to a new name like this:
<property name="ldapLogin" ... mapped-property-name="login"/>
In this example, the name attribute specifies the property name in the composite item descriptor and the
mapped-property-name attribute specifies the name of the property in the primary or contributing item
descriptor to which this property maps.
265
16 - Composite Repositories
Excluding Properties
Sometimes you may not want to expose absolutely every property from the underlying primary and
contributing item descriptors in the composite item descriptor. You can configure the item descriptor to
exclude those contributing properties that are not desired. You do this using a property tag with its
exclude attribute set to true:
<property name="password ... exclude="true"/>
Link Methods
The link-method attribute determines what happens when the composite repository needs to get a
property value that belongs to a contributing repository item. For example, suppose a process calls
CompositeItem.getPropertyValue("ldapFirstName");
When users create a new composite item via the createItem() method in
MutableCompositeRepository, then new instances of the primary item and of all contributing items
will be created. So, for example, if you have a user item type defined in your composite repository that
266
16 - Composite Repositories
borrows properties from the SQL and LDAP repositories, then any new user composite repository item
that is created will create both a SQL repository item and an LDAP repository item. However, before these
items can be added to their respective repositories, the correct link needs to exist between them. If the
items are linked by a certain property, then this property needs to be set on the primary item before the
items are added, otherwise an error will occur since theres no way in the future to link those two items
back together.
lazy
If this option is chosen, contributing items will be created only when they are needed. In this case, when
users call setPropertyValue on a property that is defined in the contributing repository, the composite
repository creates the item in the contributing then and there. There are two different behaviors
depending on whether the CompositeItem is transient or not. If the item is transient, then we wait until
the item is persisted before checking to see that all of the appropriate linking properties have been set, so
that we may propagate them to the new contributing item. If the item is not transient, we check to see if
the correct linking properties are set on the primary item and then add the contributing item to its
repository. If there any properties used for linking are missing, then an error is returned.
The check for valid linking properties occurs during the updateItem call, and not during the
setPropertyValue call on the contributing item. So if you use lazy item creation and call
setPropertyValue on a persistent item, you dont need to already have valid values set for any linking
properties on the primary item at that exact point in time. As long as the values of the linking properties
are set before updateItem is called, then the item should be successfully created.
none
If this option is chosen, then no repository items will be created in the underlying repositories under any
circumstance. Any contributing items used in the composite repository must already exist in order for
valid results to be returned from property value requests.
If there is no contributing item found, then the default value for that property in the contributing item
descriptor is returned. If there is no default value, then null is returned
null
267
16 - Composite Repositories
address of the composite repository definition file. You can configure the following properties of this
component:
Property
Description
Value
configurationFile
cumulativeDebug
Boolean. Default is
debugLevel
Integer. Default is 5
queryBatchSize
Integer. Default is
1000.
repositoryName
String
true
268
16 - Composite Repositories
composite-repository-template Tag
The composite-repository-template tag encloses the whole composite repository definition. The
composite-repository-template tag encloses a <header> tag and one or more <itemdescriptor> tags:
header Tag
item-descriptor Tag
Example:
<composite-repository-template>
<header>
...
</header>
<item-descriptor name="..." />
...
</item-descriptor>
</composite-repository-template>
header Tag
The <header> tag provides information that can help you manage the creation and modification of
repository definition files.
Child Tag
How many?
Description
name
zero or one
author
zero or more
269
16 - Composite Repositories
version
zero or one
description
zero or one
For example, the header of your template might look like this:
<header>
<name>Catalog Template</name>
<author>Neal Stephenson</author>
<author>Emily Dickinson</author>
<version>$Id: catalog.xml,v 1.10 2000/12/24 03:34:26 hm Exp $</version>
<description>Template for the store catalog</description>
</header>
item-descriptor Tag
The item-descriptor tag defines a composite item descriptor.
Attribute
Description
Value
name
Required
default
Boolean. Default is
false
display-property
String
270
16 - Composite Repositories
display-nameresource
String
name=resourceBundle>
static (default) or
dynamic
contributing-itemcreation-policy
eager
lazy (default)
none
null-contributingitem-policy
Specifies what to do if a
contributing repository
items is requested, but not
found in the underlying
repository. See Missing
Contributing Items.
default (default)
error
null
Example:
primary-item-descriptor Tag
271
16 - Composite Repositories
contributing-item-descriptor Tag
attribute Tag
primary-item-descriptor Tag
Used in:
item-descriptor Tag
One and only one item descriptor must be designated as the primary item descriptor. The function of the
primary item descriptor is to provide the ID space for the composite item descriptor.
Attribute
Description
Value
name
Required.
String.
repository-nucleus-name
Required.
String.
repository-item-descriptorname
Required.
String.
all-properties-propagate
Optional.
Boolean.
Default is false.
all-properties-queryable
Optional.
Boolean.
Default is true.
272
16 - Composite Repositories
contributing-item-descriptor Tag
The contributing-item-descriptor element specifies
Used in:
item-descriptor Tag
attribute Tag
The attribute tag is used to specify the list of feature descriptor values. The <attribute> tag is an empty
tag that defines the item descriptors feature descriptor value or values. Here is an example:
<property name="employeeNumber" data-type="string">
<attribute name="PCCExpert" value="true" data-type="boolean"/>
</property>
item-descriptor Tag
property Tag
Attribute
Description
Value
name
Required.
value
Required.
273
16 - Composite Repositories
data-type
int
short
byte
long
float
double
string
date
timestamp
Default is
string.
property Tag
The property tag allows you to explicitly map a property in a composite repository to a property in a
primary item descriptor or a contributing item descriptor. This usage handles the case where two or more
contributing item descriptors have properties with the same name. It can have one or more <attribute>
tags.
Used in:
primary-item-descriptor Tag
contributing-item-descriptor Tag
Attribute
Description
Value
name
Required
mapped-propertyname
Required
queryable
required
expert
hidden
274
16 - Composite Repositories
readable
writable
category-resource
String
<attribute
name=resourceBundle> tag, then
the category-resource attribute
String
<attribute
name=resourceBundle> tag, then
the display-name-resource
Example
primary-item-descriptor-link Tag
The primary-item-descriptor-link tag determines which method is used to link items in
contributing item descriptors to items in the primary item descriptor. It has no attributes.
275
16 - Composite Repositories
If the primary-item-descriptor-link tag encloses a link-via-id tag, then the repository ID of the
item is used for linking. If the primary-item-descriptor-link tag encloses a link-via-property
tag, then a unique property of the item, specified in the link-via-property tag, is used for linking.
Used in:
contributing-item-descriptor Tag
Examples
In this example, the contributing item descriptors items are linked to the primary item descriptors items
by the common repository ID of the items:
<primary-item-descriptor-link>
<link-via-id/>
</primary-item-descriptor-link>
In this example, a primary item is linked to an item in this contributing item descriptor if the value of the
primary items firstName property matches the value of the contributing items userFirstName
property AND the value of the primary items lastName property matches the value of the contributing
items userLastName property. This is useful in the case where no one property in the primary item
descriptor or the contributing item descriptor is uniquely valued. The relationships are AND-ed together.
<primary-item-descriptor-link>
<link-via-property primary="firstName" contributing="userFirstName"/>
<link-via-property primary="lastName" contributing="userLastName"/>
</primary-item-descriptor-link>
link-via-id Tag
The link-via-id tag is an empty tag with no attributes. If it is present, it indicates that the repository ID
of the item is used for linking the item in the primary item descriptor to the item in the contributing item
descriptor.
Used in:
primary-item-descriptor-link Tag
link-via-property Tag
The link-via-property tag is an empty tag. If it is present, it indicates that one or more properties are
used to link the item in the primary item descriptor to the item in the contributing item descriptor. See
the second example under primary-item-descriptor-link Tag.
Used in:
primary-item-descriptor-link Tag
276
16 - Composite Repositories
Attribute
Description
Value
primary
Required.
contributing
Required.
sort-property
<?xml encoding="UTF-8"?>
<!-<!-<!-<!--
=============================================================== -->
composite-repository_1.0.dtd - Composite Repository configuration spec -->
Version: $Change: 213147 $$DateTime: 2001/10/01 11:59:44 $$Author: gm $-->
=============================================================== -->
277
16 - Composite Repositories
#REQUIRED
CDATA
#REQUIRED
278
16 - Composite Repositories
all-properties-propagate
all-properties-queryable
%flag;
%flag;
"false"
"true"
>
279
16 - Composite Repositories
280
16 - Composite Repositories
<primary-item-descriptor name="user"
repository-nucleus-name="/atg/userprofiling/ProfileAdapterRepository"
repository-item-descriptor-name="user"
all-properties-propagate="true"
all-properties-queryable="true">
<!-Can also contain explicit property mappings and explicit property exclusions
-->
<property mapped-property-name="lastName" exclude="true"/>
<property mapped-property-name="email" exclude="true"/>
</primary-item-descriptor>
<!-- contributing view definition -->
<!-- name: the name of this contributing view, as it appears to the composite
repository -->
<!-- repository-nucleus-name: the nucleus path of the repository in which the
primary view resides -->
<!-- repository-item-descriptor-name: the name of the view in the given
repository which acts as the primary item descriptor for this composite item
descriptor -->
<!-- all-properties-propagate: if true, composite repository attempts to make
all properties in the primary item descriptor available in the composite item
descriptor. Default is false -->
<!-- all-properties-queryable: if true, all properties in the view are
queryable unless otherwise specified. If false, all properties are not
queryable unless otherwise specified. default is true -->
<contributing-item-descriptor name="UserProfile-LDAP"
repository-nucleus-name="/atg/adapter/ldap/LDAPRepository"
repository-item-descriptor-name="user"
all-properties-propagate="true"
all-properties-queryable="true">
281
16 - Composite Repositories
282
16 - Composite Repositories
d) If there are any two properties in the composite view that have the same
name, error. The composite repository requires that all composite property
names map explicitly to only one primary or contributing property.
-->
<!-- the primary view link describes how items in the contributing view are
linked to items in the primary view. For each primary-contributing
relationship, the user picks a unique id attribute for the primary and the
contributing view. The attribute can be either the repository id of the
item or a uniquely-valued property of the item (e.g. login). A primary item
is linked to a contributing item if its unique id attribute value matches
the unique id attribute value of the contributing item. There must be at
least one primary view link, but there is primary view link limit. -->
<!-- example: this primary view link defines a relationship where an item in
the primary view is linked to an item in this contributing view if the
contributing item has a repository id which is the same as the primary
item's id.
-->
<!-<primary-item-descriptor-link>
<link-via-id/>
</primary-item-descriptor-link>
-->
<!-- OR:
This primary view link defines a relationship where a primary view item is
linked to an item in this contributing view if the value of the primary
item's "login" property matches the value of the contributing item's
"userLoginName" property.
-->
<primary-item-descriptor-link>
<link-via-property primary="login" contributing="login"/>
</primary-item-descriptor-link>
<!-- OR:
This primary view link defines a relationship where a primary view item is
linked to an item in this contributing view if the value of the primary
item's "firstName" property matches the value of the contributing item's
"userFirstName" property AND the value of the primary item's "lastName"
property matches the value of the contributing item's "userLastName"
property. This is useful in the case where no one property in the primary
view or the contributing view is uniquely valued. The relationships are
ANDed together
283
16 - Composite Repositories
<primary-item-descriptor-link>
<link-via-property primary="firstName" contributing="userFirstName"/>
<link-via-property primary="lastName" contributing="userLastName"/>
</primary-item-descriptor-link>
-->
</contributing-item-descriptor>
</item-descriptor>
</composite-repository-template>
284
16 - Composite Repositories
17 Secured Repositories
The ATG secured repository system works in conjunction with the ATG Security System to provide finegrained access control to repository item descriptors, individual repository items, and even individual
properties through the use of Access Control Lists (ACLs).
This chapter includes the following sections:
Secured Repository Features
Creating a Secured Repository
Modifying the Underlying Repository
Configuring the Secured Repository Adapter Component
Writing the Secured Repository Definition File
ACL Syntax
Secured Repository Definition File Tag Reference
Secured Repository Definition File Document Type Definition
Secured Repository Example
285
17 - Secured Repositories
Access Rights
Access control within secured repositories is managed by building ACLs that associate certain access
rights with certain identities (not just individual users, but also groups, organizations, and roles that may
be associated with many users). The following table lists the access rights defined by the secured
repository system. Not all access rights are necessarily available in all implementations or instances of a
secured repository.
Access Right
Available Scopes
Description
CREATE
RepositoryItemDescriptor
DELETE
RepositoryItemDescriptor
286
17 - Secured Repositories
DESTROY
RepositoryItem
LIST
RepositoryItem
READ
RepositoryItemDescriptor
RepositoryItem
Property
READ_ACL
RepositoryItem
READ_OWNER
RepositoryItem
WRITE
RepositoryItemDescriptor
RepositoryItem
Property
WRITE_ACL
RepositoryItem
WRITE_OWNER
RepositoryItem
It is important to remember that the secured repository does not provide complete security within an
application, since the unprotected repository that it sits on top of is still available within the Nucleus
name space. The intent is to provide a repository that aids in creating secure applications, not one that
protects the data from rogue programmers. The ATG Control Center may be configured to hide
unprotected repositories, and an application may choose not to use an unprotected repository, so as not
to expose unprotected data to an end user.
287
17 - Secured Repositories
1.
Modify the underlying repository. For those item descriptors you want to secure, you
need to make some minor modifications to the underlying data and item descriptors
to add properties with which to store the Access Control List and owner information.
This is described in the Modifying the Underlying Repository section of this chapter.
2.
Create and configure the Secured Repository Adapter component. This components
class is atg.adapter.secure.GenericSecuredMutableRepository. See
Configuring the Secured Repository Adapter Component in this chapter.
3.
4.
Create the secured repository definition file. This is an XML file that specifies the access
rights and owner information. This is described in the Writing the Secured Repository
Definition File section in this chapter. Access rights are specified using the syntax
described in the ACL Syntax section.
<item-descriptor name="cheese">
<property name="country" data-type="string" />
<property name="runniness" data-type="int" />
<property name="ACL" data-type="string" />
<property name="cheeseOwner" component-type="user" />
</item-descriptor>
Make sure that the ACL property is of an appropriate data-type. The length of an ACL is limited by the
amount of space available in the property for the ACL. An ACL that is too long will generate a repository
exception when set. This problem is most likely to arise in cases where you use the create-group-acltemplate in the secured repository definition to define an ACL for the groups of the owner and the
owner is a member of many groups.
You can avoid this problem by defining the ACL property as an array of strings, rather than a single string.
The ACL is then stored as a collection of substrings, which are concatenated to form the ACL. For example:
<item-descriptor name="cheese">
...
<table name="test_items_acls"
type="multi"
id-column-names="id"
multi-column-name="index">
<property name="ACL" column-names="acl" data-type="array"
component-data-type="string">
288
17 - Secured Repositories
The maxFragmentSize attribute defines the longest string that will be put in any array index. The default
size is 254. You should set this value to the size of the string column in the database. For many databases,
if you use a VARCHAR of unspecified length, then 254 will be the appropriate value.
These properties in the underlying repository will be identified in the secured repositorys definition file,
using the following tags:
owner-property
acl-property
For example:
<acl-property name="ACL" />
<owner-property name="cheeseOwner" />
Property
Description
$class
Java class
atg.adapter.secure.Generic
SecuredMutableRepository
name
repositoryName
SecuredTestRepository
repository
TestRepository
configurationFile
secured-testrepository.xml
289
17 - Secured Repositories
securityConfiguration
The
atg.security.SecurityConf
iguration component to use.
/atg/dynamo/security/
SecuredRepository
SecurityConfiguration
Attribute
Description
descriptor-acl
The ACL that applies to the item descriptor. This can contain any of the
access rights that apply to the item descriptor. The value of this tag is an
ACL string, as defined in the ACL Syntax section.
default-acl
owner-property
This defines the name of the string property in the underlying repository
that is to be used to store the name of the owner of a repository item.
acl-property
This defines the name of the string property in the underlying repository
that is used to store the ACL for an individual repository item.
290
17 - Secured Repositories
creation-base-acl
An ACL fragment that is inserted into the default ACL for a newly created
repository item. Typically this defines global access rights for
administrators and limited access rights for the user base as a whole.
This ACL fragment can contain any of the access rights that apply to a
repository item.
creation-owner-acltemplate
creation-group-acltemplate
You can use a subset of these options to define ACLs for properties as well as item descriptors:
descriptor-acl
default-acl
acl-property
creation-base-acl
creation-owner-acl-template
creation-group-acl-template
Group Membership
An identity is considered to be a group that the owner (creator) is a member of if the owners Persona lists
it with its getSubPersonae() call. Exactly what is returned by this call will vary according to the
implementation of the User Authority.
The standard User Authority used here is implemented on top of the User Directory interface, and
includes every Effective Principal of the user as a sub-Persona. For the Profile User Directory, this includes
all of the Organizations, Roles, and Relative Roles of the user as well as all of the Organizations, Roles and
Relative Roles of any Organization they are members of (either explicitly or implicitly). For the Admin User
Directory, this includes all of the Groups that the ACC account is a member of, but not the Privileges that
the Group has been assigned.
291
17 - Secured Repositories
When creating ACLs, it is imperative that the Personae that are used for user identities be created by the
exact same User Authority that is being used by the secured repository. The User Authority may not be a
proxy even though the Personae produced by a proxy test is equivalent to the Personae produced by the
User Authority that it is a proxy for. This is because the identity name spaces used by a User Authority and
its proxies may not be the same, and the ACL parser cannot support multiple identity namespaces.
ACL Syntax
ACL strings in ATG are made up of a series of Access Control Entries (ACEs) separated from each other by
semicolons:
ACL ::= ACE [ ';' ACE ]+
Each ACE is made up of two parts, an identity and a list of access rights, separated by a colon, and
optionally surrounded by an ACE type specifier for determining whether the ACE is to grant or deny
rights:
ACE ::= (
( IDENTITY ':' ACCESS_RIGHTS_LIST ) |
( ( "grant" | "deny" ) '{' IDENTITY ':' ACCESS_RIGHTS_LIST '}' )
)
The "grant" modifier is the default, and may be omitted. If a "deny" ACE exists where a "grant" ACE
also applies, the standard security policy will deny access.
An identity is the literal string used by the User Authority to look up the identitys Persona. The standard
User Authority (/atg/dynamo/security/UserAuthority in Nucleus) encodes the identity as follows:
UD_IDENTITY ::= UD_NAME '$' PRINCIPAL_TYPE '$' UD_PRINCIPAL_KEY
where UD_NAME is the name of the User Directory as configured in the User Directory User Authority
(usually either Admin for the ACC account database or Profile for the Profile Repository),
PRINCIPAL_TYPE is one of user, org or role, and UD_PRINCIPAL_KEY is the primary key for looking up
the principal in the User Directory. The primary key varies between User Directory implementations. The
primary key is a numeric ID for Profile User Directories, but is the account name (for example, admin,
administrators-group) for the ACC account User Directory.
ATG comes configured with three other User Authorities:
and roles.
292
17 - Secured Repositories
These user authorities look up Persona information based on the unencoded name of the identity and are
typically used for performing authentication. They are, however, front-ends for the standard User
Authority and produce Personae that are equivalent to those produced by the standard User Authority.
(Note the caveat regarding the mixing of User Authorities in the Writing the Secured Repository Definition
File: ACLs and Personae topic.)
The list of access rights is a comma-separated list of access right names:
ACCESS_RIGHT_LIST ::= ACCESS_RIGHT [ ',' ACCESS_RIGHT ]+
Only the access rights appropriate for the ACL context are allowed. Access right names are tokens and are
not internationalizable.
ACL Examples
The following examples are coded using the syntax used by the standard
/atg/dynamo/security/UserAuthority component.
The following ACL grants everyone with an ACC account the ability to read:
Admin$role$everyone-group:read;
Note that you should always end ACL strings with a semi-colon, as shown, even when the string is the last
one in a list. Do not start ACLs with a semi-colon. Following this convention is important because it
ensures that ACLs are interpreted correctly after XML-combine operations.
The following ACL grants the ACC admin account the ability to read and write, but every other ACC user
only the ability to read:
293
17 - Secured Repositories
Admin$user$admin:list,read,write;
Admin$role$everyone-group:list,read;
The following ACL grants the ACC Administrators group the ability to read, write and delete, but denies
the ability to write and delete to ACC user Fnord even if he is a member of the Administrators group:
Admin$role$administrators-group:
list,read,write,delete;deny{Admin$user$Fnord:write,delete};
secured-repository-template Tag
The secured-repository-template tag encloses the whole secured repository definition. The
secured-repository-template tag encloses one item-descriptor tag for each item descriptor in
the underlying repository for which you want to specify access rights:
item-descriptor Tag
Example:
<secured-repository-template>
<item-descriptor name="..." />
...
</secured-repository-template>
item-descriptor Tag
You should include one item-descriptor tag for each item descriptor in the underlying repository for
which you want to specify access rights. Unlike the item-descriptor tag in the SQL repository, the
item-descriptor tag in the secured repository has just one attribute, name, which must be the same as
the name attribute in the underlying repositorys item-descriptor tag.
Example:
<item-descriptor name="feature">
<descriptor-acl value="..."/>
<owner-property name="..."/>
<acl-property name="..."/>
...
</item-descriptor>
294
17 - Secured Repositories
property Tag
default-acl Tag
descriptor-acl Tag
owner-property Tag
acl-property Tag
creation-base-acl Tag
creation-owner-acl-template Tag
creation-group-acl-template Tag
property Tag
To apply access rights at the property level, rather than to the whole item, you can use the property tag.
A property tag can enclose any or all of the following child tags:
default-acl Tag
descriptor-acl Tag
creation-base-acl Tag
creation-owner-acl-template Tag
creation-group-acl-template Tag
Example
<property name="yumminess">
<descriptor-acl value="..."/>
<acl-property name="yummy_acl"/>
<creation-base-acl value="...
...
</property>
default-acl Tag
The default-acl element specifies the ACL that is applied to either an item or property descriptor when
it has no other ACL. This ACL can contain any of the access rights that apply to the item descriptor or
property. The value attribute of the tag is an ACL string, using the syntax described in the ACL Syntax
section.
Used in:
item-descriptor Tag
property Tag
295
17 - Secured Repositories
Example:
<default-acl value="Admin$role$everyone-group:list,read;"/>
descriptor-acl Tag
The descriptor-acl element specifies the ACL that applies to the item or property specified by the
enclosing item-descriptor tag or property tag. This ACL can contain any of the access rights that
apply to the item descriptor or property. The value attribute of the tag is an ACL string, using the syntax
described in the ACL Syntax section.
Used in:
item-descriptor Tag
property Tag
Example:
<descriptor-acl value="Admin$role$administrators-group:list,read,write;
Admin$role$everyone-group:list,read;"/>
owner-property Tag
The owner-property tag has one attribute, name, which specifies the name attribute of the property that
stores the owner of the item in the underlying repository.
Used in:
item-descriptor Tag
Example
If the item descriptor in the underlying repository stores the name of the items owner in a property
named item_owner, the owner-property tag would look like this:
<owner-property name="item_owner"/>
acl-property Tag
The acl-property tag has one attribute, name, which specifies the name attribute of the property that
stores the ACL of the item in the underlying repository.
Used in:
item-descriptor Tag
property Tag
Example:
296
17 - Secured Repositories
If the item descriptor in the underlying repository stores the items ACL in a property named item_acl,
the acl-property tag would look like this:
<acl-property name="item_acl"/>
creation-base-acl Tag
The creation-base-acl tag defines an ACL fragment that is inserted into the default ACL for a newly
created repository item or property. Typically this defines global access rights for administrators and
limited access rights for the user base as a whole. This ACL fragment can contain any of the access rights
that apply to a repository item or property.
Used in:
item-descriptor Tag
property Tag
Example
The following example gives all access rights to the administrators group, but only read and list rights
to everyone else:
<creation-base-acl value="Admin$role$administrators-group:
read,write,list,destroy,read_owner,write_owner,read_acl,write_acl;
Admin$role$everyone-group:read,list;"/>
creation-owner-acl-template Tag
The creation-owner-acl-template tag specifies an ACL template that is used to generate an ACL
fragment that applies to the owner (creator) of a newly created repository item. This is a standard format
ACL string with a dollar sign ($) used to indicate the owner identity. No other identities may be used in
the template.
Used in:
item-descriptor Tag
property Tag
Example
The following example gives the owners of an item access rights to read, write, list, or destroy items they
own:
<creation-owner-acl-template value="$:read,write,list,destroy;"/>
creation-group-acl-template Tag
The creation-group-acl-template tag specifies an ACL template that is used to generate an ACL
fragment that applies to each group that the owner is a member of in a newly created repository item.
297
17 - Secured Repositories
This is a standard format ACL string with a dollar sign ($) used to indicate the group identity. No other
identities may be used in the template.
Because a user may be a member of a large number of groups, you should use this feature sparingly. It
can result in ACL strings that are too long to be stored in your repositorys ACL property. For example, the
ACC admin user may have enough groups to create an ACL that is too large for our example repository.
For a description of what constitutes membership in a group, see Group Membership.
Used in:
item-descriptor Tag
property Tag
Example
The following example gives read and list access rights to every member of every group of which the
items owner is a member:
<creation-group-acl-template value="$:read,list;"/>
<?xml encoding="UTF-8"?>
<!-<!-<!-<!--
=============================================================== -->
secured_repository_template.dtd - Definition spec for secured
-->
repositories
-->
Version: $Change: 166603 $$DateTime: 2001/04/20 20:05:51 $$Author:
bbarber $ -->
<!-- =============================================================== -->
<!-- =============================================================== -->
<!-- secured-repository-template - top level element
-->
<!-- =============================================================== -->
<!ELEMENT secured-repository-template (item-descriptor)*>
<!-- =============================================================== -->
<!-- account specifications - define an account of a particular type -->
<!-- =============================================================== -->
298
17 - Secured Repositories
#REQUIRED
#REQUIRED
299
17 - Secured Repositories
name
CDATA
#REQUIRED
>
<!-- =============================================================== -->
<!-- acl-property - specifies the name of the property in which
-->
<!-the ACL for the item or property is stored
-->
<!-- =============================================================== -->
<!ELEMENT acl-property (#PCDATA)>
<!ATTLIST acl-property
name
CDATA
>
#REQUIRED
#REQUIRED
300
17 - Secured Repositories
-- test-repository.ddl
create table test_items (
-- the ID of this item
id varchar,
-- a secured property of this item
secured_property varchar,
-- an unsecured property
unsecured_property varchar,
)
# test-repository.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE gsa-template
PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN"
"http://www.atg.com/dtds/gsa/gsa_1.0.dtd">
<gsa-template>
<header>
<name>Test Repository</name>
</header>
<item-descriptor name="test_items" default="true">
<table name="test_items" type="primary" id-column-names="id">
<property name="secured_property" column-names="secured_property"
data-type="string"/>
<property name="unsecured_property" column-names="unsecured_property"
data-type="string"/>
</table>
301
17 - Secured Repositories
</item-descriptor>
</gsa-template>
We need to add fields to the SQL and the repository definition to provide storage space for security
information, one each for storing the owner, repository item ACL, and repository item property ACL. The
following files show these changes. The SQL now looks like this:
--
Modified test-repository.ddl
<!--
302
17 - Secured Repositories
</header>
<item-descriptor name="test_items" default="true">
<table name="test_items" type="primary" id-column-names="id">
<property name="secured_property" column-names="secured_property"
data-type="string"/>
<property name="unsecured_property" column-names="unsecured_property"
data-type="string"/>
<property name="item_owner" column-names="item_owner" data-type="string"/>
<property name="item_acl" column-names="item_acl" data-type="string"/>
<property name="secured_property_acl" column-names="secured_property_acl"
data-type="string"/>
</table>
</item-descriptor>
</gsa-template>
The properties file for the GSARepository component can remain as it was.
The next step is to create the secured repository layer over this SQL repository. The secured repository has
an XML definition file, which would look like this:
303
17 - Secured Repositories
<descriptor-acl value="Admin$role$administrators-group:
read,write;Admin$role$everyone-group:read"/>
<!-- The name of the property in the item where the ACL for this
property is stored. -->
<acl-property name="secured_property_acl"/>
<!-- An ACL fragment that is assigned to this property
whenever a new item is created. -->
<creation-base-acl value="Admin$role$administrators-group:read,write"/>
<!-- Access rights that are assigned to the owner when an
item is created -->
<creation-owner-acl-template value="$:read,write"/>
<!-- Access rights that are assigned to all of the owner's
groups when an item is created. WARNING: This feature
is potentially dangerous. -->
<creation-group-acl-template value="$:read,write"/>
</property>
</item-descriptor>
</secured-repository-template>
# SecuredTestRepository.properties
$class=atg.adapter.secure.GenericSecuredMutableRepository
$scope=global
name=Test repository for the secured repository implementation
repositoryName=SecuredTestRepository
# the repository that we're wrapping
repository=TestRepository
# The template file that configures the repository
configurationFile=secured-test-repository.xml
# The security configuration component used by the repository
securityConfiguration=/atg/dynamo/security/SecuredRepositorySecurityConfiguration
# Various Dynamo services we need
XMLToolsFactory=/atg/dynamo/service/xml/XMLToolsFactory
transactionManager=/atg/dynamo/transaction/TransactionManager
WARNING: In the above example we make use of the creation-group-acl-template feature for both
repository items and the secured property. This setting should generally be removed if you are setting up
a repository based on this code. The reason for this is explained at creation-group-acl-template Tag.
Finally, in order to expose these repositories to the ATG Control Center Repository Editor, and to start
them up when your application is started, you must add each of them to the initialRepositories
property of the /atg/registry/ContentRepositories component:
initialRepositories+=/TestRepository,/SecuredTestRepository
304
17 - Secured Repositories
Performance Considerations
While care has been taken to maintain high performance, use of the secured repository does have some
impact on the performance of the repository and, in some cases, the impact is considerable.
For access control defined at the item descriptor level (e.g. Repository.getItem(),
MutableRepository.createItem(), MutableRepository.addItem(),
MutableRepository.updateItem()) the overhead of handling access checks amounts to the testing of
the access control list for the item descriptor. This is normally minimal.
The exception to this rule is with the use of the RepositoryView.executeQuery() and
RepositoryView.executeCountQuery() family of methods whenever ACLs are specified for individual
repository items. In this case, the ACL of each repository item must be consulted to determine if it should
be allowed in the result of the query, or counted as part of a count query. If the result set is large, the time
required to parse and check all of the ACLs may be large. Furthermore, in the count query case, a full
query must be done and its results counted. Thus, if your application is using count queries to limit
expensive queries, the features afforded by a secured repository will be very expensive.
Access control overhead at the repository item level is noticeable, but is incremental. When the repository
item is loaded its ACL will be parsed prior to any access checking. Results of ACL parsing are cached to
improve performance where possible. If ACLs are not being stored for each individual repository item, no
parsing needs to be done beyond what is done during the initialization of the secured repository.
Because the secured repository sits on top of an underlying repository, you can consider whether features
that need best possible performance should be written to use the underlying repository rather than going
through the secured repository at the cost of the security features.
nested exception.
The RepositoryItem.getPropertyValue() and MutableRepositoryItem.setPropertyValue()
methods are unusual in that they do not throw a RepositoryException. It is necessary, however, for
them to throw a SecurityException, so they will throw an
atg.security.RuntimeSecurityException. The
RuntimeSecurityException.getSecurityException() method can be used to determine the
nested exception.
305
17 - Secured Repositories
306
17 - Secured Repositories
18 LDAP Repositories
The ATG LDAP Repository is an implementation of the Repository API that enables you to store and access
profile data in an LDAP (Lightweight Directory Access Protocol) directory. The LDAP repository is similar in
functionality to the SQL repository, as described earlier in this guide. While by default ATG Scenario
Personalization is configured to use a SQL profile repository, you can change the configuration to use an
LDAP repository instead. See the ATG Personalization Programming Guide for information about
configuring ATG to use an LDAP profile repository. LDAP directories are widely used to store personnel
information and other kinds of data. Using an LDAP repository enables you to tap into the profile data you
already have in an LDAP directory, and to share user information across multiple applications.
In addition, you can configure ATGs application security scheme to use an LDAP repository, rather than a
SQL repository. See the Managing Access Control chapter in the ATG Programming Guide for more
information.
Just like the SQL repository, the LDAP repository implements the ATG Repository API to allow you to store,
access, modify, and query user profile information. As in the SQL repository, repository items are first
created as transient items (RAM profiles); they become persistent after they are added to the database.
It is important to note, however, that the LDAP repository implementation is not specific to user profiles in
any way. Since an LDAP directory can be used to store any kind of data (people, groups, mailing lists,
documents, printers, etc.), you could use the LDAP repository to expose any of that data in ATG. In this
chapter, we focus on using LDAP as a profile repository, because that is the most common application of
LDAP. Keep in mind, however, that other uses are possible.
This chapter includes the following sections:
Overview: Setting Up an LDAP Repository
An overview of the steps you should take in designing and setting up an LDAP
repository.
LDAP Directory Primer
A brief introduction to LDAP concepts and terminology
LDAP Repository Architecture
A description of the way item descriptors and repository items work in the LDAP
repository.
Repository Views in the LDAP Repository
How multiple Repository Views can support multiple item descriptors in the same
repository.
LDAP Repository Queries
A brief look at how queries work in the LDAP repository.
307
18 - LDAP Repositories
Create the LDAP schema on your LDAP directory server. The methods for creating and
modifying the LDAP schemas differ from server to server. Consult the documentation
for your LDAP directory server.
If your LDAP directory already exists, and you want to be able to perform ID matching
queries on the LDAP repository, make sure that your LDAP entries include a property
that corresponds to the ID property of the repository items. See LDAP Repository
Queries.
2.
Create the XML LDAP repository definition file for the LDAP repository to use. This XML
template defines the item descriptors and repository item properties contained in
your LDAP repository. It also describes the relationship of your LDAP directory entries
to the item descriptors and repository items of the LDAP repository. See the LDAP
Repository Architecture section, and especially the Item Descriptors and LDAP Object
Classes subsection therein, for information about designing your LDAP repository
components. See also the LDAP Repository Definition Tag Reference for full details of
the XML tags used to create the LDAP repository definition file. Note that while the
LDAP repository definition is similar in many ways to the SQL repository definition, the
LDAP repository definition uses its own XML document type definition and syntax.
3.
Configure the ATG LDAP repository components. See Configuring the LDAP
Repository Components.
4.
Configure ATG so that ATGs user profiling components point to the LDAP repository,
rather than to a SQL profile repository. See the Setting Up an LDAP Profile Repository
chapter in the ATG Personalization Programming Guide.
308
18 - LDAP Repositories
Directory Schema
LDAP Sources
The directory tree may be highly branched, with the entire organizational hierarchy reflected in the tree
structure, or it may be almost flat, depending on the needs of the organization. An example of an almost
flat directory structure is one where all the people entries reside under the same organizational unit entry,
such as ou=person,o=quincyfunds.com. There may also be organizational unit entries for storing other
types of information, for example, ou=Groups,o=quincyfunds.com,
ou=Customers,o=quincyfunds.com, ou=Devices,o=quincyfunds.com, and so on.
A directory may have more than one directory suffix. This typically comes into play with very large
directories which are spread across multiple machines, extranets, ISPs, etc. For example, an ISP whose
directory service needs to support multiple enterprises might have a separate directory suffix for each of
the enterprises.
309
18 - LDAP Repositories
All data associated with an LDAP entry is contained in the entrys attributes. For example, the entry whose
distinguished name is uid=nat,ou=person,o=quincyfunds.com might have the following attributes:
objectClass: top
objectClass: person
objectClass: organizationalPerson
uid: nat
cn: Natalya Cohen
cn: Nat Cohen
sn: Cohen
givenName: Natalya
givenName: Nat
Many of the attributes in an LDAP directory can be multi-valued (such as the cn, givenName, and
objectClass attributes in the example above).
One interesting point to note is that the attribute values comprising the entrys distinguished name do
not necessarily have to correspond to the attribute values contained in the entry itself. For example, the
entry above does not contain an ou attribute or an o attribute, even though the DN implies an ou value of
person and an o value of quincyfunds.com. Even more confusing situations are possible (although, of
course, not recommended by the directory providers), where the attribute is specified both in the DN and
in the entry itself, but the two values differ.
For these kinds of cases, the thing to keep in mind is that the actual directory data is contained in the
entrys attributes. The distinguished name is simply a name that can be used to uniquely identify the
entry; it does not represent the actual attribute values. For example, when the directory is searched, it is
not searched against the DN, but against the attribute values stored in the entries themselves.
Note however that you do use the DN to access a directory entry directly, without searching. Also, you
must specify the DN when you create a new entry.
310
18 - LDAP Repositories
Notice that the example entry in the LDAP Data Representation section above has three values for its
objectClass attribute: top, person, and organizationalPerson. The first two values seem
unnecessary, since they are both ancestors of the organizationalPerson type. However, they are
required because not all directory servers support type inheritance.
The objectClass values in an entry dont all have to be each others ancestors, however. For example,
one can create an entry that is both an organizationalPerson and a mailGroupMember, which itself
inherits from top. In other words, multiple inheritance of types is allowed.
Directory Schema
The total set of object classes and attributes known to the LDAP directory is referred to as the directory
schema. Each LDAP directory server comes with a standard schema that includes predefined object
classes and attributes. In addition, you can extend this standard schema to represent information unique
to your enterprise.
For each object class, the schema contains information such as the names of the superior object classes
from which this object class is derived, and the names of the required and optional attributes of the
object class. For each of the attributes, the schema contains information about its syntax and whether the
attribute is single- or multi-valued.
All LDAP directory implementations are expected to support the minimal default schema specified in RFC
2256. The tables below summarize those object classes and attributes in the default schema used by
ATGs LDAP repository. For the full list of object classes and attributes, please refer to the RFC.
Name
Parent
top
Required Attributes
Optional Attributes
objectClass
person
top
sn, cn
userPassword,
telephoneNumber
organizationalPerson
person
title, employeeNumber,
telephoneNumber,
facsimileTelephoneNumber
inetorgPerson
organizationalPerson
mail, uid
311
18 - LDAP Repositories
Name
Description
Single Value?
objectClass
false
cn
false
sn
false
name of an organization
false
ou
false
givenName
false
userPassword
false
title
false
telephoneNumber
telephone number
false
facsimileTelephoneNumber
fax number
false
uid
unique id
false
e-mail address
false
employeeNumber
employee number
false
Notice that all of the attributes listed above are multi-valued. There are actually very few single-valued
attributes in LDAP, for maximum flexibility.
312
18 - LDAP Repositories
LDAP Sources
For more information about LDAP, consult the following:
RFC 2251: Lightweight Directory Access Protocol (v3) - Specification of the LDAP
version 3 protocol.
RFC 2252: Lightweight Directory Access Protocol (v3): Attribute Syntax Definitions More of same, with more detail on attributes.
RFC 2254: The String Representation of LDAP Search Filters - Describes the LDAP
search filters, which are used by JNDI to perform directory searches. The LDAP
repositorys QueryBuilder implementation constructs Query objects, which are
essentially LDAP search filters.
RFC 2256: A Summary of the X.500(96) User Schema for use with LDAPv3 - Default
LDAP v3 schema that all LDAP directory servers are expected to support.
Your LDAP server-specific documentation. For example, if you have access to a Sun
ONE Directory Server installation, browse through the Bookshelf. The Deployment
Guide, in particular, is a good source of information.
313
18 - LDAP Repositories
This section introduces the principal features of the LDAP repository definition file. A complete reference
to the repository definition file syntax is found in the LDAP Repository Definition Tag Reference section of
this chapter.
314
18 - LDAP Repositories
<item-descriptor name="user">
<!-- special properties -->
...
<!-- object classes -->
<object-class>top</object-class>
<object-class>person</object-class>
<object-class>organizationalPerson</object-class>
<object-class>inetorgPerson</object-class>
<!-- properties -->
<property name="login" ldap-name="uid" data-type="string" required="true"/>
<property name="password" ldap-name="userpassword" data-type="string"
required="true"/>
<property name="lastName" ldap-name="sn" data-type="string" required="true"/>
<property name="firstName" ldap-name="givenName" data-type="string"/>
<property name="names" ldap-name="cn" data-type="string" multi="true"
required="true"/>
<property name="email" ldap-name="mail" data-type="string"/>
<property name="phone" ldap-name="telephonenumber" data-type="string"/>
<property name="fax" ldap-name="facsimiletelephonenumber" data-type="string"
default="(617) 555-1211"/>
<property name="department" ldap-name="ou" data-type="string"/>
<property name="title" data-type="string"/>
<property name="employeeNumber" data-type="long"/>
<!-- new item creation -->
...
315
18 - LDAP Repositories
</item-descriptor>
The object-class tags specify all the object class values corresponding to the given item descriptor. If
the object class has ancestor object classes, they must all be specified, as demonstrated above. The object
class information is required so that when a new item is created for the given item descriptor and added
to the repository, the corresponding LDAP entry is created with the given object class values. Thus, for
example, if an item is created in the context of the user item descriptor, the new LDAP directory entry has
objectclass attribute values of top, person, organizationalPerson, and inetorgPerson.
The LDAP repository definition uses <property> tags to map Profile properties to LDAP attributes. Each
such <property>tag describes a property descriptor of its item descriptor. The <property>tags in the
example above demonstrate that:
Repository item property names can be different from LDAP attribute names. For
example, the lastName property in the item descriptor maps to the sn attribute in the
LDAP directory schema. If the ldap-name tag attribute is not specified, the repository
item property name and the LDAP attribute name will be the same.
Repository item property types can be different from JNDI service provider types. For
example, userpassword is exposed as a binary type by Suns LDAP service provider,
but is a String property in the repository; employeeNumber is a String in Suns LDAP
service provider, but a Long in the repository.
Repository item properties can have default values. For example, the fax property has
a default value of (617) 555-1211.
In addition, the user item descriptor exposes only those attributes that we care about, and promotes
some of the optional attributes into required ones. For example, the password attribute is optional in
LDAP, but required in the Profile repository item.
Note that although the attributes such as givenName and sn are multi-valued in the LDAP directory, they
are exposed as single-valued properties in the repository. When getting the values for these properties,
the LDAP repository will ignore all but one of the returned values. It is not specified which of the values
will be returned. On the other hand, the LDAP repository items names property is multi-valued, and
corresponds to the LDAP directorys multi-valued cn attribute; in this case, the attribute value is returned
as a String array.
For all of this to work, the repository item descriptor must not violate the LDAP directory schema. For
example, since cn is a required attribute for the inetorgPerson class, one of the properties specified in
the item descriptor must map to the cn attribute, and it must be required. As another example, the item
descriptor cannot contain a property that does not correspond to an LDAP attribute. That is, the ldapname tag attribute value must be a legal LDAP attribute name. The LDAP repository does no checking to
ensure that the item descriptor conforms to the LDAP schema. If the schema is violated, a runtime
exception (an object schema violation) is thrown by JNDI.
316
18 - LDAP Repositories
etc. Each item descriptor would simply map to a particular set of object classes specified in the LDAP
schema.
The LDAP model also nicely supports hierarchies of item descriptors that map to object class subtypes.
For example, suppose the inetorgPerson object class has several subclasses, such as
engineeringPerson, salesPerson, etc. The engineeringPerson class contains all the same attributes
as the inetorgPerson class, and adds a few, such as engineerType and currentProject. In the LDAP
repository, we can define an item descriptor called engineer that inherits from the user item descriptor
but supports these additional attributes. The following example shows how a portion of an LDAP
repository definition might describe such an engineer item descriptor:
The optional parent property of an <item-descriptor> specifies that the item descriptor inherits all of
the parents object classes and properties. Any additional object-class and property values are added
to the list of the parents object classes and properties.
You can also specify <child-property> tags to override any parent properties that have the same
name. The only aspect of the parent property definition that can be overridden is the propertys default
value. The propertys data-type and other attributes must stay the same. The example above
demonstrates how the <child-property> tag can be used to assign the default value of Engineering
to the parents department property; the salespeople item descriptor might assign the default value of
Sales to the same property.
317
18 - LDAP Repositories
In addition to the properties you specify, the LDAP repository creates two special properties for every item
descriptor: the id attribute and the objectClasses attribute. Here is the relevant XML from the user
item descriptor definition that we examined above:
<item-descriptor name="user">
<!-- special properties -->
<id-property name="id" in-ldap="true" ldap-name="dpsid"/>
<object-classes-property name="objectClasses" ldap-name="objectclass"/>
<!-- object classes -->
<object-class>top</object-class>
<object-class>person</object-class>
<object-class>organizationalPerson</object-class>
<object-class>inetorgPerson</object-class>
<object-class>dpsUser</object-class>
<!-- properties -->
...
<!-- new item creation -->
...
</item-descriptor>
The purpose of the <id-property> tag is to expose the repository ID of a repository item as an attribute
(of type String). Thus, assuming the definition above, an item with repository ID
uid=nat,ou=Marketing,o=quincyfunds.com would have an LDAP attribute named dpsid with the
same value. The attribute value does not need to be set by the user; it is set automatically by ATG. Note
that the ID property is populated from the DN; you should not attempt to create the DN from the ID
property.
The rest of the id-property definition above specifies whether the id property of the repository item
maps to an actual LDAP attribute, and if so, the LDAP attributes name. If the value of in-ldap is false
(the default), the id attribute exists only as a property of the repository item, and does not exist as an
attribute in the LDAP entry. In that case, when the items attribute values are written out to the LDAP
directory, the ID attribute value is ignored, since there is no equivalent for it in the directory entry. If the
value of in-ldap is true, as above, the ldap-name tag attribute specifies the name of the LDAP attribute
to which the id should be written. As usual, if ldap-name is not specified, it is assumed to be the same as
name. Thus, with our example item descriptor, when an item with ID
uid=nat,ou=Marketing,o=quincyfunds.com is created and added to the repository, the resulting
LDAP entry has an attribute named dpsid with value uid=nat,ou=Marketing,o=quincyfunds.com.
Saving the ID attribute value in the LDAP entry makes it easier to perform ID matching repository queries,
as discussed in the LDAP Repository Queries section in this chapter.
The <object-classes-property> tag is similar to <id-property>: it exposes the items object class
values as an attribute. The attributes type is String[], which allows for a multi-valued attribute. For
318
18 - LDAP Repositories
example, an item with a user item descriptor will have an objectClasses attribute, the value of which
will be an array the elements of which are top, person, organizationalPerson, inetorgPerson, and
dpsUser. The dpsUser object class supports the dpsid attribute, which allows us to incorporate the
repository ID as an attribute in the LDAP entry.
The <id-property> and <object-classes-property> tags are both required in a definition of a base
item descriptor (that is, an item descriptor that doesnt have a parent); however, they are not allowed in
child descriptor definition. The child item descriptors inherit the id and objectClasses properties from
their parent.
See the LDAP Repository Definition Tag Reference in this chapter for full details.
For properties with data-type of enumerated, use <option> tags to specify the propertys value choices.
We saw an example of this in the engineerType attribute of the engineer item descriptor:
This approach is again inherited from the SQL repository definition file.
Just like the SQL repository, an LDAP repositorys <property> tags can have zero or more <attribute>
child tags. These child tags allow you to associate arbitrary name/string value pairs with any attribute. The
319
18 - LDAP Repositories
You might use a descriptor like unique to specify that a property value can be assigned to only one
repository item within the item type. This LDAP repository feature is similar to the feature described in the
User-Defined Property Types section of the SQL Repository Item Properties chapter.
You can also specify a property editor class to use with a property, using the editor-class attribute. For
example, the following tag associates a special property editor with the password property:
<property name="password" ldap-name="userpassword"
data-type="string" required="true"
editor-class="atg.beans.PasswordPropertyEditor"/>
<item-descriptor name="user">
<!-- special properties -->
...
<!-- object classes -->
...
<!-- properties -->
...
<!-- new item creation -->
<new-items allowed="false">
</item-descriptor>
<item-descriptor name="engineer" parent="user">
<!-- object classes (added to parent classes) -->
...
<!-- properties (added to parent properties) -->
...
<!-- child properties (override parent properties) -->
...
320
18 - LDAP Repositories
The <new-items> tag in the user descriptor indicates that this descriptor does not allow new items to be
created. The user descriptor basically acts as an abstract class it provides a base set of object classes
and properties for children descriptors to build on, but it does not allow items with those object classes
and properties to be instantiated.
The engineer descriptor, on the other hand, does allow new items to be created. The new-items tag
specifies where the newly created items should be placed in the LDAP directory. The new items DN is
constructed by appending the value of the parent-dn attribute to the RDN. The RDN is created using the
value of the LDAP attribute that corresponds to the repository item property specified by the rdnproperty XML attribute. For example, if a new item is created whose login property is nat, the
corresponding RDN is uid=nat (since the Profiles login property maps to the uid attribute in the LDAP
directory), and the DN is uid=nat,ou=Engineering,o=quincyfunds.com.
If a child descriptor definition does not contain a <new-items> tag, it inherits the parents item creation
behavior.
321
18 - LDAP Repositories
To summarize, the contents of each Repository View are determined by two factors: the object classes of
its item descriptor, and its search roots. When a query is performed on the view, only those items that
reside in one of the specified search roots and satisfy the views item descriptor are returned. At least one
search root must always be specified, but it may well point to the directory suffix (i.e., the search root may
span the entire directory tree).
322
18 - LDAP Repositories
If this query were applied to the ou=Engineering,o=quincyfunds.com search root of the engineer
Repository View, the query would be turned into
(&(currentProject=Quincy)
(objectclass=top)
(objectclass=person)
(objectclass=organizationalPerson)
(objectclass=inetorgPerson)
(objectclass=dpsUser)
(objectclass=engineeringPerson))
If the value of check-classes is false for the search root, however, the query is left as is, and no object
class checking is performed. Obviously, this optimization should only be turned on if you are absolutely
sure that the search root contains only entries that satisfy the item descriptor.
ID Matching Queries
The idMatching query is a special case. The LDAP search filter can only search for entries based on their
attribute values. However, the LDAP repository uses the entrys DN, rather than any attribute value, as its
ID. Thus, ID matching queries cant be constructed using search filters, unless the LDAP entrys DN is also
an LDAP attribute.
323
18 - LDAP Repositories
To implement ID matching queries, add an ID attribute to the LDAP entries, as we saw in the Id and
ObjectClasses Properties section previously in this chapter. In our example, all user LDAP entries have an
attribute called dpsid, which is mapped to the repository items id attribute. The value of dpsid is
automatically set to the DN when the item is first added to the repository. Since the ID can now be
accessed as an attribute of an LDAP entry, full support for the ID matching query is provided in this case.
Note, however, that directory entries that were not created by the repository must be manually modified
to include a dpsid attribute, or they will not be returned by the queries on the view.
If no ID attribute exists in the LDAP entries, the ID matching query is only supported as the top level
query. That is, you can have a targeting rule that matches only items with specified IDs, but you cant have
a rule that matches items with specified IDs and satisfies some other criteria. The top level query is
implemented by simply calling Repository.getItems with the specified IDs. No checking is done to
verify that the resulting items are actually contained by the view. ATG doesnt check that they have the
correct object classes, and are located inside one of the search roots.
/atg/adapter/ldap/LDAPRepository
/atg/adapter/ldap/InitialContextPool
/atg/adapter/ldap/InitialContextEnvironment
324
18 - LDAP Repositories
/atg/adapter/ldap/LDAPItemCache
/atg/adapter/ldap/LDAPItemCacheAdapter
/atg/adapter/ldap/LDAPQueryCache
/atg/adapter/ldap/LDAPQueryCacheAdapter
These LDAP repository components can be configured using the properties described below.
/atg/adapter/ldap/LDAPRepository
This component is the repository of LDAP profiles.
Property
Description
Default Value
$class
class name
atg.adapter.ldap.
LDAPRepository
cacheItemProperties
true
cacheItems
true
cacheQueries
false
definitionFile
/atg/adapter/ldap/
ldapUserProfile.xml
idGenerator
/atg/dynamo/service/
IdGenerator
325
18 - LDAP Repositories
initialContextPool
/atg/adapter/ldap/
InitialContextPool
objects.
itemCache
/atg/adapter/ldap/
LDAPItemCache
prefetchItemProperties
true
queryCache
/atg/adapter/ldap/
repositoryName
Name of repository.
LDAP
shutdownDelay
30
shutdownDynamoOnFatal
true
transactionManager
/atg/dynamo/transaction/
/atg/dynamo/service/xml/
XMLToolsFactory
XMLToolsFactory
LDAPQueryCache
TransactionManager
/atg/adapter/ldap/InitialContextPool
This component is a JNDIInitialContextPool used to pool connections to the LDAP server. This
components class extends atg.service.resourcepool.ResourcePool. See the Core Dynamo
Services: Resource Pools section of the ATG Programming Guide and the Javadoc for the ResourcePool
class in the ATG API Reference for more information about the many properties available for configuring a
connection pool. Getting connections from a resource pool results in better performance than would
creating a new connection for each request that needs to access the LDAP server. The following
properties are particular to the JNDIInitialContextPool:
326
18 - LDAP Repositories
Property
Description
Default Value
$class
class name
atg.service.resourcepool.
JNDIInitialContextPool
JNDIEnvironment
InitialContextEnvironment
createDirContexts
Should InitialDirContext
objects be created rather than
InitialContext objects?
true
createMonitoredContexts
True
/atg/adapter/ldap/InitialContextEnvironment
This component specifies the JNDI environment properties used to create a JNDI InitialDirContext.
You need to configure this component to point to your LDAP directory server. You are most likely to need
to set the following properties (other than the class definition):
Property
Description
Default Value
$class
class name
atg.adapter.ldap.
LDAPJNDIEnvironment
providerURL
ldap://localhost:389
327
18 - LDAP Repositories
securityAuthentication
simple
simple
cn=ldapadmin
securityCredentials
ldapadmin
otherProperties
null
otherProperties=
com.sun.jndi.ldap.someProperty
=someValue
securityAuthentication Property
The securityAuthentication property must be set to match an appropriate type of security
authentication for your LDAP server. For example, you can use the CRAM-MDS setting only if you have
configured your LDAP directory server appropriately. See Suns LDAP tutorial on the subject of SASL at
http://java.sun.com/products/jndi/tutorial/ldap/security/sasl.html for more information on the valid SASL
mechanisms. Note also that if you set this property to none, the LDAP server will treat the LDAP repository
as an anonymous client. Depending on how your LDAP server is configured, you may therefore be unable
to create, modify, or delete LDAP directory entries through the LDAP repository.
328
18 - LDAP Repositories
initialContextFactory
objectFactories
controlFactories
stateFactories
URLPkgPrefixes
DNSURL
authoritative
batchSize
referral
securityProtocol
language
See the Javadoc for javax.naming.Context for more information about these properties.
Furthermore, the InitialContextEnvironment component has the following properties that apply to
LDAP service providers in general or are specific to Suns JNDI LDAP service provider:
LDAPVersion
binaryAttributes
connectControls
deleteRDN
derefAliases
typesOnly
refSeparator
socketFactory
referralLimit
BERTrace
schemaBugs
See Suns JNDI Implementor Guidelines for LDAP Service Providers, section 3, Environment Properties, at
http://java.sun.com/products/jndi/jndi-ldap-gl.html#PROP for more information.
/atg/adapter/ldap/LDAPItemCache
This component is an LRU cache that maps repository item IDs to persistent repository items.
Property
Description
Default Value
$class
class name
atg.service.cache.Cache
cacheAdapter
/atg/adapter/ldap/
LDAPItemCacheAdapter
maximumEntryLifetime
-1
329
18 - LDAP Repositories
maximumCacheEntries
500
/atg/adapter/ldap/LDAPItemCacheAdapter
This component is used by the LDAPItemCache to retrieve persistent repository items from the directory.
Property
Description
Default Value
$class
class name
atg.adapter.ldap.
LDAPItemCacheAdapter
repository
/atg/adapter/ldap/
LDAPRepository
/atg/adapter/ldap/LDAPQueryCache
This component is an LRU cache that maps repository search queries to the repository item IDs of the
query results.
Property
Description
Default Value
$class
class name
atg.service.cache.Cache
cacheAdapter
/atg/adapter/ldap/
-1
maximumEntryLifetime
LDAPQueryCacheAdapter
1000
330
18 - LDAP Repositories
/atg/adapter/ldap/LDAPQueryCacheAdapter
This component is used by the LDAPQueryCache to perform repository queries.
Property
Description
Default Value
$class
class name
atg.adapter.ldap.
LDAPQueryCacheAdapter
repository
/atg/adapter/ldap/
LDAPRepository
Change this property to ensure consistency with the LDAP password encryption method youve chosen.
For Sun ONE Directory Servers, set the passwordHasher property like this:
passwordHasher=/atg/adapter/ldap/NDSPasswordHasher
The NDSPasswordHasher component supports SHA or no encryption. Set the encryption property of
the /atg/adapter/ldap/NDSPasswordHasher to the appropriate value:
encryption=SHA
331
18 - LDAP Repositories
The LDAP repository definition file uses the XML tags described in this section. See also:
at the end of this chapter. Note that while the LDAP repository definition is similar in many ways to the
SQL repository definition, the LDAP repository definition uses its own XML document type definition and
syntax. Note also that, unlike the SQL repository, the LDAP repository cannot use XML file combination to
combine XML files in the CONFIGPATH that have the same name. Instead, you need to use a single LDAP
repository definition file in your CONFIGPATH.
DOCTYPE Tag
All XML documents declare their doctype, referencing a document type definition file. You should not
need to change the reference to the DTD in the following DOCTYPE tag:
<!DOCTYPE ldap-adapter-template
PUBLIC "-//Art Technology Group, Inc.//DTD LDAP Adapter//EN"
"http://www.atg.com/dtds/ldap/ldap_1.0.dtd">
ldap-adapter-template Tag
The entire repository definition is enclosed in an <ldap-adapter-template> tag.
Child Tag
How many?
header
exactly one
view
one or more
header Tag
The <header> tag provides information that can help you manage the creation and modification of
repository definition files.
Child Tag
How many?
name
zero or one
332
18 - LDAP Repositories
author
zero or more
version
zero or one
view Tag
A repository definition file must include one <view> tag for each RepositoryView in your repository.
Attribute
Description
Value
name
default
Child Tag
How many?
item-descriptor
exactly one
search-root
one or more
item-descriptor Tag
Each RepositoryView in the LDAP repository includes a single item descriptor, which is defined by an
<item-descriptor> tag.
Attribute
Description
Value
name
parent
333
18 - LDAP Repositories
Child Tag
How many?
id-property
zero or one
object-classes-property
zero or one
object-class
zero or more
property
zero or more
child-property
zero or more
new-items
zero or one
id-property Tag
The <id-property> tag defines the profile ID property in the RepositoryItem and the LDAP entry. The
tag is always empty. For example:
<id-property name="id" in-ldap="true" ldap-name="dpsid"/>
The <id-property> tag is required in a definition of a base item descriptor (an item descriptor that
doesnt have a parent) but is not allowed in a child item descriptor that inherits from a parent.
Attribute
Description
Value
name
Required.
in-ldap
ldap-name
display-name
description
object-classes-property Tag
The <object-classes-property> tag exposes the object classes of an LDAP entry as a property of a
RepositoryItem. This tag is always empty. For example:
334
18 - LDAP Repositories
Like the <id-property> tag, the <object-classes-property> tag is required in a definition of a base
item descriptor (an item descriptor that doesnt have a parent) but is not allowed in a child item
descriptor that inherits from a parent. The propertys type is String[], a multi-valued String. For
example, if an item descriptor definition has the <object-classes-property> tag in the preceding
example and has the following object classes definition:
<object-class>top</object-class>
<object-class>person</object-class>
<object-class>organizationalPerson</object-class>
<object-class>inetorgPerson</object-class>
then its repository items will have the following objectClasses property:
objectClasses=top,person,organizationalPerson,inetorgPerson
Attribute
Description
Value
name
Required.
ldap-name
display-name
display-property
description
object-class Tag
The object-class tags specify all the object class values corresponding to the given item descriptor. If
the object class has ancestor object classes, they must all be specified. For example:
<object-class>top</object-class>
<object-class>person</object-class>
335
18 - LDAP Repositories
<object-class>organizationalPerson</object-class>
<object-class>inetorgPerson</object-class>
The object class information is required in the item descriptor specification so that when a new item is
created for the given item descriptor and added to the repository, the corresponding LDAP entry can be
created with the given object class values. Thus, for example, if an item is created in the context of the
user item descriptor, the new LDAP directory entry has objectclass attribute values of top, person,
organizationalPerson, and inetorgPerson.
property Tag
Property tags define the properties of a repository item and map the repository item properties to LDAP
entry attributes.
Child Tag
How many?
option
zero or more
attribute
zero or more
Attribute
Description
Value
name
Required.
ldap-name
data-type
336
18 - LDAP Repositories
multi
displayname
description
default
required
readable
writable
queryable
hidden
expert
editor-class
string
String
big string
String
date
java.util.Date
timestamp
java.sql.Timestamp
enumerated
String
boolean
Boolean
int
Integer
337
18 - LDAP Repositories
byte
Byte
binary
byte[]
short
Short
float
Float
double
Double
long
Long
option Tag
For properties whose data-type is enumerated, use <option> tags to indicate each of the possible values
of the enumerated property. For example:
<property name="gender" data-type="enumerated">
<option>male</option>
<option>female</option>
</property>
attribute Tag
An <property> tag can contain zero or more <attribute> tags. The <attribute> tag is an empty tag
that defines the propertys feature descriptor value or values. This tag allows you to associate arbitrary
name/string value pairs with any property. The name/value pairs are added to the propertys property
descriptor via the setValue method of java.beans.FeatureDescriptor, and can later be used by the
application. Here is an example:
<property name="employeeNumber" data-type="string">
<attribute name="unique" value="true"/>
</property>
Attribute
Description
Value
name
Required.
value
Required.
338
18 - LDAP Repositories
child-property Tag
If an item descriptor has a parent from which it inherits, you can use <child-property> tags to override
the default values of properties of the parent item descriptor. The only aspect of the parent property
definition that can be overridden is the propertys default value; the propertys data-type, etc. must stay
the same. For example, suppose there is a parent item descriptor with the following property:
<item-descriptor name="user">
...
<property name="department" default="Other"/>
...
</item-descriptor>
You can create a child property that overrides the default value of the department property:
See Item Descriptor Hierarchies and Inheritance in the LDAP Repository Architecture section of this
chapter.
Attribute
Description
Value
name
Required.
default
339
18 - LDAP Repositories
new-items Tag
The <new-items> tag describes how new items within the item descriptor are created and identified.
Attribute
Description
Value
allowed
parent-dn
rdn-property
a new item whose login property is nat will have a corresponding RDN of uid=nat (since the LDAP
repositorys login property maps to the uid attribute in the LDAP directory), and the DN will be
uid=nat,ou=Marketing,o=quincyfunds.com.
If a child descriptor definition does not contain a <new-items> tag, it simply inherits the parents item
creation behavior. See New Item Creation in the LDAP Repository Architecture section of this chapter.
search-root Tag
A Repository Views contents can be restricted to a particular location or set of locations within the
directory tree. For example, one might want to specify that the engineer view contains only entries in
the ou=Engineering,o=quincyfunds.com branch of the directory tree. Even if other items that satisfy
the engineer item descriptor are encountered somewhere in the LDAP directory (perhaps for testing
purposes), they are not considered to be part of the engineer view. We call the tree branches that
comprise a Repository View search roots, because they determine which parts of the directory tree are
searched when a repository query is constructed on the view. The <search-root> tag is a child tag of
the <view> tag that limits the Repository View to the specified roots of the LDAP directory tree.
When a query is performed on the view, only those items that reside in one of the specified search roots
and satisfy the views item descriptor are returned. At least one search root must always be specified, but
it may well point to the directory suffix (i.e., the search root may span the entire directory tree). See the
Repository Views in the LDAP Repository section of this chapter.
340
18 - LDAP Repositories
Attribute
Description
Value
dn
Required.
recursive
Boolean.
Default is true
check-classes
Boolean.
Default is true
341
18 - LDAP Repositories
342
18 - LDAP Repositories
http://www.atg.com/dtds/ldap/ldap_1.0.dtd
<?xml encoding="UTF-8"?>
<!-====================================================================
ldap-adapter-template.dtd - document type for LDAP Adapter templates
Version: $Change: 191227 $$DateTime: 2001/07/06 12:42:29 $$Author: nat $
====================================================================
-->
<!-- Flag datatype, and values -->
<!ENTITY % flag "(true | false)">
<!-- The whole template -->
<!ELEMENT ldap-adapter-template (header, view+)>
<!-- The header -->
<!ELEMENT header (name?, author*, version?)>
<!-- Name of template -->
<!ELEMENT name (#PCDATA)>
<!-- The author(s) -->
<!ELEMENT author (#PCDATA)>
<!-- Version string -->
<!ELEMENT version (#PCDATA)>
<!-- View(s) -->
<!ELEMENT view (item-descriptor, search-root*)>
<!ATTLIST view
name
CDATA
#REQUIRED
default
%flag;
"false"
>
<!-- Item descriptor(s) -->
<!ELEMENT item-descriptor (id-property?, object-classes-property?, object-class*,
property*, child-property*, new-items?)>
<!ATTLIST item-descriptor
name
CDATA
#REQUIRED
parent
CDATA
#IMPLIED
display-name
CDATA
#IMPLIED
description
CDATA
#IMPLIED
hidden
%flag;
"false"
expert
%flag;
"false"
display-property
CDATA
#IMPLIED
>
<!-- Id property -->
343
18 - LDAP Repositories
344
18 - LDAP Repositories
345
18 - LDAP Repositories
346
18 - LDAP Repositories
Index
A
access rights
repository items, 286
ACL (Access Control Lists)
access rights, 293
syntax, 292
add-item tag, 148
alias derivation method, 73
atg.adapter.gsa.* package, 47
atg.adapter.gsa.event.GSAEventServer, 133
atg.adapter.gsa.GSAPropertyDescriptor, 98
atg.adapter.gsa.invalidator.GSAInvalidatorService. See
SQL repositories, distributed cache invalidation
atg.adapter.gsa.query.SqlPassthroughQuery, 103
atg.repository.content.ContentRepositoryItem, 221
atg.repository.ItemDescriptorImpl, 19
atg.repository.loader.FileSystemMonitorScheduler, 235
atg.repository.MutableRepository, 21, 23
atg.repository.NamedQueryView interface, 112
atg.repository.ParameterSupportView, 104
atg.repository.PropertiesChangedEvent, 25, 124
atg.repository.PropertiesChangedListener, 26, 125
atg.repository.query.QueryDescriptorImpl, 112
atg.repository.QueryBuilder, 22, 29
atg.repository.QueryDescriptor interface, 112
atg.repository.QueryOptions, 30
atg.repository.Repository, 21
atg.repository.RepositoryItem, 18, 23
atg.repository.RepositoryItemDescriptor, 19
atg.repository.RepositoryUtils, 26, 150
atg.repository.RepositoryView, 22
atg.repository.RepositoryViewContainer, 22
B
BLOBs, 175
C
cache invalidation
SQL repository. See caches (SQL repository), flushing
caches (composite repository), 269
caches (LDAP repository)
item cache, 329
query cache, 330
caches (SQL repository), 123
cache contents, viewing, 139, 188
347
Index
CompositeRepository components
configuring, 267
configuration files, secured repositories. See secured
repositories, definition files
constraints
REFERENCES, 56
content repositories, SQL, 221
ContentHandler, 236
ContentRepositories component
secured repositories, 290
D
database meta data, 117
data-type correspondences
LDAP repository properties to Java types, 337
SQL database to SQL repository, 174
date properties
SQL repositories, 80
debugLevel property, 158
definition files
LDAP repositories. See LDAP repository definition files
secured repositories. See secured repositories,
definition files
SQL repositories. See SQL repository definition files
derivation methods
alias, 73
collectiveUnion, 74
firstNonNull, 72
firstWithAttribute, 72
firstWithLocale, 73
union, 74
derived properties, 68, 178
alias derivation method, 73
collectiveUnion derivation method, 74
derivation methods, 71
firstNotNull derivation method, 72
firstWithAttribute derivation method, 72
firstWithLocale derivation method, 73
override properties, 70
union derivation method, 74
distinguished names. See LDAP repositories,
distinguished names
DNs. See LDAP repositories, distinguished names
Document Type Definition (DTD)
composite repository, 277
LDAP repository definition files, 342
Repository Loader manifest file, 239
secured repository definition files, 298
SQL repository definition files, 189
DTD. See Document Type Definition (DTD)
E
enumerated properties
LDAP repository, 319
SQL repositories, 77
events
cache invalidation, 124
property changes, 124
F
file combination, XML, 49
FileSystemMonitorScheduler, 235
configuring, 243
FileSystemMonitorService, 235
configuring, 241
filterQuery property, 103
firstNonNull derivation method, 72
firstWithAttribute derivation method, 72
firstWithLocale derivation method, 73
flushing caches. See caches (SQL repository), flushing
full text search queries, 38
G
Generic SQL Adapter (GSA). See SQL repositories
GetRepositoryItem Web services, 255
I
inheritance, item descriptor
LDAP repositories, 317
SQL repositories, 62
item caches. See caches
item descriptors, 18, 19
Dynamic Beans, 20
inheritance, LDAP repositories, 316
inheritance, SQL repositories, 62
LDAP repositories, 314
J
java.sql.CallableStatement, 111
java.sql.ResultSet, 111
javax.naming.Context, 328
JNDI
accessing an LDAP directory, 312
L
last-modified properties
SQL repositories, 81
LDAP (Lightweight Directory Access Protocol), 307, 308
data representation, 310
directory schema, 311
distinguished names, 310
entries, 310
entry attributes, 310
entry types, 310
hierarchical tree structure, 309
JNDI, access through, 312
object classes, 310
LDAP repositories, 307
attribute tags, 319
configuring components, 324
348
Index
REFERENCES constraints, 56
remove-item tag, 150
repositories
data-types, 20
item descriptors. See item descriptors
LDAP. See LDAP repositories
mutable, 21, 23
queries. See queries
manifests
Repository Loader, 239
Repository Loader tags, 240
meta data
database, 117
O
Oracle
ConText full text search engine, 39
outer joins
in SQL repository queries, 116
override property
in derived properties, 70
P
parameterized queries, 104
password encryption
LDAP repository, 331
password hashing. See password encryption
PerformRQLCountQuery Web services, 259
PerformRQLQuery Web services, 257
PropertiesChangedEvent, 25
property fetching
SQL repositories, 98
property mapping
Composite repositories, 265
Q
queries. See also Repository Query Language (RQL)
ATG Control Center, 34
bypassing RQL, 103
composite repository, 268
LDAP repository, 323
named, 107
parameterized, 104
QueryBuilder interface, 29
QueryOptions, 30
Repository API, 29
Repository API example, 31
stored procedures, 110
queries, unsupported
LDAP repositories, 324
SQL repositories, 118
query caches. See caches
parameterized queries, 105
349
Index
Repository API summary, 21
secured. See secured repositories
SQL. See SQL repositories
repository filtering, 101
repository IDs, 19
repository items, 18, 23
adding, 148
cloning, 26
deleting, 150
IDs, 19
LDAP repository, 314
PropertiesChangedEvent, 25
removing references, 150
updating, 24, 150
Repository Loader, 221, 233
administration UI, 236
client, 237
ErrorPolicy, 237
example, 248
manifest tags, 240
manifests, 239
Repository Query Language (RQL), 34
ALL queries, 38
collection queries, 37
comparison queries, 35
COUNT queries, 38
examples, 42
full text search queries, 38
grammar definition, 42
ID-based queries, 39
INCLUDES ITEM queries, 38
IS NULL queries, 38
limiting result sets, 40
logical operators (AND, OR, NOT), 37
ORDER BY directives, 40
ordering query result sets, 40
parameterized queries, 41
pattern match queries, 36
property of property queries, 36
RANGE directive, 40
text comparison queries, 36
Repository View. See item descriptor
Repository Views
LDAP repository, 321
RepositoryUtils. See atg.repository.RepositoryUtils
required properties
SQL repositories, 80
RL module. See Repository Loader
RLClient, 237
hints file, 238
RQL. See Repository Query Language (RQL)
RQL filters. See repository filtering
rqlFilterString property, 103
S
secured repositories, 285
access rights, 286
ACL property, 288, 296
ACLs, 292
350
Index
SqlPassThroughQuery, 103
startSQLRepository script, 152
arguments, 153
syntax, 153
stored procedures, 110
T
table ownership, 117
tag attribute, 148
timestamp properties
SQL repositories, 80
transactions
repositories, 47
transient properties, 88
caching, 125
composite repositories, 264
TypeMapper, 235
configuring, 244
TypeMapping, 235
configuring, 244
U
union derivation method, 74
unique properties
SQL repositories, 80
unsupported queries. See queries, unsupported
update-item tag, 150
User Authority components, 292
user-defined property types, 92
W
Web services, 255
GetRepositoryItem, 255
PerformRQLCountQuery, 259
PerformRQLQuery, 257
security policies, 261
wildcard characters
SQL repository queries, 115
X
XML file combination, 49
Xml2RepositoryContentHandler, 236
configuring, 247
351
Index