As One Framework White Paper

June, 2001

Technologies

As One Technologies
5572 Smetana Drive Minnetonka, MN 55343 Ph (952) 996-0084 Fax (952) 996-0085

Peer-to-Peer Agent-Based Computing

As1 Framework White Paper June, 2001

Introduction ......................................................................................................................... 2 Shortcomings of existing models ........................................................................................ 2 Problems not addressed by existing models ....................................................................... 4 As1 framework introduction ............................................................................................... 5 Communication between agents ......................................................................................... 8 Extensible service model .................................................................................................... 9 Extensible communication model ....................................................................................... 9 Metadata ............................................................................................................................ 10 As1 programming methodology summary ....................................................................... 10 Summary ........................................................................................................................... 11

as

Page 1

As One Framework White Paper

June, 2001

As1 Framework
Introduction As1s technology offering is a modular component-based framework that can be applied to a number of vertical products and solutions. While the primary goal of As1 is to simplify the development, deployment and management of distributed information systems, this technology is beneficial for non-distributed systems as well. To date application development tools and technologies, and middleware offerings are separated. Component architectures of today target developers and produce monolithic applications. As systems complexities increase, businesses and developers recognize inherent limits of these architectures and approaches. The Internet and globalization dictates a distributed application environment that exacerbates the problems and limitations normally encountered in typical enterprise application scenarios. Technologies used today are not flexible enough to satisfy the following: Large-scale, widely distributed systems require a decoupled, asynchronous interaction model. Large-scale, heterogeneous systems with many autonomous entities and parallel activities require a new programming model. Future computing environments require software systems that support customization, adaptation and dynamic modifications. Shortcomings of existing models Existing component technologies (COM/DCOM, Java Beans/RMI, and CORBA components) are all built as attempts to objectify client-server architecture and all based on some incarnation of RPC (remote procedure call). They all assume that some glue code will be written to pull components into the final application, provide concurrency, data access, application services, etc. This glue, as well as the application architecture, is outside of these component technologies, leaving it up to application developers to build it from scratch for each new application (Fig 1). Even if an application is built using components, the resulting code creates deep interdependencies between individual components as well as between components and the glue code. Problems escalate when the application has to be distributed, adding inherent problems of RPC-related proxies and stubs, network latencies, error handling, and synchronous access to information. Developers have to address these technical problems rather than concentrate on the business domain problems. Another problem comes from the encapsulation and method-based access to data that objects and components provide. Application developers need to know the internal implementation of components to identify a sequence of calls necessary to invoke functionality. Because this information is not present in an interface specification or IDL it becomes implementation specific. It also means that a client that expects a specific interface from the component may not work with all components that implement that interface, resulting in application code that is rigid and implementation specific. Such applications, once

Page 2

As One Framework White Paper

June, 2001

deployed, are difficult to change, maintain, and integrate with other applications. They also have an inherent tight coupling with the underlying deployment schema.
Execution Thread Application Component Instance Component Instance Component Instance Component Instance Component Instance Component Instance Component Instance

All information about components is hardcoded as part of the glue code

Glue Code

Component Calls

Fig 1 Application servers (COM+, Java Beans EE) basically extend the client-server architecture with additional tiers for business/application logic, web presentation and optimized database access. These technologies allow for better transactional scalability and some manageability improvements, but dont solve the problem of monolithic, handcoded, closed-for-changes applications. By only promoting component-based development to application developers they leave after-deployment configuration and/or customization problems untouched. Web services, while recognizing a need for integration and distribution, only provide a way to expose the server-side of client-server through open protocols such as SOAP and ebXML. They dont address implementation of the functionality but rather access to that functionality. Web services still require glue code to pull it all together and, by being platform agnostic, are not in position to offer any high level architecture. Middleware products (MQ, MSMQ, Tibco) have attacked the distribution problem. They can deliver data from point A to point B reliably and on time. The problem with middleware offerings is that they dont have an integrated component model; they deal with data mapping and delivery, not with application architectures (Fig 2). There is no processing of any kind or programmability built-in to middleware offerings; it is up to application developers to plug middleware into their applications, usually by mapping components internal data to simple data types and data structures that middleware products are operating on using middleware specific APIs. This process creates deep interdependencies on each end of the data exchange, producing tightly coupled and rigid applications.

Page 3

As One Framework White Paper

June, 2001

Application

Application

Application

API Calls

API Calls

API Calls

Middleware

Fig 2 The bottom line is that applications are still built from scratch without realizing the full benefits of components, existing middleware offerings, and open web standards. Component technologies are used by software developers but arent exposed to the endusers. Programmability or reconfiguration of applications, even when available, is not adequate because the applications themselves arent componentized. Although most developers recognize that frameworks are the main architectural technique for supporting flexibility and reusability, few in the middleware arena use or offer them. Moreover, frameworks typically exploit software design patterns targeted for monolithic application development, not patterns that distributed component systems could utilize. Problems not addressed by existing models Many applications exist outside of the client-server or n-tier paradigm, forcing developers and end-users to build proprietary systems from scratch, without the benefit of component technologies, efficient tools and powerful languages. These products and/or applications have the following in common: Distributed information gathering and processing in the real world information exists in many places. It is not practical, and in many cases not possible, to move data to the central location to be processed. Real-time information processing same as above with added timing requirements where network latencies and/or throughput become a limiting factor. Programmable by end-user a lot of products would benefit if they could provide end-user with a domain-specific programming/configuration paradigm. Configure-to-order as part of mass customization similar to the above, but with programmability option used by ISV or OEM to essentially customize every copy of the product to the end-user requirements. Dynamically adaptable business rules, processes, requirements are changing constantly, dynamic enterprise is a buzzword, but can you change your application on the fly? Need for a distributed integration platform today applications are rarely stand-alone. Need for development framework and methodology how many top software developers can you hire and make them work together? Can you translate you application requirements to code? How much time are you willing to spend to develop all the required application plumbing?

Page 4

As One Framework White Paper

June, 2001

As1 framework introduction The As1 framework comprises a software development methodology and runtime environment for agent-based information systems, sub-systems or applications. With this paradigm software is as a collection of concurrent agents performing individual tasks; through collaboration these agents can perform an infinite number of complex tasks. Agents are intelligent, long-lived, autonomous, self-aware entities implemented as software modules built to run within As1 frameworks runtime (Fig 3).
As1 OLEDB Provider

As1 OPC Server

Custom Client

As1 COM API

Shared System Repository

As1 Development and management tools

Custom Development and management tools

Ethernet

Ethernet

Configuration And Management Access

Tagname Data Access

Local System Repository and Runtime Metadata

As1 Container

As1 Service Manager Data Distribution Services

As1 Dynamic Protocol Switch

As1 Transport Manager

Tracing and Debugging

Persistence

As1 Messaging

Local System Repository and Runtime Metadata

Ethernet

Fig 3 Page 5

UDP or TCP/IP

IBM MQ

Custom Services

Custom Transports

Tasking

MSMQ

SOAP

As One Framework White Paper

June, 2001

As1 frameworks runtime consists of following required components: As1 Container agents hosting environment As1 Dynamic Protocol Switch agent to agent collaboration As1 Local System Repository configuration and runtime metadata As1 Service Manager generic extensibility mechanism As1 technology offers new development paradigm that supports numerous levels of abstraction and is inherently programmable and configurable. As1 technology can be packaged as a stand-alone product, an embeddable engine, or a development tool/modeling paradigm. There are number of optional components that may be utilized when customizing framework or framework-based products. While recognizing that existing and emerging technologies have their proponents and opponents, and assuming that no one technology could satisfy everyone, As1s technology is component, language, and middleware agnostic. In other words it can be implemented on top of existing component technologies, can be used from multiple programming languages, and can utilize numerous middleware offerings and open web standards. Packaged as components, libraries of agents can be created to target various application domains. Agent collaboration is configurable and separated from agent implementation. This modular paradigm allows large complex systems to be developed incrementally. By identifying system components that can be modeled as stand-alone tasks, developers can design, build, and test them one by one or in small groups, eventually getting a complete system up and running. An agent can be implemented as state machine that models a continuous task, or it can model a processing task that only processes incoming data when it changes. The general rule is that each agent encapsulates a task or process and can exchange data with other agents. This is fundamentally different from exposing functionality for others to use. The agent inherently has the ability to perform processing even when running with no peers. How a system is modeled is completely up to the developer, who balances goals and requirements to identify system decomposition. Agent could represent, for example, one business rule that verifies and formats incoming data, or it could represent an entire business process that encompasses all the business rules. The smaller and simpler agents are, the simpler and faster they are to develop and test, and the system they form becomes more modular. At the same time small agents complicates system assembly because a large number of them are required to create a system. Large agents, while more difficult to develop, may simplify system assembly by reducing the overall number of agents. Depending on the system requirements there are tradeoffs between size and number of agents in the system (Fig 4).

Page 6

As One Framework White Paper

June, 2001

Agent
Port Port

Port

Port

Agent

Port

Proccess Coordinator Maintenance Agent

Port

Agent
Port Port

Port

Port

Port

Agent
Port

Port

Agent

Port

Agent

Port

Machine1

Machine2

Maintenance Agent

Optional To Quality Manager Agent Optional To MES Manager Agent

Agent
Port

Port

Optional To Maintance Manager Agent

Fig 4 Using As1s methodology an information system can be assembled and configured from components (agents), with no conventional programming required, by the domain expert(s) who best understands all the requirements of the new system. The As1 framework provides a dynamic run-time environment for agents. A system can be modified or reconfigured while it is up and running. Individual agents can be shutdown and replaced, new agents can be added, the agent environment could change, and collaboration relations between agents could change. This also means that systems built using the As1 framework can be deployed gradually, rapidly changed and adapted, and evolve over time.

Page 7

As One Framework White Paper

June, 2001

Communication between agents The As1 framework recognizes that collaboration between agents may be arbitrarily complex and cant be described in terms of simple data passing or a client-server relationship that is unidirectional (always initiated by the client) and has an inherent coupling problem. At the same time the distributed nature of the systems and the autonomy of the agent concept doesnt map well to RPC or RMI type synchronous unidirectional communication. Agents collaboration is best described as a peer-to-peer paradigm where each agent can interact with any other agent as peers, freely exchanging data and commands. To define these exchanges the As1 framework uses protocols. In this paradigm a protocol is defined as collection of ordered, directional messages (incoming or outgoing), each with its own data payload. Part of the protocol definition is a declaration of its ability to support one-to-one, one-to-many, many-to-one, and manyto-many communication. Once defined, a protocol may be implemented by any number of agents. Each agent exposes its implemented protocols as ports. A port represents one side of a bi-directional communication channel that uses a protocol. Ports on opposite ends of the channel are complementary. In other words one port is a mirror implementation of the second one; incoming messages of one port are outgoing messages of its complimentary port. An agent may support one or more protocols by implementing appropriate ports to enable collaboration with other agents. The frameworks tools use protocol information to define agent-to-agent relations, making it possible creating very complex multi-agent solutions. Existing ways to define component interaction (IDL, WSDL) dont offer any way to define and extract sequences of method calls. Developer using a component is charged with a task to guess how component developer implemented a definition. By adding ordering instructions to the protocols As1 framework is solving this problem. These instructions allow framework to reinforce appropriate ordering of exchanges as well as documenting collaboration requirements for developers. This takes assumptions about sequences of exchanges out of programming. Protocols provide a powerful design paradigm that allows not only functionality to be componentized, but collaboration as well. A set of well-designed protocols may be used as a foundation or functional skeleton for a new system or application. For example, a SCADA (supervisory control and data acquisition) system may define protocols for I/O (input/output) access, alarm handling, data logging, etc. A BI (business intelligence) system may define protocols for data entry, data processing, storage, and process flow management. Products built using the As1 framework could provide predefined protocols as part of the offering or include tools that allow product users to define new protocols, or a combination of the above.

Page 8

As One Framework White Paper

June, 2001

Extensible service model As1s framework includes base services that provide such functionality as tasking, scheduling, persistence, tracing and debugging. To support run-time engine extensions As1 provides the ability to add product specific run-time services (custom services on Fig 3). These services encapsulate functionality that is accessible to agents at runtime, however agents have to be developed with intimate knowledge of the services they use. Agents are able to search for a specific service and, if found, use its functionality to perform specific tasks. Additional services may include data access, I/O drivers, shared data access, web server integration, etc. Extensible communication model To provide integration with middleware products and to support various message delivery mechanisms the As1 framework defines transport components (Fig 3 under As1 Transport manager). Multiple transport components can be used concurrently to enable operations in environments where multiple communication channels are present (Fig 5). This ability of the framework can be used to develop integration or bridge nodes that tie stand-alone systems together. This also allows systems that span multiple sites that are connected by the Internet. In this case a system can use more efficient UDP-based messaging within sites and SOAP-based messaging to connect sites together.

As1 Engine

As1 Engine

As1 Engine

SOAP Transport

UDP Transport

UDP Transport

UDP Transport

MQ Transport

Ethernet

Internet Ethernet

SOAP Transport

MQ Transport MQ Transport

As1 Engine

As1 Engine As1 Engine

Fig 5

Page 9

As One Framework White Paper

June, 2001

Object-based systems have to communicate and integrate with tag based systems and technologies such as OPC, ODBC, OLE DB, SECS-GEM, etc. These technologies operate on individual data items or groups of data items, where data item represents a simple data type (Boolean, Integer, Double, String, etc.) They dont have a notion of commands, events, or messages associated with object-based technology. This impedance mismatch between the two camps makes it difficult to translate information back and forth. The As1 framework allows components to publish simple data types into the integrated tag database (see Fig 3 Publishing Database). This memory resident publishing database, while optional, supports exception-based access to the tags that are managed by individual agents. The framework maintains tag naming as well as optional tag persistence. Custom handlers could be created to integrate with various tag-based systems or to build extensions that can be better modeled with tag type data access. An OLE DB driver, for example, makes a running system look like a real-time distributed database to the end-user, where an OPC driver allows integration with SCADA type system. Metadata The As1 framework uses metadata, generated by development tools, to dynamically manage agents lifetime, environment, and collaboration with other agents. The metadata repository is also available for services and transport components to store their configuration data. Metadata is integrated with run-time components, ensuring that they are synchronized at all times. At the same time the repository is accessible to any application-specific extensions, allowing the same level of integration as offered to the frameworks base components. The repository uses XML to describe metadata; however, physical data storage is abstracted to allow multiple options such as XML files or relational database. Depending on packaging, the As1 framework-based system may support a centralized repository and concurrent development by multiple developers and administrators, or it can be packaged as a stand alone, file-based, single developer system. As1 programming methodology summary From the developers point of view the agent paradigm simplifies the amount of work needed to get a job done. Components are designed, built, and tested before system assembly has begun, simplifying overall system debugging and troubleshooting. Protocols provide well-defined collaboration abstraction, and provide a decoupling mechanism. Developers are also presented with a much simpler set of run-time requirements, allowing them to concentrate on the actual code related to the applications domain. This methodology can have multiple implementations to satisfy the diverse needs of many markets. For example, a toolset that creates personal collaboration systems would likely be very different from a toolset for plant-floor control. The common denominator of these toolsets is the As1 runtime, which guarantees that agents for these diverse applications can co-exist and interoperate.

Page 10

As One Framework White Paper

June, 2001

Developers have freedom to design agents for their systems as simple or as complex as necessary, as well as present to end-users, if necessary, a configuration/development view that models that users domain as close as possible. This ability to adapt on multiple levels makes As1s framework simple to integrate into existing products as well as a compelling platform for the new product development. It provides the following benefits, not only to products developers but also to the end-users: Plug-and-play of agent components Software reuse A natural means of controlling concurrency Evolving behavior A closer mapping of software implementation to the enterprise Improved performance through distribution of processing Improved fault tolerance through distribution of processing Inherent programmability With the As1 methodology connections between physically remote components are as easy to establish, manage and modify as connections between components on a single computer. This is because the As1 methodology separates logical configuration from the physical. Summary As1 technologies present a new paradigm for creating software. Applications are no longer monolithic and fixed, but are modular and adaptable. Relationships between software entities are no longer master/slave, with clients initiating all behavior and servers only responding to clients. Components are now autonomous entities that interact as peers, yielding more powerful solutions that are more easily created, managed and adapted. These technologies can be applied to create highly integrated systems that can perform a wide range of tasks. In a manufacturing environment, an agent that manages a piece of equipment can collaborate with an e-business component. Simultaneously, the equipment management agent can collaborate with an agent that performs quality control. Agents can also be configured to keep people informed. Agents can collectively perform everything from plant-floor control to e-business. The end-result is a vertically and horizontally integrated enterprise that behaves as a single entity to improve productivity and responsiveness.

Page 11