Virtually all large computer-based systems are now distributed systems

a collection of independent computers that appears to the user as a single coherent system.

Distributed Software Engineering

Information processing is distributed over several computers rather than confined to a single machine Distributed software engineering is therefore very important for enterprise computing systems

37

Resource sharing
Sharing of hardware and software resources

Openness
Use of equipment and software from different vendors

Concurrency
Concurrent processing to enhance performance

Scalability
Increased throughput by adding new resources

Fault tolerance
The ability to continue in operation after a fault has occurred

Distributed systems are more complex than systems that run on a single processor Complexity arises because different parts of the system are independently managed as is the network There is no single authority in charge of the system so top-down control is impossible

38

39

Ideally, users should not be aware that a system is distributed and services should be independent of distribution characteristics In practice, this is impossible because parts of the system are independently managed and because of network delays
Often better to make users aware of distribution so that they can cope with problems

To achieve transparency, resources should be abstracted and addressed logically rather than physically. Middleware maps logical to physical resources

Open distributed systems are systems that are built according to generally accepted standards Components from any supplier can be integrated into the system and can inter-operate with the other system components Openness implies that system components can be independently developed in any programming language and, if these conform to standards, they will work with other components Web service standards for service-oriented architectures were developed to be open standards

40

41

The scalability of a system reflects its ability to deliver a high quality service as demands on the system increase
Size It should be possible to add more resources to a system to cope with increasing numbers of users Distribution It should be possible to geographically disperse the components of a system without degrading its performance Manageability It should be possible to manage a system as it increases in size, even if parts of the system are located in independent organizations

There is a distinction between scaling-up and scaling-out. Scaling up is more powerful system; scaling out is more system instances

When a system is distributed, the number of ways that the system may be attacked is significantly increased, compared to centralized systems If a part of the system is successfully attacked then the attacker may be able to use this as a back door into other parts of the system Difficulties in a distributed system arise because different organizations may own parts of the system. These organizations may have mutually incompatible security policies and security mechanisms

42

43

The types of attack that a distributed system must defend itself against are:
Interception, where communications between parts of the system are intercepted by an attacker so that there is a loss of confidentiality Interruption, where system services are attacked and cannot be delivered as expected
Denial of service attacks involve bombarding a node with illegitimate service requests so that it cannot deal with valid requests

Modification, where data or services in the system are changed by an attacker Fabrication, where an attacker generates information that should not exist and then uses this to gain some privileges

The quality of service (QoS) offered by a distributed system reflects the systems ability to deliver its services dependably and with a response time and throughput that is acceptable to its users Quality of service is particularly critical when the system is dealing with time-critical data such as sound or video streams
In these circumstances, if the quality of service falls below a threshold value then the sound or video may become so degraded that it is impossible to understand

44

45

In a distributed system, it is inevitable that failures will occur, so the system has to be designed to be resilient to these failures

You know that you have a distributed system when the crash of a system that youve never heard of stops you getting any work done.
Distributed systems should include mechanisms for discovering if a component of the system has failed, should continue to deliver as many services as possible in spite of that failure and, as fast as possible, automatically recover from the failure

Distributed systems that are accessed over the Internet are normally organized as client-server systems In a client-server system, the user interacts with a program running on their local computer (e.g. a web browser or phone-based application). This interacts with another program running on a remote computer (e.g. a web server) The remote computer provides services, such as access to web pages, which are available to external clients

46

50

C2

C3

C4

C12 C11

s1,s2 c1
Server Process
CC1 SC2 CC2

c2
CC3

c3,c4

C1

s1

s4

C10 C5 s2 s3 C9

Server computer
Client Process

Network
c5,c6,c7 c8,c9
CC4 SC1 CC5 CC6

c10,c11,c12 Client computer

C6 C7

C8

s3,s4

51

52

Widely used ways of organizing the architecture of a distributed system:

Master-slave architecture, which is used in realtime systems in which guaranteed interaction response times are required Two-tier client-server architecture, which is used for simple client-server systems, and where the system is centralized for security reasons Multi-tier client-server architecture, which is used when there is a high volume of transactions to be processed by the server

Distributed component architecture, which is used when resources from different systems and databases need to be combined, or as an implementation model for multi-tier clientserver systems Peer-to-peer architecture, which is used when clients exchange locally stored information and the role of the server is to introduce clients to each other

53

54

Master-slave architectures are commonly used in real-time systems where there may be separate processors associated with data acquisition from the systems environment, data processing and computation and actuator management The master process is usually responsible for computation, coordination and communications and it controls the slave processes Slave processes are dedicated to specific actions, such as the acquisition of data from an array of sensors

Control Room Processor

Sensor Processor
Sensor Control Process

Coordination and display Process

Traffic Light Control Processor

Light Control Process

Master

Slave

Slave

Operator Consoles Traffic Flow sensors and cameras Traffic lights

55

56

In a two-tier client-server architecture, the system is implemented as a single logical server plus an indefinite number of clients that use that server
Thin-client model, where the presentation layer is implemented on the client and all other layers (data management, application processing and database) are implemented on a server Fat-client model, where some or all of the application processing is carried out on the client. Data management and database functions are implemented on the server

Used when legacy systems are migrated to client server architectures

The legacy system acts as a server in its own right with a graphical interface implemented on a client

A major disadvantage is that it places a heavy processing load on both the server and the network

57

58

More processing is delegated to the client as the application processing is locally executed Most suitable for new C/S systems where the capabilities of the client system are known in advance More complex than a thin client model especially for management. New versions of the application have to be installed on all clients

Presentation Thin-Client model

Server Client
Database Data management Application Processing

Presentation Application Processing

Server
Fat-Client model

Client

Database Data management

59

60

In a multi-tier clientserver architecture, the different layers of the system, namely presentation, data management, application processing, and database, are separate processes that may execute on different processors This avoids problems with scalability and performance if a thin-client two-tier model is chosen, or problems of system management if a fat-client model is used

Tier 1. Presentation
Client

HTTPS interaction

Client

Web Server
SQL query
Account service provision

Database Server
SQL Customer account database

Client

Client

Tier 2. Application Processing and Data Management

Tier 3. Database processing

61

62

Architecture

Applications

Architecture

Applications

Two-tier clientserver architecture with thin clients

Legacy system applications that are used when separating application processing and data management is impractical. Clients may access these as services. Computationally intensive applications such as compilers with little or no data management. Data-intensive applications (browsing and querying) with nonintensive application processing. Browsing the Web is the most common example of a situation where this architecture is used.

Two-tier client-server architecture with fat clients

Applications where application processing is provided by off-the-shelf software (e.g., Microsoft Excel) on the client. Applications where computationally intensive processing of data (e.g., data visualization) is required. Mobile applications where internet connectivity cannot be guaranteed. Some local processing using cached information from the database is therefore possible. Large-scale applications with hundreds or thousands of clients. Applications where both the data and the application are volatile. Applications where data from multiple sources are integrated.

Multi-tier clientserver architecture

63

64

There is no distinction in a distributed component architectures between clients and servers Each distributable entity is an object that provides services to other components and receives services from other components Component communication is through a middleware system However, distributed component architectures are more complex to design than C/S systems

Comp1 Common services

Comp2 Common services

Comp3 Common services

Comp4 Common services

Communication Middleware

Client

Client

Client

Client

Client

65

66

Database1
Integrator1

Report gen.

Distributed component architectures suffer from two major disadvantages:

They are more complex to design than clientserver systems. Distributed component architectures are difficult for people to visualize and understand Standardized middleware for distributed component systems has never been accepted by the community. Different vendors, such as Microsoft and Sun, have developed different, incompatible middleware

Database2

Visualizer

Integrator2

Database3
Display

As a result of these problems, service-oriented architectures are replacing distributed component architectures in many situations

67

68

Peer to peer (p2p) systems are decentralised systems where computations may be carried out by any node in the network The overall system is designed to take advantage of the computational power and storage of a large number of networked computers Most p2p systems have been personal systems but there is increasing business use of this technology

The logical network architecture

Decentralised architectures Semi-centralised architectures

Application architecture
The generic organisation of components making up a p2p application

Focus here on network architectures

69

70

Discovery server (Super peer)

n4 n6 n8 n14 n4 n7 n2 n3 n13 n9 n10 n11 n6 n5 n2 n1 n5 n12 n1 n3

71

72

Architecture Design
Faculty of Engineering, Bar-Ilan University

The design process for identifying the subsystems making up a system and the framework for sub-system control and communication is architectural design The output of this design process is a description of the software architecture

Slides are based on slides by Ian Sommerville, by Mira Balaban and by Peter Bunus

An early stage of the system design process Represents the link between specification and design processes Often carried out in parallel with some specification activities It involves identifying major system components and their communications

Vision system

Object identification system

Arm Controller

Gripper controller

Packaging selection system

Packaging system

Conveyor controller
4

Architecture in the small is concerned with the architecture of individual programs. At this level, we are concerned with the way that an individual program is decomposed into components Architecture in the large is concerned with the architecture of complex enterprise systems that include other systems, programs, and program components. These enterprise systems are distributed over different computers, which may be owned and managed by different companies

Stakeholder communication
Architecture may be used as a focus of discussion by system stakeholders

System analysis
Means that analysis of whether the system can meet its non-functional requirements is possible

Large-scale reuse
The architecture may be reusable across a range of systems Product-line architectures may be developed

Simple, informal block diagrams showing entities and relationships are the most frequently used method for documenting software architectures But these have been criticised because they lack semantics, do not show the types of relationships between entities nor the visible properties of entities in the architecture Depends on the use of architectural models. The requirements for model semantics depends on how the models are used However, useful for communication with stakeholders and for project planning

As a way of facilitating discussion about the system design

A high-level architectural view of a system is useful for communication with system stakeholders and project planning because it is not cluttered with detail. Stakeholders can relate to it and understand an abstract view of the system. They can then discuss the system as a whole without being confused by detail

As a way of documenting an architecture that has been designed

The aim here is to produce a complete system model that shows the different components in a system, their interfaces and their connections

Architectural design is a creative process so the process differs depending on the type of system being developed However, a number of common decisions span all design processes and these decisions affect the non-functional characteristics of the system

Is there a generic application architecture that can be used? How will the system be distributed? What architectural styles are appropriate? What approach will be used to structure the system? How will the system be decomposed into modules? What control strategy should be used? How will the architectural design be evaluated? How should the architecture be documented?

10

Systems in the same domain often have similar architectures that reflect domain concepts Application product lines are built around a core architecture with variants that satisfy particular customer requirements The architecture of a system may be designed around one of more architectural patterns or styles
These capture the essence of an architecture and can be instantiated in different ways Discussed later in this lecture

Performance
Localise critical operations and minimise communications. Use large rather than fine-grain components

Security
Use a layered architecture with critical assets in the inner layers

Safety
Localise safety-critical features in a small number of subsystems

Availability
Include redundant components and mechanisms for fault tolerance

Maintainability
Use fine-grain, replaceable components

11

12

What views or perspectives are useful when designing and documenting a systems architecture? What notations should be used for describing architectural models? Each architectural model only shows one view or perspective of the system
It might show how a system is decomposed into modules, how the run-time processes interact or the different ways in which system components are distributed across a network. For both design and documentation, you usually need to present multiple views of the software architecture

A logical view, which shows the key abstractions in the system as objects or object classes A process view, which shows how, at run-time, the system is composed of interacting processes A development view, which shows how the software is decomposed for development A physical view, which shows the system hardware and how software components are distributed across the processors in the system Related using use cases or scenarios (+1)

13

14

Patterns are a means of representing, sharing and reusing knowledge An architectural pattern is a stylized description of good design practice, which has been tried and tested in different environments Patterns should include information about when they are and when the are not useful Patterns may be represented using tabular and graphical descriptions
15

Name Description

MVC (Model-View-Controller) Separates presentation and interaction from the system data. The system is structured into three logical components that interact with each other. The Model component manages the system data and associated operations on that data. The View component defines and manages how the data is presented to the user. The Controller component manages user interaction (e.g., key presses, mouse clicks, etc.) and passes these interactions to the View and the Model. See Figure 6.3. Figure 6.4 shows the architecture of a web-based application system organized using the MVC pattern. Used when there are multiple ways to view and interact with data. Also used when the future requirements for interaction and presentation of data are unknown. Allows the data to change independently of its representation and vice versa. Supports presentation of the same data in different ways with changes made in one representation shown in all of them. Can involve additional code and code complexity when the data model and interactions are simple.

Example When used

Advantages

Disadvantages

16

Browser Controller
Maps user actions to model updates Selects view View selection

View
Renders model Requests model updates Sends user events to controller

Controller
HTTP request processing Application-Specific logic Data validation Update request

User events

Forms to display

View
Dynamic page generation Forms management

State change

Change notification

User events

State query

Model
Encapsulates application state Notifies view of state change

Change notification

Model
Business logic Database

Refresh request

17

18

Used to model the interfacing of sub-systems Organises the system into a set of layers (or abstract machines) each of which provide a set of services Supports the incremental development of subsystems in different layers. When a layer interface changes, only the adjacent layer is affected However, often artificial to structure systems in this way

Name Description

Layered architecture Organizes the system into layers with related functionality associated with each layer. A layer provides services to the layer above it so the lowest-level layers represent core services that are likely to be used throughout the system. See next slide. A layered model of a system for sharing copyright documents held in different libraries. Used when building new facilities on top of existing systems; when the development is spread across several teams with each team responsibility for a layer of functionality; when there is a requirement for multi-level security. Allows replacement of entire layers so long as the interface is maintained. Redundant facilities (e.g., authentication) can be provided in each layer to increase the dependability of the system. In practice, providing a clean separation between layers is often difficult and a high-level layer may have to interact directly with lower-level layers rather than through the layer immediately below it. Performance can be a problem because of multiple levels of interpretation of a service request as it is processed at each layer.
20

Example When used

Advantages

Disadvantages

19

User Interface LIBSYS login

Web Browser Interface

User Interface management Authentication and authorization Core business logic/application functionality System utilities

Forms and query manager

Print manager

Distributed Document search retrieval

Rights manager Accounting

System support (OS, database etc.) DB1 DB2

Library index DB3 DB4 DB5

21

22

Sub-systems must exchange data. This may be done in two ways:

Shared data is held in a central database or repository and may be accessed by all sub-systems Each sub-system maintains its own database and passes data explicitly to other sub-systems

Name Description

Repository All data in a system is managed in a central repository that is accessible to all system components. Components do not interact directly, only through the repository. Figure 6.9 is an example of an IDE where the components use a repository of system design information. Each software tool generates information which is then available for use by other tools. You should use this pattern when you have a system in which large volumes of information are generated that has to be stored for a long time. You may also use it in data-driven systems where the inclusion of data in the repository triggers an action or tool. Components can be independentthey do not need to know of the existence of other components. Changes made by one component can be propagated to all components. All data can be managed consistently (e.g., backups done at the same time) as it is all in one place. The repository is a single point of failure so problems in the repository affect the whole system. May be inefficiencies in organizing all communication through the repository. Distributing the repository across several computers may be difficult.
24

Example

When used

When large amounts of data are to be shared, the repository model of sharing is most commonly used a this is an efficient data sharing mechanism

Advantages

Disadvantages

23

UML editors

Code generators Java editor

Distributed system model which shows how data and processing is distributed across a range of components
Can be implemented on a single computer

Design translator

Project repository Python editor Design analyzer Report generator

Set of stand-alone servers which provide specific services such as printing, data management, etc Set of clients which call on these services Network which allows clients to access servers

25

26

Name Description

Client-server In a clientserver architecture, the functionality of the system is organized into services, with each service delivered from a separate server. Clients are users of these services and access servers to make use of them. Next slide is an example of a film and video/DVD library organized as a clientserver system. Used when data in a shared database has to be accessed from a range of locations. Because servers can be replicated, may also be used when the load on a system is variable. The principal advantage of this model is that servers can be distributed across a network. General functionality (e.g., a printing service) can be available to all clients and does not need to be implemented by all services. Each service is a single point of failure so susceptible to denial of service attacks or server failure. Performance may be unpredictable because it depends on the network as well as the system. May be management problems if servers are owned by different organizations.

Client 1

Client 2

Client 3

Client 4

Example When used

Internet

Advantages

Disadvantages

Catalog server Library catalogue

Video server Film store

Picture server Photo store

Web Server Film and photo info.

27

28

Functional transformations process their inputs to produce outputs May be referred to as a pipe and filter model (as in UNIX shell) Variants of this approach are very common. When transformations are sequential, this is a batch sequential model which is extensively used in data processing systems Not really suitable for interactive systems

Name Description

Pipe and filter The processing of the data in a system is organized so that each processing component (filter) is discrete and carries out one type of data transformation. The data flows (as in a pipe) from one component to another for processing. Next slide is an example of a pipe and filter system used for processing invoices. Commonly used in data processing applications (both batch- and transaction-based) where inputs are processed in separate stages to generate related outputs. Easy to understand and supports transformation reuse. Workflow style matches the structure of many business processes. Evolution by adding transformations is straightforward. Can be implemented as either a sequential or concurrent system. The format for data transfer has to be agreed upon between communicating transformations. Each transformation must parse its input and unparse its output to the agreed form. This increases system overhead and may mean that it is impossible to reuse functional transformations that use incompatible data structures.

Example When used

Advantages

Disadvantages

29

30

Issue receipts Read issued invoices Identify payments Find payments due Invoices Payments

Receipts

Issue payments reminder

Reminders

A lexical analyzer, which takes input language tokens and converts them to an internal form A symbol table, which holds information about the names of entities (variables, class names, object names, etc.) used in the text that is being translated A syntax analyzer, which checks the syntax of the language being translated A syntax tree, which is an internal structure representing the program being compiled
31 32

A semantic analyzer that uses information from the syntax tree and the symbol table to check the semantic correctness of the input language text A code generator that walks the syntax tree and generates abstract machine code

Symbol table Syntax tree

Lexical analysis

Syntactic analysis

Semantic analysis

Code generation

33

34

Lexical analyzer

Syntax analyzer

Semantic analyzer

Pretty printer

Abstract syntax tree

Grammar definition

Optimizer

Editor

Symbol table

Output definition

Code generator

Repository

35