Journal of Earthquake Engineering, Vol. 9, No.

5 (2005) 741753 c Imperial College Press

TECHNICAL NOTE A FRAMEWORK FOR MULTI-SITE DISTRIBUTED SIMULATION AND APPLICATION TO COMPLEX STRUCTURAL SYSTEMS

OH-SUNG KWON and NARUTOSHI NAKATA Graduate Research Assistant, University of Illinois 205 North Mathews, Urbana, Illinois 61801, USA okwon2@uiuc.edu nakata@uiuc.edu AMR ELNASHAI Willett Professor, Director, Mid-America Earthquake Center University of Illinois, 205 North Mathews Urbana, Illinois 61801, USA aelnash@uiuc.edu BILL SPENCER Nathan M. and Anne M. Newmark Endowed Chair University of Illinois, 205 North Mathews Urbana, Illinois 61801, USA bfs@uiuc.edu Received 8 November 2004 Reviewed 29 November 2004 Accepted 10 December 2004 In this technical note, the development of a framework for multi-site distributed simulations is presented. The algorithm is suitable for any combination of physical (laboratory) and analytical (computer) distributed simulations of structures, their foundations and the underlying sub-strata subjected to static and dynamic loading. Two examples of multi-site testing and multi-platform simulation are given. The main contribution in this note is the separation between time-step integration and stiness formulation, which enables the use of static analysis and testing as modules of the main control module referred to as the simulation coordinator. The approach proposed is intuitive, simple and ecient. It is therefore recommended for use in distributed analysis using dierent programs, distributed testing facilities (e.g. the NEES equipment sites) or a combination of analysis and testing. Keywords: Pseudo-dynamic; substructure; simulation; distributed test; NEES.

1. Introduction The most realistic assessment method for coupled structural capacity-seismic demand is the dynamic testing of the full-scale structure, its foundations and the underlying soil when subjected to ground motion representative of the expected site hazard. Many laboratories are equipped with shaking tables (earthquake simulators) for this purpose. However, most tables are too small to
741

742

O.-S. Kwon et al.

test even full-scale models of a single storey structure. The largest shaking table in the world, recently completed in Miki City, Japan, is 15 m 20 m (http://www.bosai.go.jp/sougou/sanjigen/3De/index.htm). Notwithstanding that the Miki table has a capacity to test a full-scale 6-storey building, it is extremely costly to run and does not provide a test environment for bridges and other horizontally extended structures. Within the NEES equipment sites in the USA, there are movable and recongurable shaking tables (e.g. at the University of Nevada, Reno, and the State University of New York, Bualo). They, however, have limited load-carrying capacity, hence are not suitable for testing large piers of bridges with long spans. The distance between the tables is also limited, thus restricting viable tests of small-scale or very short-span bridges. As an alternative to real-time shake table tests, pseudo-dynamic (PSD) tests can be used in which the inertial and damping forces are analytically computed and the corresponding displacements are applied to the structure. The concept of PSD testing was developed in the mid-1970s in Japan [Takanashi et al., 1975] and has since been adopted in many experiments. In Europe, the rst Pseudo-Dynamic test setup was developed and veried at Imperial College [Elnashai et al., 1990] in collaboration with Japanese researchers. Considerable work and impressive developments were undertaken by the Earthquake Engineering Research Center, University of California, Berkeley [Mahin and Shing, 1985] and at the Japan Building Research Institute [Nakashima et al., 1987]. Recently, a full-scale, three-dimensional, irregular RC frame was tested pseudo-dynamically in Ispra, Italy [Jeong and Elnashai, 2004]. Pseudo-dynamic testing is applicable to large-scale and extended structures, when combined with sub-structuring. As versatile as the approach is, it suers from the capacity limitations of the experimental site, and the nite capabilities of analysis software. Experimental sites have varying and often complementary features. For example, extensive and large-scale structural facilities exist at the University of Illinois at Urbana-Champaign (UIUC), within the NEES community, while Rensselaer Polytechnic Institute (RPI) has a high speed geotechnical centrifuge which has an internal shaking table. A parallel situation exists in analysis software. Dierent analysis platforms exhibit dierent strengths and weaknesses. For example, ZeusNL, a product of MAE centre [Elnashai et al., 2002], is capable of stable collapse analysis of space frames, but does not have plate, shell and solid elements. ABAQUS is a proven commercial code in which there are numerous elements, but has limitations on reinforced concrete analysis. OpenSees, the PEER Center analysis platform, includes geotechnical constitutive models [Yang, Elgamal and Parra, 2003] not available in most other widely available codes. It is concluded from the above discussion that establishing a framework for multi-site testing, multi-code analysis and a mixture of both would avail of tools for investigative assessment that are hitherto not available. In this note, a framework for multi-site substructure pseudo-dynamic (PSD) test and simulation is developed that builds on the framework for the MOST experiment

Framework for Multi-Site Distributed Simulation

743

[Spencer et al., 2004]. The development of this framework requires the following three key components: Integration scheme for PSD tests, Communication among sub-structured components, Sub-structuring (sub-division) of the complex system. In subsequent sections, the integration scheme for PSD test is introduced and the framework of the distributed simulation is described. Through the developed framework, referred to as UI-SIMCOR, many testing and analysis research facilities may be linked through NEESgrid to undertake assessment of complex structuralgeotechnical systems, while optimising the use of the most powerful features of each site and analysis platform.

2. Integration Scheme for PSD Test The literature abounds with PSD test algorithms. The implicit time-step integration scheme, in which initial stiness is required, has been used for PSD test in two main thrusts; an iterative implicit method achieving equilibrium at each time step through sub-cycling [Shing et al., 1990, 1991], or linearly implicit and non-linearly explicit, operator splitting (OS) method [Nakashima et al., 1987]. Ghaboussi et al. [2004] developed a Predictor-Corrector (PC) algorithm which gives better result compared to the -OS scheme when the response ventures deeply into the inelastic range. The structure of UI-SIMCOR is independent of the integration scheme, hence any feasible scheme may be appended to the developed software. Currently, Operator Splitting method in conjunction with -modied Newmark scheme (-OS method) is implemented as one of the PSD test algorithm. The new PC algorithm [Ghaboussi et al., 2004] is under implementation. The accuracy and stability of the -OS method was thoroughly studied by Combescure and Pegon [1997], and their main observations are summarised below. When the initial stiness matrix is higher or equal to the instantaneous tangent stiness, the scheme is unconditionally stable. The scheme behaves accurately for the frequency band of interest by an adequate choice of the time step. When the structure under consideration does not lose too large part of its initial stiness during the test, the non-iterative (incremental) -OS is a good alternative to the more complex iterative schemes. When the structure undergoes severe stiness degradation, the scheme is still accurate and stable for the low frequency modes of the structure, which are usually dominant.

744

O.-S. Kwon et al.

The equation of motion of a structure may be expressed in terms of the second order partial dierential equation below: Ma(t) + Cv(t) + r(t) = f(t), (1)

where M and C are mass and stiness matrices, r(t) is a restoring force vector, and f (t) is an applied force vector. A state of a structure, i.e. displacement, velocity, and acceleration, at the n + 1th step consists of known terms from the nth step and unknown terms which need to be determined. In the PSD test using -OS method, the known terms, or predicted terms, are applied to a structure, and the measured restoring forces from the structure are used to evaluate the unknown terms. The PSD test procedure using -OS scheme can be summarised as below [Combescure and Pegon, 1997]: (1) Choose t Evaluate M M = M + t(1 + )C + t2 (1 + )KI (2) Set n = 0 r Initialise d0 , d0 = d0 , v0 , a0 , 0 and f0 (3) Input excitation fn+1 (4) Compute dn+1 and vn+1 2 dn+1 = dn + tvn + t (1 2)an
2

vn+1 = vn + t(1 )an (5) Impose dn+1 to the structure (6) Measure restoring force, m , and displacement, dm rn+1 n+1 I m m (7) Compute n+1 = r r K d dn+1
n+1 n+1

(8) Compute n+1 = (1 + )fn+1 fn + n (1 + )n+1 + C n r r v f (1 + )C n+1 + (tC + t2 KI )an v (9) Solve for an+1 from Man+1 = n+1 f (10) Compute dn+1 and vn+1 dn+1 = dn+1 + t2 an+1 vn+1 = vn+1 + tan+1 (11) Set n = n + 1 and go to step 3 In the above procedure the initial stiness matrix, KI , is used to correct the restoring force in step 7 and to establish equivalent mass matrix in step 1, which is used to calculate the acceleration of the next step, an+1 in step 9. Thus, to use the -OS method for PSD tests, the initial stiness matrix should be established prior

Framework for Multi-Site Distributed Simulation

745

to dynamic analysis. The stiness matrix may be formed from simple pre-tests or pre-analysis. In this study, the -OS integration scheme is implemented in the main body of the software, UI-SIMCOR, which controls each module and performs integration, and all the static analysis or experiment is conducted in the separate module. This structure enables attaching a new module, be it a test specimen or an analysis program, in a manner which is exceptionally simple.

3. Communication Among Sub-Structured Components In the PSD test, both the experimental and computational modules are deformation-controlled. The main integration unit, UI-SIMCOR, sends commands to each module to apply deformations and to collect resulting actions. Because the substructure PSD test or substructure PSD simulation involves more than one module, it is essential to establish a network for communication between UI-SIMCOR and each module. Earlier version of this development utilised the TCP-IP protocol using Winsock library to send command and data. Recently, the NEESgrid project (http://it.nees.org) has developed a protocol termed NEESgrid Teleoperation Control Protocol [NTCP, Pearlman et al., 2004], for the purpose of tele-operation over the network on NEESgrid infrastructure. NTCP provides secured communications between modules, which is an important feature especially for multi-site hybrid testing. Through the NTCP server, UI-SIMCOR communicates data with each module as shown in Fig. 1. Because of the exibility of UI-SIMCOR, any type and number of computational software and testing equipments can be integrated into the simulation. The TCP-IP version of UI-SIMCOR with manual is available for download from the following link: http://mae.ce.uiuc.edu. The NTCP version can be downloaded from the site: http://it.nees.org/software.

UI-SIMCOR w/ -OS integration scheme NEESpop

NTCP Server Matlab Plugin

NTCP Server LabView Plugin

NTCP Server LabView Plugin

NTCP Server LabView Plugin

FedeasMDL FEDEAS Lab

NEES-SAM ZEUS-NL

NEES-SAM OpenSees

LabView Equipment Experimental Module

Static Analysis Modules (SAM)

Fig. 1.

Networking conguration of UI-SIMCOR.

746

O.-S. Kwon et al.

4. Sub-Structuring of the Complex System From the result of simulation with large number of degrees of freedom (DOF) in a complex system, only those DOF corresponding to applied actions or points of interest are needed. Other DOF may be temporarily removed using static condensation methods. When a structure is subjected to base excitation, only the nodes where the masses are dened are subjected to inertial forces, i.e. the right hand side term of Eq. (1). The equation may be rewritten as follows: d Mi 0 0 li ai Mi 0 0 i Kii Kij Kik 0 0 0 aj + Kji Kjj Kjk dj = 0 0 0 lj Ag , Kki Kkj Kkk 0 0 0 0 0 0 ak dk lk (2) where the indices i, j, and k represent DOF where masses are dened, DOF of interest, and DOF that can be condensed out, respectively. The damping matrix is generally assumed to be a linear combination of mass and stiness matrices (Rayleigh damping). In the following matrix manipulation, the damping term is omitted to simplify the derivation. This is not necessary, but is justied on the basis of the dominance of hysteretic damping in most inelastic response applications. Equation (2) can be rewritten after rearranging its terms such that its right hand side can be represented as a force vector with only i DOF, since all other components of the mass matrix except i DOF are zero. di Fi Kii Kij Kik (3) Kji Kjj Kjk dj = 0 , Kki Kkj Kkk dk 0 (4) where Fi = (Mi li + Mi ai ). Static condensation leads to the elimination of k DOF in the equation: Kii Kji = Kij Kjj K ii K ji K ij K jj K ii K ji Kik Kjk di dj [Kkk ]1 [Kki Kkj ] = Fi 0 , di dj (5)

where * indicates the condensed matrices. After rearranging the terms, Mi 0 0 0 ai aj + K ij K jj di dj = Mi 0 0 0 li lj Ag , (6) (7)

or M a + K d = M l Ag .

Framework for Multi-Site Distributed Simulation

747

Equation (7), in which redundant DOF are condensed out, is equivalent to Eq. (1). When a structure is tested pseudo-dynamically, it is impossible to control all DOF. For instance, for a PSD test of a 4-storey structure, usually only one or two actuators at each oor level are used to simulate inertial forces. Thus, all other DOF are condensed out of the equations of motion as long as the displacements of the DOF are not required immediately. When dynamic analysis is conducted for seismic evaluation of framed structures, the translational acceleration is usually applied at support points. In most cases the masses are assumed to be lumped at beam-column joints. Thus if the displacements of all DOF in the frame are not required, the DOF without lumped masses can also be condensed out. The condensed stiness matrix may be determined from Eq. (5) if all stiness quantities are known a priori. Retrieving full stiness matrices from various analysis applications, however, is dicult and requires modication of source code in most cases. As an alternative, it is possible to perform the preliminary test either using static analysis modules or directly from the experiment (i.e. applying known deformations corresponding to each DOF, measuring the reactions and extracting the associated stiness matrix. As an example, consider a column with 5 nodes and 30 DOF, as shown in Fig. 2(a). If the x-directional translation at the top node of the column in Fig. 2(a) is of interest, the lateral stiness can be calculated by static condensation when the full 30 30 stiness matrix is known priori. If the matrix cannot be retrieved but a static test can be performed, the condensed stiness is easily determined by applying unit deformations at the top and by measuring the corresponding actions, as shown in Fig. 2(b). The condensed mass and stiness matrices in Eq. (6) are linear summations of the contribution from various structural elements. Thus if the structure is
d

y z x

(a) System with 30 DOF Fig. 2.

(b) Condensed system

Static condensation of simple system.

748

O.-S. Kwon et al.

subdivided into m modules, the mass and stiness matrices are determined as follows:
m

Mi =
l=1

Mi,l , K ii K ji K ij K jj
m

(8) K ii,l K ji,l K ij,l K jj,l ,

K =

=
l=1

(9)

where the subscript l is used to dene module number. The main module, as mentioned before, is responsible for time-step integration, while the other modules perform static analyses and/or experiments in separate m modules. As long as an analysis platform or an experimental site is able to return actions corresponding to the imposed deformations, it is eligible for participation as a part of the PSD test. Theoretically, the structure may be subdivided into as many modules as necessary, and each module may have an arbitrary number of DOF. In UI-SIMCOR, only the mass matrix in Eq. (6) is dened when the PSD test starts. The condensed stiness of a structure from either an analytical package or a physical test is evaluated from the pre-tests. The owchart of the framework implementation is given in Fig. 3.

Start Start SimConfig.m Load Configuration InitializeVariables.m Initialize Variables: M, C, K, F, etc. InitializeNetwork.m Establish connection to static module through NTCP Server InitializeModules.m Initialize Each Static Module
open()

Initialization

For i=1:Num_Static_Step IncrementStat.m Determine target displacements for initial loading

Static loading stage

propose()

ProposeTgtDisp.m
query()

NTCP Server
Curr.Data execute() Meas.Data

CheckRelaxation.m NTCP Server ExecutePSDT.m TriggerDAQ.m

NTCP Server NTCP Server

set()

NTCP Server

CheckTolerance.m I_Modification.m Modify measured force Write data

For i=1:Num_DOF IncrementInit.m Determine target displacements for initial stiffness calculation ProposeTgtDisp.m Send target displacement to each module CheckRelaxation.m Relaxation check ExecutePSDT.m Relaxation check TriggerDAQ.m Trigger data aquisition CheckTolerance.m Check Tolerance Stiffness formulation and write data Store Stiffness Matrix
propose()

Initial stiffness formulation

For i=1:Num_Dynamic_Step NTCP Server

query() Curr.Data execute() Meas.Data

Dynamic loading stage

IncrementDyna.m Determine predictor displacements for dynamic loading

propose()

NTCP Server

ProposeTgtDisp.m
query()

NTCP Server
Curr.Data execute() Meas.Data

CheckRelaxation.m NTCP Server ExecutePSDT.m TriggerDAQ.m CheckTolerance.m I_Modification.m CorrectDisp.m Calculate corrected displacement Write data

NTCP Server NTCP Server

END END

Fig. 3.

Flowchart of substructure PSD test.

Framework for Multi-Site Distributed Simulation

749

By completely separating the integration part from stiness calculations, it is possible to use arbitrary combinations of various analytical and/or experimental modules with any number of control points. As simple as the approach of separating the time-step integration from the stiness formulation is, it enables overcoming major problems in multi-site and multi-platform testing and analysis, respectively, and their combination in a hybrid analytical-experimental simulation, as described below.

5. Application Examples The developed software was subjected to several verication studies. As an example of combining analysis and experiment in one simulation, the Multi-Site SoilStructure-Foundation Interaction Test (MISST) is used. MISST is intended to provide a realistic test bed application with which to verify and extend all components of the NEESgrid project as well as all components of the sites taking part in the distributed simulation. Part of the under-crossing of the I-10 (Santa Monica Freeway) that was damaged in the 17 January 1994 Northridge earthquake is idealised in MISST. At an early stage of the MISST project, the bridge was subdivided into three modules; (i) left pier, (ii) deck and middle pier, and (iii) right pier as shown in Fig. 4(a), (b) and (c), respectively. For the purposes of testing the developed framework, the left pier, Fig. 4(a) was tested using physical experiment. At a later stage, two piers and soil foundation will be tested at three NEES sites and the deck and middle column will be simulated. The result from hybrid substructure PSD simulation is compared with an analysis result where the whole bridge was modeled as a single structure. Figure 5 shows the lateral displacement of deck at the left abutment. The responses from the two alternatives are identical. Such results lend credence to the UI-SIMCOR formulation.

Fig. 4.

MISST project test conguration.

750

O.-S. Kwon et al.

40

Abutment displacement (mm)

30 20 10 0 -10 0 -20 -30 -40 Time (sec) Single model analysis Hybrid substructure PSD test 2 4 6 8 10 12 14 16 18 20

Fig. 5.

Comparison of experiment and simulation MISST.

Fig. 6.

Soil-foundation-structure model.

The second verication example is intended to demonstrate the use of dierent analytical platforms utilising the best features of each. Figure 6 shows the conguration of an idealised soil-foundation-structure model where the frame members represent a cantilever structure, such as an elevated water tank and its footing, resting on soil and embedded in it. For substructure PSD simulation, the soil was modeled using plane strain elements in OpenSees and the frame members were modelled using Zeus-NL. The results from substructure PSD simulation are compared with the results from analysis where the whole soil-frame system was modelled in

Framework for Multi-Site Distributed Simulation

0.15 0.10

751

Top displacement, m

0.05 0.00 0.0 -0.05 Whole model(OpenSees) -0.10 Soil(OpenSees)+Column(ZeusNL) -0.15 Time, sec 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

Fig. 7. Comparison of single model analysis and multi-platform simulation Soil-foundationstructure model.

OpenSees. Figure 7 shows the response at the column top in which identical values were observed from the OpenSees single model, substructure, and substructure PSD test using OpenSees for the soil part and Zeus-NL for the frame component.

6. Conclusion A framework for multi-site substructure test and simulation is introduced in this technical note. The main contribution is the simple concept of separating timestep integration and stiness matrix assembly. Employment of this concept leads to a much simplied and versatile implementation compared to other simulation coordinators options. Using the separation of time-step integration and stiness formulations enables running distributed dynamic simulations using static analysis software platforms and static tests, whilst including inertia forces in the integrator part of the simulator. The approach is not only ideally suitable to distributed experimental-analytical simulations, but is also valuable for combining more than one analysis platform, to optimise the use of the best models and member formulations regardless of to which platform they belong. Two verication examples have been given to demonstrate applications in hybrid (mixed experimental-analytical) simulation and the use of more than one analysis platform to model multi-material assemblies. Even with distributed hybrid simulation, large portion of a structure needs to be modeled analytically where the test result may depend on the numerical model performance. Moreover, distributed mass systems (such as earth dams) still require dynamic testing since they cannot be discritised for hybrid simulations. These limitations, however, exist in all other methods where analytical models and lumped representations are used. The accuracy and stability of the hybrid PSD algorithms have not been investigated; these awaits future studies.

752

O.-S. Kwon et al.

Acknowledgment This research was supported by Mid-America Earthquake Center project DS-3, the NEES project at UIUC and NEESgrid. The Mid-America Earthquake Center is an Engineering Research Center funded by the National Science Foundation under cooperative agreement reference EEC-9701785. NEES at UIUC is funded by the George E. Brown, Jr. Network for Earthquake Engineering Simulation (NEES) program of the National Science Foundation under award reference CMS-0217325 and NEESgird is funded by the same NSF Program, award reference CMS-0117853.

Appendix Notations , and t a(t) f (t) dn+1 dm

n+1

r(t) m rn+1 n i, j, and k v(t) C K KI M M M

Integration scheme parameters Analysis time step Acceleration vector External force as a function of time Predicted displacement Measured displacement Restoring force vector Measured force Analysis step number Index for DOF with mass, DOF of interest and DOF that can be condensed out Velocity vector Stiness matrix Condensed stiness matrix Initial stiness matrix Mass matrix Equivalent mass matrix Condensed mass matrix

References
Combescure, D. and Pegon, P. [1997] -operator splitting time integration technique for pseudodynamic testing. Error propagation analysis, Soil Dynamics and Earthquake Engineering 16, 427443. Elnashai, A. S., Elghazouli, A. Y. and Dowling, P. J. [1990] Verication of pseudodynamic testing of steel members, Journal of Constructional Steel Research 16, 153161. Elnashai, A. S., Papanikolaou, V. and Lee, D. [2002] Zeus NL A System for Inelastic Analysis of Structures, Mid-America Earthquake Center, University of Illinois at Urbana-Champaign, Program Release September 2002. Ghaboussi, J., Yun, G. J. and Hashash, Y. M. A. [2004] A novel predictor-corrector algorithm for substructure pseudo dynamic testing, Earthquake Engineering and Structural Dynamics (Submitted).

Framework for Multi-Site Distributed Simulation

753

Jeong, S. H. and Elnashai, A. S. [2004] Analytical and experimental seismic assessment of irregular RC buildings, Proc. 13th World Conf. on Earthquake Eng., Vancouver, B.C., Canada August 16, 2004, No. 113. Nakashima, M. and Kato, H. [1987] Experimental error growth behavior and error growth controling on-line computer test control method, Building Research Institute, BRI-Report No. 123, Ministry of Construction, Tsukuba, Japan, 1987. Pearlman, L., DArcy, M., Johnson, E., Kesselman, C. and Plaszczak, P. [2004] NEESgrid Teleoperation Control Protocol (NTCP), Technical Report NEESgrid-2004-23, http://it.nees.org/ Spencer, B. F., Elnashai, A., Nakata, N., Seliem, H., Yang, G., Futrelle, J., Glick, W., Marcusiu, D., Ricker, K., Finholt, T., Horn, D., Hubbard, P., Keahey, K., Liming, L., Zaluzec, N., Pearlman, L. and Stauer, E. [2004] The MOST experiment: Earthquake engineering on the grid, Technical Report NEESgrid-2004-41, http://it.nees.org/ Mahin, S. A. and Shing, P. B. [1985] Pseudodynamic method for seismic testing, Journal of Structural Engineering-ASCE 111(7), 14821503. Shing, B. and Manivannan, T. [1990] On the accuracy of an implicit algorithm for pseudodynamic tests, Earthquake Eng. Struct. Dyn. 19, 631651. Shing, B., Vannan, M. T. and Cater, E. [1991] Implicit time integration for pseudodynamic tests, Earthquake Eng. Struct. Dyn. 20, 551576. Takanashi, K., Udagawa, K., Seki, M., Okada, T. and Tanaka, H. [1975] Nonlinear earthquake response analysis of structures by a computer-actuator on-line system, Bull. of Earthquake Resistant Structure Research Centre, No. 8, Institute of Industrial Science, University of Tokyo, Japan, 1975. Yang, Z., Elgamal, A. and Parra, E. [2003] Computational model for cyclic mobility and associated shear deformation, Jour. of Geotechnical and Geoenvironmental Eng. 129(12), 11191127.