An imaging data model for concrete bridge inspection

Osama Abudayyeh
a,
*
, Mohammed Al Bataineh
b
, Ikhlas Abdel-Qader
c
a
Department of Civil and Construction Engineering, Western Michigan University, Parkview Campus, Kalamazoo, MI 49008-5316, USA
b
Department of Construction Engineering and Management, Western Michigan University, Kalamazoo, MI 49008, USA
c
Department of Electrical and Computer Engineering, Western Michigan University, Kalamazoo, MI 49008, USA
Received 22 January 2004; revised 3 June 2004; accepted 18 June 2004
Abstract
Bridge management systems were developed to address the data organization and decision making aspects of bridge inspection and
maintenance. However, these systems still neglect the automation aspects of bridge monitoring and inspection. Automation may result in
monetary savings and can optimize the inspection process. This paper discusses the data model that was developed to support automated
imaging inspection of concrete bridges. The paper discusses the framework for an automated bridge inspection methodology and provides a
detailed discussion of the data modeling efforts involved in the development of the imaging information model.
q 2004 Elsevier Ltd. All rights reserved.
Keywords: Bridge inspection; Bridge maintenance; Database management; Information management; Information systems; Image techniques; Images
1. Introduction
The road network in the United State consists of 3.8
million miles, and 565 thousand bridges. More than 70% of
these bridges were built prior to 1935 [1]. Further,
transportation-based industries account for 20% of the
gross national product and approximately 800,000 jobs [2].
A recent Federal Highway Administration (FHWA) study
indicated that one third of existing bridges in the United
States are structurally decient, 10% of pavements require
immediate repair, and 60% of pavements need rehabilitation
[2,3]. Moreover, FHWA estimates that an annual $50 billion
would be necessary to maintain the roads in present
condition and $215 billion would be needed to rehabilitate
all decient roads and bridges. Therefore, a good transpor-
tation infrastructure management system is key to the
maintenance of the transportation system. Hence, providing
a high level of serviceability through periodic inspection
and maintenance is important to keeping the transportation
system operational in order to avoid major replacement
efforts.
However, for a number of years, many departments of
transportation (DOT) spent most of their planning and
budgeting efforts and most of their allocated monies on new
construction, while maintenance and rehabilitation gener-
ally were managed with less formal methods [4]. In several
cases, crises or disasters were the driving force behind
initiating maintenance and/or rehabilitation actions, par-
ticularly when funds are limited. Such an approach,
however, was no longer sufcient nor appropriate, particu-
larly when most of the transportation infrastructure had
reached its design service life. As a result, pavement and
bridge management systems (BMS) evolved to help plan
maintenance and rehabilitation of pavements and bridges,
and to avoid the crisis reaction approach to maintaining the
transportation infrastructure [46]. Examples of National
efforts include the development of Pontis (Preservation,
Optimization and NeTwork Information System), BRID-
GIT (BRIDGe Information Technology), and PENBMS
(PENnsylvania Bridge Management System) [7]. Pontis has
subsequently become the National standard and has been
adopted by most departments of transportation [8]. The
focus of the research project described in this paper is on the
automation of bridge inspection in support of the mainten-
ance process. The project integrates the automated system
with Pontis BMS.
0965-9978/$ - see front matter q 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.advengsoft.2004.06.010
Advances in Engineering Software 35 (2004) 473480
www.elsevier.com/locate/advengsoft
* Corresponding author. Tel.: C269-276-3252; fax: C269-276-3257.
E-mail address: abudayyeh@wmich.edu (O. Abudayyeh).
2. Bridge inspection issues and needs
Bridge condition data is a major component of bridge
management [4,9,10]. Such data are periodically collected
and analyzed to determine the optimum allocation of funds
among new construction, maintenance, and rehabilitation
programs. Since more time and funds are now being spent
on maintenance and rehabilitation of the existing system,
then, there is a pressing need to develop effective BMSs that
enhance the method of collecting, organizing, and using
data for proper planning and optimization of the mainten-
ance process. Particularly, there is a need for developing
automated bridge monitoring and inspection methods and
systems.
Bridge monitoring and inspection are expensive, yet they
are essential tasks in maintaining a safe infrastructure. The
primary method used to monitor bridges is visual inspection.
During a typical bridge inspection, the various components
of a bridge are examined at close range by trained inspectors,
who evaluate the condition of the components and give them
a condition rating. This rating is a qualitative evaluation of
the current condition based on a set of guidelines and on the
inspectors experience. For many situations, this type of
evaluation is appropriate and effective. However, due to the
subjective nature of this evaluation, ratings of the conditions
of similar bridge components can vary widely frominspector
to inspector, and from state to state. Moreover, inspections
are not necessarily always performed at the appropriate or
critical times.
There are several factors that can contribute to the
selection of inspection procedures and may alter the timing
of inspections [11,12]. These factors include the condition
of a bridge, age, size and complexity; trafc density;
impacts of trafc disruption; availability of personnel and
equipment; environmental conditions; geographic location;
and, construction methods. Any one of these factors can
affect the deterioration rate of the bridge and the need for
maintenance, rehabilitation, or reconstruction. Therefore,
transportation agencies must develop inspection strategies
that address the frequency of inspection, the nature of
observations, and the equipment for measurements.
Many evaluation methods, known as non-destructive
evaluation, have been designed to operate upon existing
bridges without damaging their usability. Some of these
methods may be very broad and versatile, and can be used in
a number of applications, while others are very specialized.
More recently, bridge inspection has included remote
monitoring devices that record stress and vibration in real-
time in order to more accurately determine the wear on a
particular bridge and aid in early detection of problems.
Remote monitoring used in conjunction with automated
imaging techniques may signicantly reduce the costs
associated with bridge inspection and maintenance. Cost
savings are achieved through: (1) the reduction or
optimization of the number of trips a bridge inspector
must make over the lifetime of a bridge by depending on
data from the more frequent automated inspection tech-
niques, and (2) serving as an early warning system prior to a
signicant bridge deterioration. Furthermore, systematic
and timely inspection in this case will certainly extend the
service life of the bridge and will reduce or eliminate any
hazardous situations to the public.
3. A framework for an automated bridge imaging
inspection system
The authors contend that there is a need for an automated
bridge inspection methodology that takes advantage of
imaging methods and technologies. A number of imaging
systems have been developed for pavement condition
assessment [13,14]. However, these systems are not suitable
for bridge inspection since they typically use van-mounted
cameras that are moving over the pavement at highway
speeds. Bridge inspection requires xed or slow-moving
cameras that can capture images of the deck as well as other
bridge elements such as girders. The proposed framework
for the bridge imaging inspection methodology is shown in
Fig. 1. The system framework consists of three major
components: data acquisition, data analysis and interpret-
ation, and an information model (data storage and proces-
sing). The following paragraphs briey describe each
component.
Data acquisition consists of the methods and technol-
ogies used in capturing and transmitting bridge element
images to the data analysis and interpretation compo-
nent for condition assessment. The methods and
technologies must provide continuous monitoring
Fig. 1. Proposed imaging inspection information system for concrete
bridges.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 474
capabilities and eliminate the need for an inspector to visit
the bridge site. Therefore, the selection of a camera
technology and location becomes a critical factor in
achieving quality condition data and in increasing the
safety of the inspection process. Cameras can be
permanently mounted at key locations on the bridge and
connected to a central database through cables, or they
may use wireless transmission techniques to send the
acquired images. An operator should have the capability
of remotely controlling the functions of the camera
(e.g. zooming and panning). During real-time image
acquisition, a number of factors must be considered such
as weather condition, time of day, shadows, and location
of camera. This component, however, is not the focus of
this paper and, therefore, no further discussion on real-
time image acquisition issues is provided.
Data analysis and interpretation consists of the image
processing algorithms used to reason about the infor-
mation contained in the digital images that are captured
and stored in the central database. For instance, one
technique data interpretation is the crack-detection
algorithm(s) that isolate the cracks from the rest of the
scene in the image to produce what is called a crack
image [15]. After the crack images are produced,
additional image processing techniques can be developed
and used to reason about the attributes of these cracks
such as type, width, depth, and length to provide a
condition assessment of the bridge element under
investigation. Further analysis and automated techniques
can also be used to identify other types of surface defects.
This paper, however, does not provide any further
discussions on this component.
The information model is the main focus of this paper.
It consists of a relational data model in support of the
automated imaging inspection methodology. The infor-
mation model is designed to be integrated with Pontis
through the Pontis bridge element denition, as shown in
Fig. 1. The remainder of the paper describes the
modeling and development efforts undertaken in the
design of the imaging information model. At this time,
this model stores crack and other surface defect images,
and relies on the inspector for condition assessment
analysis rather than on an image processing algorithm.
4. Modeling methodology
The imaging inspection information model was devel-
oped using a four-step modeling methodology shown in
Fig. 2 and described as follows:
The rst step in the modeling effort, problem denition,
involves identifying the data items needed and des-
cribing the behavior of the information model with
respect to the methods and mechanisms used in acquiring
and processing data. In this step data items are identied
by: (1) analyzing the bridge inspection process and
forms, (2) a thorough study of Pontis, and (3) a literature
review relevant to bridge inspection.
The second step, conceptual modeling, is the graphical
representation of the imaging information model dened
in the rst step. The outcome of this step is an Entity-
Relationship (E-R) conceptual data model that represents
the design of the information system.
The third step, computational modeling, transforms the
E-R data model developed in step two to a relational
database schema suitable for implementation in an
automated environment. The relational schema is
optimized to the third normal form.
The fourth and nal step of the modeling process,
computer modeling, implements the schema and devel-
ops the automated computer system solution to the
problem using a database management system.
4.1. Problem denition
Pontis is a comprehensive bridge management infor-
mation system. Its main functions include information
gathering, interpretation, prediction, cost accounting,
decision-making, budgeting, and planning [1]. A bridge is
the basic component in Pontis and consists of forty-seven
elements grouped into four main categories: deck/slab
elements, superstructure elements, substructure elements,
and miscellaneous elements. To identify the data items that
must be included in the imaging information model the
following were conducted: (1) An analysis of the Michigan
Department of Transportations (MDOT) bridge inspection
forms that support its Pontis bridge management functions
Fig. 2. Four-step information system modeling methodology.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 475
[16], (2) a thorough study of the Pontis system, (3) a
comprehensive literature review of bridge inspection and
maintenance procedures, and (4) an analysis of the needs of
the automated imaging framework. The following is the
resulting nal set of data items:
Bridge number: a local bridge identication number
assigned by the state
Facility: name of the road that the bridge is part of
Feature: which crossing road, railroad, or river intersects
with the bridge
Location: location of the bridge
Length: length of the bridge
Width: width of the bridge
Federal ID: bridge Federal ID assigned by FHWA and
used by Pontis
Fig. 3. E-R diagram for the imaging information system.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 476
Inspection date: date of the inspection
Weather: weather condition at the time of inspection
Inspector name: name of the inspector performs the
inspection
Inspector ID: inspector identication number
Year built: year when the bridge was built
Element number: standard Pontis element identication
number
Element description: Pontis description of the element
Condition rating: Pontis element condition assessment
number (from 0 to 9)
Condition rating description: description of the 09
condition ratings
Crack description: description of the type and shape of
the crack
Crack length: length of the crack
Crack width: width of the crack
Crack depth: depth of the crack
Presence of scaling: indicates the presence of scaling
Presence of spalling: indicates the presence of spalling
Presence of exposed reinforcement: indicates the pre-
sence of exposed reinforcement
Original image: image of the element before processing
Deformed image: image of the element after processing
and after isolating cracks or deformation
4.2. Conceptual data modeling
An E-R conceptual data model for the bridge imaging
information system was developed using the nal list of data
items (see Fig. 3). In this gure, the E-R data model consists
of nine entities: Bridge, Element, Date, Image, Crack,
Surface Wearing, Inspector, and Condition Rating. Each
entity has data items called attributes. For example the
Element entity has two attributes: Element ID, and
Element Description. Entities are connected to each other
by relationships. These relationships can be one-to-one,
one-to-many, or many-to-many relations. An example of a
one-to-many relationship is the relation between the
Element entity and the Image entity where each element
has more than one image, but each image describes a single
element. An example for a one-to-one relationship is the
relation between the Inspector and the Rating entities,
where one inspector gives only one rating for each element
in a bridge, and each element has one rating.
4.3. Relational data modeling
The relational model for the imaging inspection
information system was created from the E-R
conceptual model. Relations are expressed using the
following format
Relation-nameattribute-1; attribute-2; .; attribute-n
where the underlined attributes represent the key of the
relation. The relational model is optimized to the third
normal form and consists of the following nine relations:
(1) Bridge Information (Bridge ID, Federal ID, Location,
Length, Width, Facility, Feature, Year Built)
Fig. 4. Database schema design.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 477
(2) Element Information (Element ID, Element
Description)
(3) Date (Date, Weather, Temp.)
(4) Inspector Image (Date, Bridge ID, Element ID, Original
Image, Crack Image)
(5) Crack (Bridge ID, Element ID, Date, Crack Description,
Crack Length, Crack Width, Crack Depth)
(6) Surface Wearing (Bridge ID, Element ID, Date,
Scaling, Spalling, Expose Reinforcement)
(7) Inspector Information (Inspector Name, Inspector ID,
Agency Consultant)
(8) Element Condition Rating (Inspector ID, Bridge ID,
Element ID, Date, Rating, Comment)
(9) Condition Rating (Rating, Description)
Fig. 4 shows the relational schema design and shows
the relationships between the various relations in the
database.
4.4. Computer modeling
To develop the automated information system, the
relational data model was implemented using a Structured
Query Language (SQL)-based database management sys-
tem. The discussion in this section is intended to
demonstrate how the information model was used. The
implementation consisted of three major components:
Tables, Electronic Forms, and Reports. The Tables
component is essentially an implementation of the same
nine relations described in Section 4.3. The Electronic
Forms component consists of a number of forms that are
developed to facilitate the data entry and data manipulation.
Example electronic forms are shown in Figs. 5 and 6. Fig. 5
is the Inspector Image Query Form used to view and analyze
a bridge image. To use this form, the user must rst provide
the Bridge ID, Element ID, and Date. This form will then
display the original image and the cracked image produced
Fig. 5. Inspector image query form.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 478
Fig. 6. Detected cracks form.
Fig. 7. Bridge inspection report.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 479
by the data interpretation module of the imaging inspection
system. The user can then enter a condition assessment of
the bridge element under investigation by invoking the
Detected Cracks form (see Fig. 6).
Finally, the Reports component consists of a number of
output reports that are developed to present information
stored in the imaging system. An example report is shown in
Fig. 7. This report is entitled Bridge Inspection Report,
presents information such as element ID, element rating,
and comments on all inspected elements for a given bridge
on a given date. The report also lists the detailed bridge
identication information and the inspector data. The
reports component is intended to demonstrate an example
of what the system is capable of providing in terms of
reports. Once an optimum (third Normal Form) database
model has been designed and implemented, users can
develop as many reports as needed by a simple manipulation
of the data in the relational database model.
5. Concluding remarks
Inspection is critical for the safety and serviceability of
bridges throughout their service life. Visual inspection,
which is the primary method in use, is slow and potentially
hazardous for the inspector. In this research project, an
automated imaging inspection framework for concrete
bridges was proposed. Then, the imaging information
model in support of the automated framework was
developed in an attempt to provide more reliable and
automated techniques for condition assessment. The
underlying thesis of the research project described in this
paper is that inspection data using advanced non-destructive
techniques such as imaging need to be properly maintained
through the optimum design of a data model that is
integrated with standard BMSs such as Pontis. This
integration and optimum design have been demonstrated
in this paper.
The automated imaging system approach increases the
safety of the inspection process and provides a faster method
for inspection. Also, the imaging data can be revisited and
evaluated several times, if needed, since they are stored in the
system. Additionally, since the ultimate goal is to have
continuous automated image capture and interpretation
mechanisms, the inspection frequency can be signicantly
increased.
Furthermore, the research project described in this paper
essentially proposes extending the scope of Pontis rather
than replacing Pontis. This makes the proposed framework
easy to adopt by all highway departments once a complete
system is developed, tested, and evaluated. Finally, the use
of a third normal form relational data model makes the
information system extendable, allowing future expansion
of the system without the need for database schema
redesign. This is critical for future addition of an automated
condition assessment image processing algorithm(s).
It is worth noting that the framework proposed in this
paper will require relentless development, verication, and
revision efforts before any nal system development is
deployed. This is particularly true for safety-critical
elements of a bridge such as girders and piers. This means
that some bridge elements such as deck condition assess-
ment may adopt automated techniques faster than others. It
may also mean that some aspects of the framework can be
implemented while others are still under development
and/or consideration.
Acknowledgements
The authors would like to thank Ms Zhizhen Liu, P.E., a
Bridge Engineer at the Michigan Department of Transpor-
tation (MDOT), for her help during the course of this
research project.
References
[1] Golabi K, Thompson P, Hyman W. Pontis: a network optimization
system for bridge improvements and maintenance Technical manual.
Publication number FHWA-SA-94-031: US Department of Trans-
portation, Federal Highway Administration; 1993.
[2] Brecher A. Infrastructure: a national priority. Soc Women Eng 1995;
4(16):1416.
[3] Roberts E, Shepard R. Bridge management for the 21st century.
Transport Res Rec 2000;1696:197203.
[4] Haas R, Hudson WR, Zaniewski J. Modern pavement management.
Malabar, FL: Krieger; 1994.
[5] Abudayyeh O, Al-Battaineh H. An as-built information model for
bridgemaintenance. J Comput Civil Eng, AmSoc Civil Eng2003;17(2).
[6] Gole B. Management vs. crisis reaction. Report. Chicago, IL:
American Public Works Association; 1985.
[7] Ryall MJ. Bridge management. Oxford: ButterworthHeinemann;
2001.
[8] Czepiel E. Bridge management systems literature review and search.:
Northwestern University BIRL Industrial Research Laboratory; 2002.
Available at: http://www.iti.northwestern.edu/publications/techni-
cal_reports/tr11.html.
[9] AASHTO. Guidelines for bridge management systems. Washington,
DC: American Association of State Highway and Transportation
Ofcials; 1993.
[10] Saito M, Sinha C. Data Collection and analysis of bridge rehabilita-
tion and maintenance costs. Transport Res Rec 1990;1276.
[11] AASHTO. Manual for condition evaluation of bridges. Washington,
DC: American Association of State Highway and Transportation
Ofcials; 2000.
[12] Silano L. Bridge inspection and rehabilitation: a practical guide. New
York, NY: Wiley; 1993.
[13] Lee H. Evaluation of PAVEDEX computerized pavement image
processing system in Washington. Proceedings of the Pavement
Distress Data Recognition Seminar, Ames, IA, June 1990.
[14] Longenecker K. Pavement surface video image work in Idaho
Proceedings of the Pavement Distress Data Recognition Seminar,
Ames, IA, June 1990.
[15] Abdel-Qader I, Abudayyeh O, Kelly M. Analysis of Edge Detection
Techniques for Crack Identication in Bridges. J Comput Civil Eng
Am Soc Civil Eng 2003;17(4).
[16] MDOT. MDOT form information on-line: Michigan Department of
Transportation(MDOT); 2002http://www.mdot.state.mi.us/webforms/.
O. Abudayyeh et al. / Advances in Engineering Software 35 (2004) 473480 480