Index

A
Analysis report, types, 6263
API, see Applications programming interface
Applications programming interface (API), 119, 119f
AS9100, 1112
ATE, see Automatic test equipment
Automatic test equipment (ATE), military systems,
534
Autonomy, rule-based autonomy in spacecraft
development, 361, 362t

B
BIT, see Built-in-test
BITE, see Built-in-test equipment
Built-in-test (BIT), military systems, 534
Built-in-test equipment (BITE), military systems, 534

C
Capabilities based assessment (CBA), defense
acquisition, 463464
Capability Maturity Model Integration (CMMI)
definitions, 14
framework, 15
ISO 9001 comparison, 1415, 14t
maturity levels, 13t
process areas, 13t, 16t
CBA, see Capabilities based assessment
CBE, see Current best estimate
CCB, see Change control board
Change control board (CCB), 188190, 191
CMMI, see Capability Maturity Model Integration
Code inspection, 41
Code of Federal Regulations 820, see Quality
Systems Regulations
Commercial off-the-shelf components and equipment
(COTS)
military use, 461462, 541b
spacecraft modules and subsystems, 326328

Configuration Management Plan, components, 46

Configuration management
design repository, 43
file structure, 4445
military systems, 498499
obsolete documents, 45
rationale, 42
records responsibility, 43
scope, 42
spacecraft development, 370
system and location, 531532
training, 4546
version control, 531532
COTS, see Commercial off-the-shelf components and
equipment
Current best estimate (CBE), systems engineering, 353

D
Department of Defense (DoD), see also Joint
Capabilities Integration Development System;
Military projects
acquisition process, 464466, 464f
decision-making support systems, 273f
Design control, see Medical devices
Design description, components, 6364
Design history file (DHF), medical software,
180182
Design review, 40
Development plan, components, 5758
Development process
example, 2933, 30f, 31f, 32f
overview, 27
processes versus procedures, 27
spiral model, 28, 29f
V-model, 2728, 28f
DHF, see Design history file
DO-178B, 523525
DO-178C, 525, 538b
DO-254, 525526

571

Index

Earned value management (EMV), integrated systems

engineering, 553b
ECN, see Engineering change notice
ECR, see Engineering change request
Electromagnetic compatibility (EMC), military project
testing, 531532
Electronic Protected Health Information (EPHI), 258
Electrostatic discharge (ESD), spacecraft
development, 369
EMC, see Electromagnetic compatibility
EMV, see Earned value management
Engineering change notice (ECN), components, 5253
Engineering change request (ECR), components, 52
EPHI, see Electronic Protected Health Information
ESD, see Electrostatic discharge

implementation, 361362
New Horizons mission, 411412, 412f, 413t, 414t
requirements, 356360, 358t
rule-based autonomy, 361, 362t
Fault tree analysis (FTA), risk analysis, 196
FDA, see Food and Drug Administration
510(k)
premarket submission for medical devices, 238240
special form, 247
FMEA, see Failure mode and effects analysis
FMECA, see Failure modes and effects criticality
analysis
Food and Drug Administration (FDA), see also Quality
Systems Regulations
approval process, 236
audits, 167168
device risk classes
determination, 238
overview, 236242
guidance documents, 154b
ISO standard differences, 157
life cycle model for software development, 151154,
152f
metrics for software, 233236, 235f
overview, 147-148
premarket approval, 240241
premarket documentation for software
major level of concern, 245246
minor level of concern, 244
moderate level of concern, 244245
off-the-shelf software, 246247
premarket submission, 148, 238240, 241242, 247
review process, 248
Software of Unknown Pedigree
evolutionary prototypes, 251, 252
overview, 248257
retrospective verification and validation, 255257
third-party validation, 252254
throwaway prototypes, 249250
software verification and validation, 149151
FTA, see Fault tree analysis
Functional requirements, components, 59

Failure mode and effects analysis (FMEA), risk

analysis, 196
Failure modes and effects criticality analysis
(FMECA), risk analysis, 196
Fault protection, spacecraft development
design, 360361

GNAT Pro, 537b

Documentation
best practices, 4849
coverage and responsibility, 4647
good practice summary, 420422
Pluto-Kuiper Belt mission, see New Horizons
mission
medical device software premarket documentation
major level of concern, 245246
minor level of concern, 244
moderate level of concern, 244245
project development
program management documents, 66t
technical project documents, 72t
purposes, 46
spacecraft development
corporate documentation, 395401, 400t
example of subsystem requirements document,
428444
project documentation, 392395, 392t
revision and approval, 399401
safety documentation, 395, 396t, 398t
tools, 402
types, 47f, 48
Documentation Plan, components, 62
DoD, see Department of Defense

H
Hazard analysis
insulin pump example, 198b
overview, 99

572

Index
L

spacecraft design, 375

Health Insurance Portability and Accounting Act
(HIPAA)
compliance, 257259
security practices
administrative support, 259
malicious software, 259
physical security, 260
risk analysis, 260
security rule, 257
HIPAA, see Health Insurance Portability and
Accounting Act
Hubble telescope, 150b
Humanity, definition, 4

Legacy systems, military, 535537

I
ICD, see Interface control document
IEC, see International Electrotechnical Commission
IMP, see Integrated master plan
Insulin pump, hazard analysis example, 198b
Integrated master plan (IMP), military projects, 500
Integration and test process, see Spacecraft
development
Integrity, definition, 4
Interface, definition, 4
Interface control document (ICD), components,
5657
International Electrotechnical Commission (IEC)
guidance documents, 154b
overview, 154156
International Organization for Standardization (ISO)
medical device quality requirements, 157158
overview, 154
ISO, see International Organization for Standardization
ISO 9001, 1011, 1415, 14t
ISO/IEC 15288:2008, 471472
Iteration, definition, 4

J
JCIDS, see Joint Capabilities Integration Development
System
Jet Propulsion Laboratory (JPL), mission assurance
and safety guidelines, 312
Joint Capabilities Integration Development System
(JCIDS)
effectiveness, 466467
limitations and guidelines, 462464
Joint Strike Fighter (JSF), 543b
JPL, see Jet Propulsion Laboratory
JSF, see Joint Strike Fighter

Medical devices
certification, 137138
comparison with other systems, 6t
design control
change control, 182184
design changes, 180
design history file, 180182
design input, 172176
design output, 176177
design review, 177178
project planning, 170172
purpose, 168170, 168f
software problem reporting
corrective action process, 191192
evaluation phase, 188190
outcomes from evaluation, 190191
process workflow, 189f
reports, 193
steps, 187188
system test phase outcomes, 192193
version control, 193
verification and validation
auditable build, 228
integration tests, 226227
overview, 178179, 206248, 207f
regression testing, 227228
review process, 212215
static tests, 224226
test harnesses, 217224
unit testing, 216217, 219f
version control system, 181, 184187, 225
evaluation process
compliance reports, 143144
noncompliances, 144145
preliminary evaluation, 141143
testing, 143
frequently asked questions, 264266
metrics for software, 233236, 235f
privacy and security, see Health Insurance
Portability and Accounting Act
regulations, see also Food and Drug
Administration; Quality Systems
Regulations
Canada, 137
Europe, 136137
United States, 135136

573

Index
Medical devices (Continued)
risk management
failure mode and effects analysis, 196
failure modes and effects criticality
analysis, 196
fault tree analysis, 196
insulin pump hazard analysis example, 198b
overview of process, 193206, 195f
software level of concern, 242244
standards, 138141, 140t
systems validation, 232
traceability, 232233
Metrology, concerns and procedures, 65
Microwave oven
hardware interlocks, 117f, 118
magnetron safety, 127, 128129
risk analysis, 116
Military projects
avionics standards
DO-178B, 523525
DO-178C, 525
DO-254, 525526
overview, 522526
capabilities based assessment process,
463464
case studies, 537b, 538b, 539b, 541b, 543b,
550b, 553b, 555b
comparison with other systems, 6t
configuration management, 498499
cost-effectiveness analysis, 498
data management, 499
defense acquisition, 464466, 464f, 465f
Department of Defense resources, 502504
design
overview, 489492
verification process, 491492, 491t, 492f
EIA 632 standards, 470471
functional allocation, 488
functional analysis, 486488
historical perspective of systems engineering,
461472
IEEE 1220 standards, 468472
input information for systems engineering, 476477,
477f
integrated master plan, 500
interface management, 499
ISO/IEC 15288:2008 standards, 471472
Joint Capabilities Integration Development system
effectiveness, 466467
limitations and guidelines, 462464

Mil-STD-499B
standards and handbooks, 505t, 525526
systems engineering planning and
implementation, 474476
miscellaneous standards and specifications, 496t,
518522
obsolescence and legacy systems, 535537
output for systems engineering, 479
processes, procedures, and tasks, 472501, 473f,
474f, 480t
requirements analysis
functional analysis, 484, 484t, 485f
functional verification, 484486, 486t, 487f
overview, 480486
process, 481, 481t, 482f
validation, 481484, 483f, 483t
synthesis process, 488, 489, 489t, 490f
systems analysis and control, 493494, 494t,
495f, 496t, 497f
technical objectives
life-cycle support data, 478479
overview, 477479
specifications and baselines, 478
technical performance measurement, 500
technical review, 501
tests and evaluation
automatic test equipment, 534
built-in-test, 534
built-in-test equipment, 534
electromagnetic compatibility testing, 531532
environmental tests, 530531
field tests, 532533
inspection, 527
integration, 529530
manufacturing test, 533
overview, 527534
peer review, 527
subsystem tests, 527529
tradeoff studies
functional analysis and allocation, 496497
requirements analysis, 496
synthesis, 498
Minutes, review, components, 51
Mission assurance
definition, 2
processes, 25
spacecraft, see Spacecraft design
supporting disciplines, 2527
Mission critical, definition, 3
Multidimensions, definition, 5

574

Index
N
NASA, see National Aeronautics and Space
Administration
National Aeronautics and Space Administration
(NASA)
integration and test, 316317
mission assurance and safety, 275316
mission operations, 317318
project management guidelines, 270272, 270f, 271f
reliability preferred practices, 275, 276t
summary of standards, 319t
systems engineering guidelines, 272275
National Environmental Protection Act, approval of
space launch, 375, 375f
New Horizons mission
Announcement of Opportunity, 402403
circuit board fabrication and assembly, 417
fault protection, 411412, 412f, 413t, 414t
instrument suite, 405t
launch and mission operations, 418419
mission assurance and safety, 412417
objectives, 404t
overview, 402419
project management, 406, 406f, 407f
subsystem testing, 257261
systems engineering
design tradeoffs, 407410
margin management, 411, 411f
requirements flow-down, 407, 408f
risk management, 410, 410t

O
Object-oriented programming, 9394, 93b
Obsolescence, military systems, 535537
Off-the-shelf software, premarket documentation for
medical devices, 246247
Operation Plan, components, 65
Operational requirements, components, 5859
Outside-the-box testing, 89b

P
Peer review, 3940
Performance requirements, components, 5961
PERRU, see Plan, execute, review, report, and
update
Plan, execute, review, report, and update (PERRU),
26f, 27
Pluto-Kuiper Belt mission, see New Horizons mission
PMP, see Project management plan
PRCA, see Problem report/corrective action

Problem report/corrective action (PRCA),

components, 51
Procedure
definition, 3
military project systems engineering, 472
process comparison, 39
review
action and updates, 4142
content, 41
frequency, 41
rationale, 39
responsibilities, 42
types, 3941
Process
definition, 3
military project systems engineering, 472, 473f,
479, 480t
procedure comparison, 39
review
rationale, 39
types, 3941
frequency, 41
content, 41
action and updates, 4142
responsibilities, 42
Project, definition, 18
Project management
definition, 18
documents for project development, 66t
inputs and outputs, 18
operations, 1819
scheduling, 1920
spacecraft development
communication and teamwork, 329333, 334t
National Aeronautics and Space Administration
guidelines, 270272, 270f, 271f
New Horizons mission, 406, 406f, 407f
phases of project, 335336, 336t
review process, 337345, 338t, 345f
risk management, 346349, 346f, 347t, 348f, 348t
schedule, 335
team roles and responsibilities, 329, 330f, 330t
work breakdown structure, 333334, 334f
Project management plan (PMP), components,
5356

Q
QMS, see Quality management system
QSR, see Quality Systems Regulations
Quality, definition, 3

575

Index
Quality management system (QMS)
components, 89
development and certification
AS9100, 1112
Capability Maturity Model Integration, 1215,
13t, 14t
ISO 9001, 1011, 1415, 14t
Six Sigma, 12
measurement, analysis, and improvement, 1517
overview, 7
product realization, 10
rationale, 78, 8f
resource management, 9
responsibilities and roles, 9, 9t
training, 15
Quality Systems Regulations (QSR)
overview, 157158
subparts
acceptance activities, 162163
corrective and preventive action, 163164
design controls, 159
document controls, 159161
general provisions, 158
handling, storage, distribution, and installation,
164165
identification and traceability, 159
labeling and packaging control, 164
nonconforming product, 162163
production and process controls, 162
purchasing controls, 161162
quality system requirements, 158159
records, 165166
servicing, 166167
statistical techniques, 166167

R
Readability, software design, 9192, 91b
Real-time operating system (RTOS), 111b, 122
Review
action and updates, 4142
content, 41
frequency, 41
rationale, 39
responsibilities, 42
types, 3941
Risk management
Health Insurance Portability and Accounting
Act, 260
medical devices
failure mode and effects analysis, 196

failure modes and effects criticality analysis, 196

fault tree analysis, 196
insulin pump hazard analysis example, 198b
overview of process, 193206, 195f
spacecraft development, 346349, 346f, 347t,
348f, 348t
Risk Management Plan, components, 61
RTOS, see Real-time operating system

S
Safety-critical system, definition, 3
Safety laboratory markings, 154b
SEMP, see Systems engineering management plan
SEP, see Systems engineering plan
SETD, see Systems engineering tasking description
Six Sigma, 12
Software design
error messages, 109b
hardware as software thought exercise, 116132
medical devices, see Food and Drug Administration
overview, 8385
principles
failsafe, 8999
provability, 97
readability, 9192, 91b
redundancy, 94
safety and mission first, 99103
verification and redundancy in implementation,
103105
process, 8589, 86f, 88f
real-time operating system, 111b, 122
tight-loop, queued design, 113, 114f
tight-loop, switched design, 112, 113f
use cases of failure, 101, 102, 103
user interface, 105110
Software level of concern, medical devices, 242247
Software of Unknown Pedigree (SOUP)
evolutionary prototypes, 251, 252
overview, 248257
retrospective verification and validation, 255257
third-party validation, 252254
throwaway prototypes, 249250
Software problem reporting (SPR)
corrective action process
assign phase, 191
in-progress phase, 192
requirements analysis phase, 191
system test phase, 192
evaluation phase
outcomes from evaluation

576

Index
analyze, 191
defer, 190
drop, 190191
repair, 191
overview, 188190
process workflow, 189f
reports, 193
steps, 187188
system test phase outcomes
fixed, 192193
reopen, 192
version control, 193
SOUP, see Software of Unknown Pedigree
Spacecraft development
commercial off-the-shelf modules and subsystems,
326328, 541b
comparison with other systems, 6t
documentation
corporate documentation, 395401, 400t
example of subsystem requirements document,
428444
project documentation, 392395, 392t
revision and approval, 399401
safety documentation, 395, 396t, 398t
tools, 402
fault protection
design, 360361
implementation, 361362
requirements, 356360, 358t
rule-based autonomy, 361, 362t
government regulation, see National Aeronautics
and Space Administration
industry standards
fault protection, 324
integration and test, 325
miscellaneous standards, 325326
mission assurance and safety, 324
overview, 318326
integration and test process
anomaly reporting, 389390
closure of tests, 390
component-level tests, 380381
data trending, 378
elements of successful program, 379
example of test procedure, 386f
flow chart, 383f
independent data review, 378379
outside organization test activities, 380
overview, 376391
performance tests, 387388

requirements and test, 377378

retest requirements, 390
small test plan example, 445460
spacecraft-level tests, 381387, 383t
special tests, 388
test closure process, 379
test conducting, 388389
test equipment calibration and certification,
390391
military standard summary, 321t
mission assurance and safety
configuration management, 370
contamination control, 373
custom design, 365366
data management, 370371
electrical, electronic, and electromechanical part
procurement, 367368
hazard analysis, 375
materials and process control, 368369
National Environmental Protection Act approval
of launch, 375, 375f
nonconformance processes, 371373
overview, 363375
radiation hardness, 368
reliability, 367
safety
overview, 373
personnel safety, 374
system safety, 373374
software, 366
workmanship, 370
mission operations procedures, 395401
project management
communication and teamwork, 329333, 334t
phases of project, 335336, 336t
review process, 337345, 338t, 345f
risk management, 346349, 346f, 347t, 348f, 348t
schedule, 335
team roles and responsibilities, 329, 330f, 330t
work breakdown structure, 333334, 334f
systems engineering
architecture/design trades, 356
interface management, 352
lessons learned, 353355
overview, 350356
requirements engineering, 350352, 351f
resource management, 352353, 354t
SPR, see Software problem reporting
Standards organizations
aviation and aerospace organizations, 3435

577

Index
Quality Systems Regulations, 159
Trajectory correction maneuver (TCM), fault
protection, 411412, 412f

Standards organizations (Continued)

general organizations, 3334
industry-based organizations
electrical, 34
mechanical, 34
military organizations, 34
Summary task, definition, 21
System, definition, 2
Systems engineering
military projects, see Military projects
definitions, 2, 2024
inputs and outputs, 21
operations, 9
requirements
development, 2324, 24f
overview, 2122
spacecraft development
architecture/design trades, 356
interface management, 352
lessons learned, 353355
New Horizons mission
design tradeoffs, 407410
margin management, 411, 411f
requirements flow-down, 407, 408f
risk management, 410, 410t
overview, 350356
requirements engineering, 350352, 351f
resource management, 352353, 354t
verification, 362363
Systems engineering management plan (SEMP),
Mil-STD-499B, 476
Systems engineering plan (SEP), example, 425427
Systems engineering tasking description (SETD),
Mil-STD-499B, 474

U
Use cases, software failure, 101, 102, 103

V
Validation, see Verification and validation
VCS, see Version control system
Verification and validation (V & V)
definition, 3
medical device software
auditable build, 228
integration tests, 226227
overview, 149151, 178179, 206248, 207f
regression testing, 227228
retrospective verification and validation, 255257
review process, 212215
static tests, 224226
test harnesses, 217224
unit testing, 216217, 219f
software design, 87, 89
spacecraft testing
controls, 447
instrument software, 449
validation definition, 3
Version control system (VCS)
management planning, 184187
software archive, 181
software problem reporting, 193
V-model
development, 2728, 28f
process, 87, 88f
V & V, see Verification and validation

T
TCM, see Trajectory correction maneuver
TDD, see Test-driven development
Technical performance measurement (TPM), military
projects, 500
Test harnesses, software certification and validation,
217224
Test Plan, components, 6480
Test-driven development (TDD), systems engineering,
23, 24f
Therac-25 radiation therapy, 124b
TPM, see Technical performance measurement
Traceability
medical devices, 232233

W
Waterfall model
life cycle model for software development, 151154,
152f
medical device design, 178179
process, 8589, 86f
WBS, see Work break-down structure
WO, see Work order
Work break-down structure (WBS)
definition, 20
spacecraft development, 333334, 334f
Work order (WO), components, 51
Work package, definition, 21

578