81 Care About The Ilities PDF

Why You Should Care About the -ilities!
August 2014
Alan Southwick, CSP, CRE & CQE

Whole Life Engineering Directorate (WLED),
System Safety Engineering Section
Integrated Defense Systems (IDS),
Seapower Capability Center (SCC)
Raytheon Company
Alan_E_Southwick@Raytheon.com
1847 West Main Road
Nimitz Building 1W-2W C191, Mail Code N28
Portsmouth, Rhode Island USA 02871-1087
Copyright 2014 Raytheon Company. All rights reserved.
Customer Success Is Our Mission is a registered trademark of Raytheon Company.
What are the -ilities?

Most grew out of Quality Control Support Activities
The -ilities are
typically Support
Engineering roles.
Specialty
Engineering is the
management of
specialty
requirements and
products.
QUALITY
RELIABILITY
MAINTAINABILITY
SUPPORTABILITY
HUMAN FACTORS
SAFETY
SECURITY
Customer Success Is Our Mission is a registered trademark of Raytheon Company
Slide 2
Quality, Quality Control, and Quality Engineering

Specialty Engineering Definition
Specialty Engineering Roles and Relationships
Reliability
Maintainability
Supportability
Human Factors
Safety
Security
Slide 3
Quality and Quality Control
Classic Definitions of Quality (Juran):

Degree to which an inherent characteristic
fulfills requirements.
The Quality Function is the entire collection of
activities through which we achieve fitness
for use, no matter where these activities are
performed.
Reference: Jurans Quality Handbook, The Complete
Guide to Performance Excellence, Sixth Edition
Publisher: McGraw Hill International

Slide 4
Quality and Quality Control
Classic Definition of Quality Control (Juran):

Quality control is a universal managerial process
for conducting operations so as to provide stability
over time, and to prevent adverse change and to
maintain the status quo.
Quality control takes place by use of the feedback
loop.
Quality control entails the maintenance or
restoration of the operating status quo and as
measured by (meeting) the acceptable level of
defects and provision of customer needs.
Reference: Jurans Quality Handbook, The Complete
Guide to Performance Excellence, Sixth Edition
Publisher: McGraw Hill International

Slide 5
Quality Engineering
Interrelation Among Specialty Engineering

Disciplines
Specialty Engineering disciplines can be sketched out in
the form of a tree which helps bring out the
interrelationships with Quality and Quality Engineering.

Slide 6
Quality Engineering (again, from Juran)

All -ilities!
All these activities
typically require some
QUALITY
form of Proprietary
OF
Controls (Security) to
DESIGN
protect and track the
design, manufacture, and
sometimes the customer.
QUALITY OF
CONFORMANCE
QUALITY OF MARKET RESEARCH

QUALITY OF CONCEPT
QUALITY OF SPECIFICATION
TECHNOLOGY
MANPOWER
MANAGEMENT
FITNES FOR USE
RELIABILITY
AVAILABILITY
MAINTAINABILITY
LOGISTICAL SUPPORT
PROMPTNESS
FIELD
SERVICE
COMPETENCE
INTEGRITY
Slide 7
Quality Engineering
Myron Tribus Perversity Principle:

If you try to improve the performance of a system
of people, procedures, practices and machines by
setting goals and targets (and incentives) for the
individual parts of the system, the system will
defeat you every time, and you will pay a price
where you least expected it.

Slide 8
Ways That Quality Can Improve Things

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Maintain Continuous Communication with the everyone on the

project, including the customers.
Institute Formalized, living Processes and Procedures that are
understandable, straightforward and easy to implement.
Design Quality into the Front End First by leveraging the -ilities
through Specialty Engineering.
Introduce Quality Techniques into the Product Design,
Development, Manufacturing, Distribution and User Processes.
Focus on Efficiencies in Designs and Materials.
Work on Current, Prioritized tasking.
Provide Necessary Tools, Training, and Supplies to perform tasks
efficiently to Quality Standards.
Use Appropriate Materials, Tools, Equipment, and Training.
Do Not Reward Predatory Behavior for getting it done!
Eliminate Bottlenecks [Alligators thatll eat your productivity]!
Implement Statistical Process Controls (SPC) to track Quality.
Use Technology Wisely, not because its the latest n greatest!
Slide 9
Ways That Quality Can Improve Things

TO ASSURE FITNESS FOR USE!
TRY TO FOCUS UPON
Application of Jurans Pareto Principle
or,
80 / 20 RULE
[after Vilfredo Pareto (15 July 1848 19 August 1923)]
80% of your problems occur with 20% of your sources.
THE VITAL FEW AND TRIVIAL MANY

Slide 10

Reliability
Maintainability
Supportability
Human Factors
Safety
Security
Slide 11
Specialty Engineering Benefits

The Specialty Engineer:
Helps the Systems Engineers with in-depth knowledge
and experience in a narrow specialty.
Coordinates with Systems Engineers to address
specific technical aspects of the program and review
work products.
Coordinates with customers to appraise the specialty
requirements and review the work products.
Works with both Systems Engineers and Customers to
reduce risks and control costs.
Slide 12
Why? Consider these Headlines

LED Lamps Last Over 50,000 Hours! [MTTF]
Automaker offers 10 Year or 100,000 Mile Power Train Warranty
Microwave Oven Returned for 6th Replacement in Two Months
Automaker Recalls nearly 2,000,000 Vehicles for Safety Problems
Credit Card Account Data Hacked
$30 Timing Belt Takes 2 Weeks to Replace and Costs $800
[MTTR]
Rate of Aircraft Near Misses on Increase according to FAA.
Controllers asleep again! [MTBF & Safety]
Do some of these sound familiar?

How do these issues relate to the -ilities?
And, what do they have to do with Quality?
Slide 13
Percent of Life Cycle Cost Incurred

During Each Phase of a Program
LIFE C YC LE CO ST
OPER ATIO N A N D S UP P OR T
SY S TEM A C QU ISIT ION
Conceptual
Engineering
Program
Definition
Risk Reduction
Engineering
Manufacturing
Development
DISPOSAL
5 5%
30 %
5%
1 0%
Time
Non-recurring (NR) costs

(Mostly)
Recurring costs
(Mostly)
NR costs for disposal

Slide 14
Roles of Specialty Engineers

Provide skills in special area(s)
Support concurrent engineering and account for cross product
and design dependencies:
Reliability and maintainability / life cycle costs and logistics

Producability and supportability / performance
System Safety and Human factors / liability and costs
Quality / Assures product design and manufacture integrity
Design and Development Proprietary Issues (Security)
Program Conceptualization and Capture

System Requirements
Design, Development and Manufacture
Operation and Maintenance
End of Life Equipment Disposal
Specialty Engineers have roles in all phases of the systems

lifecycle:

Slide 15
Design Influence and Interfaces

Objectives:
Through the system engineering process, identify items / processes that

1) Impact the design from inception throughout the life cycle;
2) Facilitate supportability to maximize availability;
and,
3) Enhance system effectiveness and capability at the lowest life cycle cost
Reliability, Availability, Maintainability (RAM)
Standardization and Interoperability
Survivability
Testability
Environmental (e.g., air, waste, noise)
Transportability
Corrosion & Non-Destructive Inspection
Energy Management
System Safety
Hazardous Material (HAZMAT)

Management
Human Factors, Human System Interface &

Usability
Affordability
Quality
Security
Slide 16
Relationship with Systems Engineering

Systems Engineers must plan for a timely and effective process to
establish and coordinate specialty disciplines.
Systems Engineering integrates the technical specialties into the
program.
Defining technical requirements for specialty engineering, with their assistance.

Ensuring each Specialty understands its requirements.
Include necessary Specialties in program plans.
Include Specialty Engineers in product development.
Monitor, assess, and report status of technical specialty areas.
Where applicable, Specialty Engineers should be consulted for design

concept support, proposal inputs, program planning, requirements
definition, development support, manufacturing support, deployment
support, security issue management, and end-of-life disposal.
Failure to do so may have considerable repercussions across the system
and / or product life cycle affecting both schedule and costs.
Slide 17
One Type of Risk Assessment Chart

Slide 18

Reliability
Maintainability
Supportability
Human Factors
Safety
Security
Slide 19
Definition of Reliability
Reliability is the probability that an item will perform its

specified or intended function under specified conditions for a
specified interval.
4 parts NECESSARY to defining reliability:
It must be quantifiable (probability)
It has to do with getting some specific work done
(intended function)
It assumes a specific operating environment
(under specified, stated conditions)
It depends on duration (specified interval)
References: ANSI / GEIA-STD-0009, IEEE 1633

Slide 20
Reliability Engineering
Reliability represents the percent of items that

would survive a predetermined time with the
specified (applied) stresses.
A more understandable application of reliability
applies to Warranty Periods as it includes the
operating environments.
Reliability is a cradle to grave discipline designed
to characterize and define a systems life cycle
performance.
Slide 21
Hardware Reliability Engineering

Reliability may be applied to many systems, but is most
commonly applied to hardware equipment and software.
The more familiar application is for hardware through the Bathtub Curve
model, which is really a poor theoretical approximation for the real world.
Hardware model Bathtub Curve Useful Life
Infant mortality
(Constant failure rate)
Failure Rate
Wear out
Time
Availability of Service (Reliability is typically high, e.g. 99.9%)

Slide 22
Software Reliability Engineering

Unlike hardware, where improvements stop with the
termination of a test, software improvements continue in

the field if not continuously, then in batches with regular
software updates/upgrades. This means that software
reliability can be projected in the field as the software
continues to improve in deployment. Field data is also
analyzed to further project the software reliability growth.
Milena Krasich Raytheon Company
private correspondence
16 December 2013

Slide 23
Software Reliability Engineering

Software / Firmware Reliability requires a different approach
based on a number of factors:
Stepwise life cycle
Cyclical Test Analyze And Fix (TAAF). Usually a one-shot fix / repair
for one software issue, but can cause other faults in software
functionality
Program Branching typically leads to Software Errors / Faults
Creation of Protected Memory to prevent critical data overwrites
Adequate memory resources / space for program resources
Software / Firmware Obsolescence Often driven by Hardware
Obsolescence

Slide 24
Software Reliability Definitions

The American Institute of Aeronautics and Astronautics (AIAA)
the application of statistical techniques to data
collected during system development and operation to
specify, predict, estimate, and assess the reliability of
software-based systems.
IEEE 1633 Recommended Practice on Software
Reliability:
(A) The probability that software will not cause the failure of a system
for a specified time under specified conditions.
(B) The ability of a program to perform a required function under
stated conditions for a stated period of time (Classic Reliability
Definition).
Slide 25
Software Reliability
Capability Maturity Model
Model Assumptions
The defect rate observed during the development process is positively correlated with the
defect rate in the field (The more area under the curve, the higher the field defect rate).
Given the same error injection rate, if more defects are discovered and removed earlier,
fewer will remain in later stages.
Reference Reading: Metrics and Models in Software Quality Engineering,

by Stephen Kan, Addison Wesley Publishing
Defect Rate
The higher the Software Engineering Institutes (SEI) Level

the more efficient and organization is in detecting defects
early in software development
SEI Level V
SEI Level IV
SEI Level III
SEI Level II
SEI Level I
The better the process, the better the

process capability ratings and the
better the delivered code, developed
under that process, will perform
Defects will be lower.
Time
Compare to Hardware Reliability Bathtub Curve.

Slide 26
How IEEE 1633 Aligns Software

Development Process
A 3 step process leveraging IEEE 1633:
Step 1 Keene Model for early software predictions
weighs Software Engineering Institutes (SEI) CMMI
Process Capability (e.g. CMMI Levels) to Software Size
(e.g. 10KSLOCs)
Step 2 Software Error Estimation Program (SWEEP)
Tool for tracking growth of Software Trouble Reports
(STRs) and mitigations through Design Change Orders
Step 3 Computer Aided Software Reliability
Estimation (CASRE) Tool for tracking failures in test
through FRACAS (e.g. Software Integration and Test
[SWIT] and Software Acceptance Testing [SAT] failures
during development and integration)

Slide 27
Another Promising Software

Reliability Assessment Process
Recently, a Power Law (Krasich-Peterson) Model
has been developed and a Raytheon patent is
pending utilizing an adaptation of the Weibull
Intensity Function (an instantaneous rate, NOT to
be confused with the Weibull Reliability Distribution
Function) to characterize the early life cycles of
software which appears to closely agree with actual
software performance based on their research.
Regrettably, the authors have yet to present a
paper pending approval of their patent application.
Slide 28
Effort
Software Reliability Value Added Benefits

Reduce cost of failures later in the development
process (similar to Hardware Reliability)
Track failure trends of probabilistic conditions (e.g.
race conditions) and systemic process-related issues
Drive software design corrective actions to improve
reliability results in a lower customer Total Cost of
Ownership (TCO)
Definition
1X
Reliability,
Warranty
& Rework
Integration
& Test
10X
Time
Cost = 100X
1,000X
10,000X
Note curve similarities to previous life cycle costs.

Slide 29
Reliability Design and Evaluation

Reliability design and evaluation covers:
Single Point of Failure
Failure Mode, Effects and Criticality Analysis (FMECA,
or, FMEA No Quantification of Criticalities incorporated)
Electronic Parts/Circuit Tolerance Analysis (EP/CTA)
Component Stress Analysis (Life Tests, HALT, HASS, etc.)
Reliability Modeling MIL, IEEE, NASA Handbooks, etc.
Reliability Predictions [Failure Rates Failures Per Million Hours
(FPMH) or Failures in Time (FITS)] - MIL Handbooks,
Telecordia, Belcore, etc.
Reliability Assessment and FRACAS
Slide 30
Examples of Potential Reliability

Requirements
Mean Time Between Failure (MTBF) or Mean Time To Failure

(MTTF)
MTBF is the Reciprocal of an items Failure Rate (FR)
Provides a false idea of a Reliability attribute without
the inclusion of the applied Stresses when FR is
developed.
Usually based on measured failure data
Physics of failure analysis (e.g. durability,
accelerated life tests, environmental extremes, etc.)
Calculated per industry methods (e.g. MIL-HDBK-217,
BELCORE, etc.)
Mean Time Between Critical Failure (MTBCF)
Inherent reliability of the system taking into account
redundancies and allowable degradation, but its not
real helpful as a metric because its after the fact!
Slide 31
Example Reliability Requirements
Mean Time Between Demand (MTBD)

Fielded reliability of units / modules taking into account
inherent failures, performance failures, FD/FI, BIT false
alarms, etc.
Example: Flight hours divided by Unit Removals
Mean Time Between Maintenance Action (MTBMA)
Fielded reliability of the system taking into account
inherent failures, performance failures, FD/FI, BIT false
alarms, pilot observations, S/W false alarms,
maintenance practices, etc.
Example: Flight hours divided by Maintenance Actions
A Warranty Period provides a much more

understandable representation for reliability
performance and system availability. WHY?

Slide 32
Reliability versus Availability

FOR SPECIFIED STRESSES AND ENVIRONMENT Reliability defines how often, on average, failures can occur for a
particular system, subsystem or component. Reliability is typically
specified as Mean Time Between Failures (MTBF) for repairable
items.
Availability typically defines the average percentage of time a
system, subsystem or component is available for use.
Ao = _Uptime__
Total Time
_____MTBF_______
MTBF + MTTR + MLDT
MTBF = Mean Time Between Failure (Reliability)

MTTR = Mean Time to Repair (Maintainability)
MLDT = Mean Logistics Delay Time
NOTE: Total Time may include Storage, Standby, and Multiple
Operational Intervals (e.g. Missile Launch, Powered Flight, Free Flight,
Detonation, etc.); and may also be broken into individual intervals.
Slide 33
What is FRACAS?
FRACAS is a closed-loop coordinated system that identifies
failures, implements and verifies corrective actions to preclude
recurrence of the failure.
Design
Product
Build
Product
Test
Product
Factory
Floor
Control
System
Implement
Design or
Process
Corrective
Action
Repair
Product
Non
Conformance
Data
Field
Product
Field
Performance
Data
Common
Data
Warehouse
FRB
Root Cause
Corrective Action
FRACAS Tool
Trend Analysis
Failure Analysis
PERFORMANCE
BASED
LOGISTICS
Reference: MIL-HDBK-2155
Slide 34
Reliability Tasks and Tools
Requirements Analysis
Allocation to system elements/components
Reliability design guidelines
Reliability Modeling
Reliability Prediction
Reliability critical items and risks
Hazard tracking
Failure Reporting, Analysis and Corrective Action System
(FRACAS)
Failure Mode, Effects, and Criticality Analysis (FMECA)
Fault Tree Analysis (FTA)
Reliability Growth through Analysis, Test, and Operations
Reliability Stress Testing (HALT, HASS, etc.)
Failure Review Board (FRB)
Subcontractor guidelines/control
Slide 35
Why Reliability Matters

Reliability analyses and assessments help identify Critical
Failures & Failure Modes leading to potentially imminent
hazards
In conjunction with Maintainability and Supportability, helps
determine system Availability for mission success
Clarifies equipment and system Life Cycles (e.g. Missiles
spend most of their life cycle in storage)
Defines System Warranty periods
The higher the reliabilitythe greater the availabilityless
time for corrective maintenanceand less spares required
Assists with Design to Failure and Environmental Performance
Stress Testing and Accelerated Life Testing help determine
overall system performance limitations and lifetimes
Helps Manage System Performance, planning, Costs and
Risks
Slide 36

Reliability
Maintainability
Supportability
Human Factors
Safety
Security

Slide 37
Maintainability Engineering
Maintainability is the ability to quickly, economically, and safely
maintain readiness or restore a failed system to its operational
state.
(Maintenance is the activity necessary to prevent failure or
restore an item to serviceability.)

Slide 38
Maintainability Tasks
Flow-down maintainability requirements
Develop design criteria
Assess maturity
Perform Trade Studies
Identify potential design improvements
Manage Maintenance Costs and Risks

Slide 39
Example Maintainability Requirements

Example maintainability requirements are:
Mean Time To Repair (MTTR)
Maximum Time to Repair (MMAX)
Maintenance Manhours to Repair (MMH/repair)
Maintenance Manhours to Overhaul (MMH/OH)
Maintenance Manhours per Flight Hour (MMH/FH)

Slide 40
Maintainability Design and Evaluation

Maintainability design and evaluation regards issues such as:
Modeling
Predictions
Data Collection
Data Analysis
Maintainability Testing
Testability Assessment and Analyses
Failure Reporting, Analysis and Corrective Action System
(FRACAS)
Maintainability Growth through Analysis, Test, Operations, and
Design Updates
Failure Review Board (FRB)
References: MIL-HDBK-470, MIL-HDBK-2165
Slide 41
Reliability Centered Maintenance
Reliability Centered Maintenance is a process merging

Reliability and Maintainability Analyses (along with
practical user / maintainer / customer focus) to determine
what must be done to ensure that any asset continues to
do what its users want or need it to do in its present
operating context.

Slide 42

The technical standard SAE JA1011, Evaluation Criteria for RCM
Processes, sets out the minimum criteria that any process should
meet before it can be called RCM. This starts with the 7 questions
below, worked through in the order that they are listed:
1.
2.
3.
4.
5.
6.
7.
What is the item supposed to do and its associated performance

standards? (Requirements)
In what ways can it fail to provide the required functions? (FTA)
What are the events that cause each failure? (FMEA)
What happens when each failure occurs? (FMECA and/or FTA)
In what way does each failure matter? (the C of FMECA)
What systematic task can be performed proactively to prevent, or to
diminish to a satisfactory degree, the consequences of the failure?
(Logistics Support and Preventive Maintenance or PM)
What must be done if a suitable preventive task cannot be found?
(Risk Assessment)
Slide 43

Among some of the paradigm shifts inspired by RCM are:
Failures are not necessarily linked to the age of the asset (this is often modeled by
the "memoryless exponential probability distribution)
Changing from efforts to predict life expectancies to trying to manage the process of
failure, through analysis processes such as Physics of Failure (POF) and
FRACAS
Understanding of the difference between the requirements of assets from a user
perspective, and the design reliability of the asset (User / Customer Focused)
Acknowledging the importance of managing assets on condition (often referred to
as condition monitoring, condition based maintenance, and predictive
maintenance)
Implementing the four basic routine maintenance tasks:
1. Inspect to Detect a Potential Failure
2. Rework Prior to Failure
3. Discard Prior to Failure
4. Ascertain or Find Failures that have Already Occurred
Link levels of tolerable risk to maintenance strategy development
Slide 44
References:
As mentioned, RCM is defined in:
SAE JA1011, Evaluation Criteria for ReliabilityCentered Maintenance (RCM) Processes.
A more recent reference is:
NAVSEA S9081-AB-GIB-010, Reliability-Centered
Maintenance (RCM) Handbook.
Slide 45
Why Maintainability Matters

Helps identify specific maintenance tasks to support deployed
equipment and system(s).
Defines maintenance times for both individual tasks and
system wide maintenance: scheduled and un-scheduled.
Identifies times, equipment, and maintenance talents
required to accomplish specifically defined maintenance tasks.
In conjunction with Supportability, defines levels of maintainer
training required to accomplish specific maintenance tasks.
Helps define individual and overall system maintenance
periods, management planning, costs and risks.

Slide 46

Reliability
Maintainability
Supportability
Human Factors
Safety
Specialty Engineering Security
Slide 48
Supportability Engineering (alias: LOGISTICS)

Supportability ensures that the appropriate structure is in
place to supply and maintain a system throughout its life
cycle, from cradle to grave; often referred to as Logistics.
Supportability = Logistics = Integrated Logistics Support (ILS)
Reference: MIL-HDBK-2097
Slide 49

TO PARAPHRASE AN EARLIER ADVERTISING
CAMPAIGN BY A WORLDWIDE SHIPPING
CORPORATION:
Logistics makes the world work better
When it's planes in the sky
For a chain of supply
That's logistics
When the needed parts
come precisely on time
That's logistics
Slide 50
When technology knows

right where everything goes
That's logistics

Slide 51
Yup! Thats
LOGISTICS!

Slide 52
Trivia Questions
1. Any idea what the name of the shipping
company is? [Hint: their catch phrase is
We Love Logistics]
2. Who was the well known media star who
used the tune for this companys
Logistics advertisement campaign as a
signature theme song?
3. And, what was the name of the original
song?
Slide 53
Supportability, Integrated Logistics Support

Elements integrated in Supportability:
Maintenance Planning
Support and Field Test Equipment
Supply Support
Manpower, Personnel, and Training
Facilities
Packaging, Handling, Storage, and Transportation (PHS&T)
Post Production Support Costs
Computer Resources
Technical Data, including maintenance and repair documents
Design Influence (e.g., Repair Level Analysis: Local, Intermediate,
Depot / Manufacturer)
Slide 54
Supportability Tasks
Requirements Allocation
Guidelines Flow-down
Software Supportability (Type, Versions, Revision Level
Tracking)
Support Equipment Identification
Testability Design and Evaluation
Supportability Design and Evaluation
Supportability Management
Slide 55
Why Supportability Matters

The greater the availabilityless assets required to perform a
mission and higher state of readiness
More time spent performing scheduled and unscheduled
maintenance reduces system availability and may increase
required manpower, tools, and costs
For Testability, a careful balance of the right amount of alerts,
minimizing false alarms and high level of confidence in quickly
locating the fault
FRACAS and FRBs provide data to focus on critical issues
related to Supportability.

Slide 56

Reliability
Maintainability
Supportability
Human Factors
Safety
Security

Slide 57
Human Systems Integration (HSI)

or, Human Factors Engineering (HFE)
Human factors is the discipline involved with ensuring that
the human operator and/or maintainers physiological,
psychological and anthropometric factors are an integral
part of the system design for operation and maintenance.
CAN THIS SOLDIER, OPERATOR, USER, MAINTAINER
WITH THIS TRAINING
PERFORM THESE TASKS
TO THESE STANDARDS
UNDER THESE CONDITIONS?

Slide 58
Human Systems Integration (HSI) is

One Discipline with Many Parts
Cognitive
Psychology
Joint
Technical
Architecture
Task Based
Design
User
Interface
Common Look
and Feel
Customer
Voice
Usability
Testing

Slide 59
Human Factors Tasks

Improve human and overall system performance
Make effective use of manpower and personnel
Optimize the human-machine interface
Minimize training needs and cost
Minimize equipment losses
Maximize equipment effectiveness
Increase safety
Increase acceptance by the user
Ensure system fits the user population

Slide 60
Human Factors Design and Evaluation

Human factors affect design and evaluation by:
Designing for human limitations
Designing for environmental conditions (i.e. Temperature,
Noise Levels, Communications, Vibration, Shock, Fatigue,
etc.)
Completing Human Factors Analyses Review of Design and
Final Equipment for Effective System Implementation
Reference: MIL-STD-1472
Slide 61
Why Human Factors Matters
Standardization: The process of developing and agreeing

upon technical standards.
Goal is to help with the independence of suppliers,
compatibility, interoperability, safety, repeatability, quality,
and/or commonality of function across platforms
Unique standards drive up cost with some doubt of their
effectiveness and longevity for supportability
Interoperability: Product/system properties with understood
common interfaces which facilitates usage with other
present/future systems without restricted access or
implementation.
Note that the definition does not discuss backward capability
which is also very significant
Slide 62
Why Human Factors Matters
Ease of use and learn-ability of a human-made object

Includes a software application, website, book, tool, machine,
process, or just about anything with which a human interacts.
Design goals:
How easy is it to learn?
Once a user is trained, how quickly can they perform tasks?
How quickly can proficiency be re-gained after not using it
for a while?
How many user errors? How severe? How recoverable?
How pleasant is it to use the human interactive design?
Is there a usage Fatigue Factor making it prone to errors?
When the market is densely populated with competing
brands usability can become a distinguishing characteristic
Item should be designed with the human user in mind,
sometimes referred to user-friendly
Slide 63
Human Factors May Include Operations While

Wearing Personal Protective Equipment (PPE)
MOPP (Mission Oriented Protective Posture) (acronym
pronounced as "mop") is a military term used to describe
protective gear to be used in a toxic environment, i.e., during a
chemical, biological, radiological, or nuclear (CBRN) attack.
[This is similar to whats known as Hazmat apparel or PPE.]
There are 5 levels ranging from no protection (0) to:
MOPP4. Personnel completely encapsulate themselves by closing
their overgarments, adjusting all drawstrings to minimize the
likelihood of any openings, and putting on their protective gloves.
MOPP4 is used when the highest degree of protection is required, or
if CB agents are present but the actual hazard is not determined. As
with every other MOPP level, flexibility is built into the system to
provide relief to the individual. Once the hazard is identified and risk
assessment measures are employed, the overgarment may be left
open.
Slide 64
A Simple Example
CONSIDER TRYING TO USE
YOUR CELLPHONE WEARING
LEVEL 4 MOPP GEAR
(Equivalent to being in a space suit)!
NOTE:
Its much more difficult than
wearing a snowsuit and gloves!
Slide 65
Sample HSI / HFE Assessment
What are some HSI (and

Safety) Issues with this
Control Station?

Slide 66

Reliability
Maintainability
Supportability
Human Factors
Safety
Security
Slide 67
Safety Engineering
System Safety is the application of engineering and
management principles, criteria, and techniques to optimize
all aspects of safety within the constraints of operational
effectiveness, time, cost, and risks throughout all phases of
the system life cycle.

Slide 68
Safety Tasks
Requirements Analysis
Program Tailoring
Program Plan
System Safety Program Plan (SSPP)

Hazardous Material Management Plan (HMMP)
Checklists, Hazard Tracking & Mitigations
Design Requirements and Guidelines (including COTS)

Test and Evaluation Safety (Plans)
Material Safety Data Sheets (MSDS)
Environmental Health & Safety (EHS, Facilities and
Environment)
E3 (Electromagnetic Environmental Effects, including Lasers)

Slide 69
Safety Design and Evaluation
Safety Design and Evaluation may Include:
Hazard Analyses, Identification and Tracking

Safety Verification and Validation
Identification of Safety Procedures
Identify Safety Critical Functions in both H/W and S/W
Hazard Mitigation(s) and Safety Testing
Commercial Off The Shelf (COTS) and Nationally Recognized
Test Laboratory (NRTL) Listings for Products & Components
Safety Assessment
Hazardous Material (Hazmat) Identification and Traceability
(generally for Maintenance, Demilitarization / Teardown, and
Disposal)
References: MIL-STD-882, ANSI / GEIA-Std-0010, UL / EN
Standards (i.e. UL/EN 60950)
Slide 70
Simple Safety Evaluation
Have a look around this

presentation room.
What do you see that might
be either a hazard or a
mitigation for a hazard?
Slide 71
Hardware Mishap Severity Categories

Slide 72
Hardware Mishap Probability of Occurrence

Slide 73
Typical Hardware Safety Mishap Risk Index Matrix

Slide 74
What About
SOFTWARE
and
SAFETY?
Slide 75

Slide 76
Typical Software Safety Criticality Indices
Software Criticality Indices

Slide 77
Typical Software Safety Criticality Matrix

, Continued

Slide 78
Typical Software Safety Risk Index Matrix

Slide 79
Hardware & Software Complexities have

led to a New Safety Analysis Technique
A Systems Safety Analysis Technique
For COMPLEX & AUTONOMOUS SYSTEMS
Developed by Dr. Nancy Leveson, Ph.D.
Professor of Aeronautics, Astronautics & Engineering
Systems
Massachusetts Institute of Technology
SYSTEMS-THEORETIC ACCIDENT MODEL and PROCESS
[STAMP]
&
SYSTEMS-THEORETIC PROCESS ANALYSIS [STPA]
Slide 80
A Brief Overview of STAMP / STPA

Control processes operate between levels of
control.
Controller
Model of
Process
Control
Actions
Feedback
[REALITY TRUMPS THE MODEL!]

Mishaps occur when model of
process is inconsistent with real
state of process, and the controller
provides inadequate control actions
for the real state.
Feedback channels are critical
-- Design
-- Operation
Controlled Process

Slide 81

In its simplest incarnation, a system is a closed loop
function. relating inputs, activation, control(s) and
corrective feedback as detailed in the following diagram.
A Standard System
Control Loop
With Feedback
Control Algorithms
Set Points
Controller
Actuators
Sensors
Controlled
Variables
Process
Inputs
Measured
Variables
Controlled
Process
Process
Outputs
Disturbances

Slide 82

Basic Steps to evaluate the requirements for
Controller-Action Response roles or sequences for the
system:
1) Action required but not provided causes hazard
2) Unsafe action provided incorrectly causes hazard
3) Incorrect Timing / Order
4) Stopped too soon or Applied too long

Slide 83

Identification
of Potential
Control Flaws
Inappropriate,
ineffective, or
missing
control action
Control input or
external
information wrong
or missing
Controller
Inadequate Control
Algorithm
(Flaws in creation,
process changes,
incorrect modification or
adaptation)
Process
Model
(inconsistent,
incomplete, or
incorrect)
Inadequate
operation
Incorrect or no
information provided
Delayed
operation
Conflicting control actions

Process input missing or wrong
Inadequate or
missing
feedback
Feedback
Sensor Delays
Inadequate
operation
Actuator
Controller
Missing or wrong
communication
with another
Controller
controller
Controlled Process
Component
failures
Changes over time

Unidentified or
out-of-range
disturbance
Measurement
inaccuracies
Feedback delays
Process output
contributes to
system hazard
Diagram 2011, N.Leveson

Used with permission

Slide 84

System Requirements must provide comprehensive,
validatable, and verifiable attributes for every step
in a system life cycle, including but not limited to:
Concept
Design
Development
Parts Selection & Procurement
Manufacture
Operation
Operational Environments
Deployment
Field Maintenance
Logistics Support
Upgrades & Updates
End-Of-Life and Disposal
Slide 85
Development and review of such extensive

requirements, covering such cradle to grave
system performance requires a much more
comprehensive assessment methodology for
complex systems.
From a Systems Theory perspective, a different but
effective methodology for a more comprehensive
safety hazard analysis assessment emerges.

Slide 86

(The) Value of system safety is
doing what engineers do not do. A
different viewpoint.
For more detailed information,
please consult Dr. Levesons
textbook on the topic.
Engineering a Safer World
Systems Thinking Applied to Safety
2011 MIT Press
A Google Search on STAMP / STPA turns up a

considerable amount of information and papers.
Slide 87

For a tutorial paper presented at the 2011 ISSC by
Drs. Leveson and Thomas, A Google Search for
STAMP STPA turns up a paper at the following
System Safety Society site:
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=
7&cad=rja&ved=0CE8QFjAG&url=http%3A%2F%2Fwww.systemsafety.org%2Fconferences%2F2011%2Fpapers%2FPerforming%2520Hazard
%2520Analysis%2520on%2520Complex%2C%2520Software%2520and%2520Human-Intensive%2520Systems.pdf&ei=AjWUeOZNqeBygH1g4HYDw&usg=AFQjCNG4kFq6Pksrz7FgowNAsJihE82_NA&
sig2=RNNn5HhptK-ddKEkehCjiA
Dr. Thomas has also written a primer on how to

perform a STAMP analysis, and has created some
online tools to assist in the process.
Slide 88

Dr. Leveson has also alerted folks to the creation of a new
Consortium on Systems Approaches to Safety

and Security (C-SASS).
For further information on STAMP/ STPA and C-SASS, contact
Dr. Leveson:
Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142
Telephone: 617-258-0505
Email: leveson@mit.edu
URL: http://sunnyday.mit.edu
Slide 89
Why Safety Matters

Safetys goals are to eliminate or mitigate hazard probabilities
and severities in the following order of precedence:
1. Design the hazard out (eliminate hazards, no probability of

occurrence)
2. Incorporate safety devices (protect against potential hazards,
protects against occurrence)
3. Provide warning devices (alert to potential hazards, try to reduce
probability of occurrence through human error)
4. Develop procedures and training (train personnel to avoid
hazards, try to reduce probability of occurrence through human
error)
Those hazards that cannot be completely eliminated are known

as Residual Risk
By far the most difficult risk to manage is human error.
Slide 90

Reliability
Maintainability
Supportability
Human Factors
Safety
Security
Slide 91
Security Engineering
Security Engineers design, procure, implement, build and test
security components of systems to prevent espionage. Such
espionage can be both international and inter-corporate (to
say nothing of the NSA activities!).

Slide 92
Security & Information Assurance
In the Headlines
What is Information Assurance (IA)?
Why is IA Important?
Acceptance of IA Requirement
Pursuit/Capture Considerations
IA Impact on Program
Security is Part of the Design
IA and MSI Pursuit, Capture and Execution

Slide 93
In the Headlines
The Wall Street Journal Computer Spies Breach Fighter-Jet
Project (external attack)
The Los Angeles Times Cyber-Attack Hits U.S. Central
Command (malware)
The San Francisco Chronicle Sabotage attacks knock out
phone service (physical threat)
WIRED
U.S. Intelligence Analyst Arrested in

Wikileaks Video Probe (insider threat)
Agence France-Presse Slick Cyber Attack Hits US Security Titan

RSA (external phishing attack)
Associated Press
Banks, Credit-Card Issuers Warn of e-mail

Breach! (external attack)
National Security Agency U.S. Intelligence Analyst Sought for

Release of Classified Data (insider threat)
Cyber attacks take many forms; e.g., digital, physical!
Slide 94
What is Information Assurance (IA)?

Per National Information Assurance (IA) Glossary
(CNSS Instruction No. 4009, Revised June 2006)
Basic Services
Measures that protect and defend

information and information systems by
ensuring their
Integrity
Confidentiality
Availability
Authentication
Basic Functions
These measures include providing for

restoration of information systems by
incorporating protection, detection, and
reaction capabilities
Protect
Detect
React
Non-repudiation

Slide 95
Why is IA Important?
Acting on bad informationwhether youre a soldier or system
can lead to deadly and / or major profit loss repercussions to say
nothing of company reputations.
This is a military
snapshot, but
the same holds
true for financial
institutions,
companies and
governments.
Complexity of todays
systems and networks
presents significant
security challenges for
producers and
consumers of
information
Mission Assurance
in networks and data!
Slide 96
Pursuit/Capture Considerations
Customer IA requirements are getting more specific in

Request for Proposals, Statement of Works, Performance
Specifications, etc.
Staff requirements getting more specific as well
Basis Of Estimate development must capture the
requirements
What happens if IA is ignored?
During the proposal:
Unable to bid due to inexperience
After submittal:
Non-compliant or weak bid
After winning contract:
Underestimate the task
Prior to deployment:
Cant gain approval to turn on
After deployment:
Vulnerabilities are exploited.
Thus putting many people and systems at risk, compromising
system effectiveness and damaging organizational reputations
Bake IA into the Solution
Slide 97
IA Impact on Program
Impact for incorporating IA into a typical program
Cost
Plan for it; Budget for it to control costs

Avoid Retrofitting later to comply with IA directives
Lots of Documentation
Certification and Accreditation
Reviews
Hardware (e.g., TEMPEST, Secure Guards)

Software (e.g., access control, authentication)
Encryption and Key Management
Schedule
Technical
Like all the -ilities, Security generally has to be built

into the system and not bolted on afterward.
Proper Planning Makes IA Affordable!
Slide 98
Security Tasks
Security Policies
Security Management
Physical Security
Risk Management
Disaster Recovery
Incident Response
Computer Forensics
Networks and Communications
Slide 99
Security Design and Evaluation

Security Engineers evaluate requirements based on:
Policy documents such as:
National Industrial Security Program Operating Manual (NISPOM,

DoD 5220.22-M)
Standards:
ISO-17799/BS-7799 Information Security Standard

Generally Accepted System Security Procedures (GASSP)
e.g., NIST Special Publication 800-14,
DCID
Best practices
Risk management concepts and practices
Slide 100
Summary
Think of Specialty Engineering as a Picture Puzzle, there
are various pieces that fit together to provide cradle to
grave engineering support for a system:
Reliability
&
Maintainability
Information
Assurance
ALL are typically Tasks supported through Quality

Engineering Concepts.
Slide 101
Summary
Specialty Engineers:
Provide Subject Matter Expert skills in special area(s)

Apply specialty expertise at successive levels of the system
Support concurrent engineering and account for cross dependencies
Assist and Guide Systems Engineers with in-depth knowledge and
experience in narrow specialties
Avoids the Systems Engineer needing to learn and become familiar in
depth with the Specialty Engineering subject matter
Early integration of Specialty Engineering:
Identifies many specialty risk areas

Reduces overall product life cycle costs through Concept, Design,
Development, Manufacture, Delivery, Deployment, Support and End
of Life Disposal
Slide 102
Questions? Comments?

Slide 103
Credits
Vanessa Battaglia, Raytheon Company
Gene Bridgers, ResultsMA (Retired)
Nancy Callaghan, Raytheon Company
Michael Castner, Raytheon Company (Retired)
Donald Chaloupka, Raytheon Company
David M. DeVarney, formerly with Raytheon Company
Anthony Demarinis, Sealed Air, Incorporated
Joseph Dzekevich, Raytheon Company (Retired)
Fredrick Grund, Raytheon Company
Louis Gullo, Raytheon Company
Avery Hevesh, Raytheon Company (Retired)
Milena Krasich, Raytheon Company
Jeffrey Mead, Raytheon Company (Retired)
Douglas Hunt, Raytheon Company
Paul Michael McCormick, formerly with Raytheon Company
Isadore Piaseczny, Jr., Lockheed-Martin Company (Retired)
Harvey Salzman, Centronics Corporation (Retired)
Robert Sprick, Raytheon Company
Ernest Ware, Raytheon Company
Jennifer Zhao, Raytheon Company
And countless other professors, teachers, professionals, managers, and peers across my career
who have provided insights, encouragement, patience, and golden keys to understanding
products from ideas and concepts, to design, development, manufacture, deployment, use,
disposal, and security.
Slide 104
Reference Sources:
American National Standards Institute (ANSI)
http://www.ansi.org/
Military Specifications and Standards (FREE!)
https://assist.daps.dla.mil/quicksearch/
Underwriters Laboratories
http://www.ul.com/global/eng/pages/
Any reputable bookstore (e.g. American Society for Quality,
Amazon.com, Barnes & Noble, etc.)
Jurans Quality Handbook, Sixth Edition
Dimitri Kececioglu texts and lectures on Reliability
(University of Arizona)
Dr. Nancy Leveson papers and textbooks on System
Safety, including Engineering a Safer World, Systems
Thinking Applied to Safety, 2011 MIT Press
Peter Druckers texts and lectures on business
management.
Slide 105
Thats Logistics Jingle Lyrics
[Intro - Guy]
Logistics makes the world work better
[Verse 1 - Nadia Ackerman]
When it's planes in the sky
For a chain of supply
That's logistics
When the parts for the line
Come precisely on time
That's logistics
A continuous link
That is always in sync
That's logistics
Carbon footprint's reduced
Bottom line gets a boost
That's logistics
With new ways to compete
Therell be cheers on Wall Street
That's logistics
When technology knows
Right where everything goes
That's logistics
Bells will ring, ring-a-ding
Ring-a-ding ring-a-ding
That's logistics
There will be no more stress
Cause you called ---That's logistics
Slide 106

81 Care About The Ilities PDF

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

81 Care About The Ilities PDF

Загружено:

Авторское право:

Доступные форматы

Why You Should Care About the -ilities!

Alan Southwick, CSP, CRE & CQE

What are the -ilities?

Quality, Quality Control, and Quality Engineering

Quality and Quality Control

Classic Definitions of Quality (Juran):

Copyright 2014 Raytheon Company. All rights reserved.

Quality and Quality Control

Classic Definition of Quality Control (Juran):

Copyright 2014 Raytheon Company. All rights reserved.

Interrelation Among Specialty Engineering

Copyright 2014 Raytheon Company. All rights reserved.

Quality Engineering (again, from Juran)

QUALITY OF MARKET RESEARCH

FITNES FOR USE

Myron Tribus Perversity Principle:

Copyright 2014 Raytheon Company. All rights reserved.

Ways That Quality Can Improve Things

Maintain Continuous Communication with the everyone on the

Ways That Quality Can Improve Things

[after Vilfredo Pareto (15 July 1848 19 August 1923)]

80% of your problems occur with 20% of your sources.

THE VITAL FEW AND TRIVIAL MANY

Copyright 2014 Raytheon Company. All rights reserved.

Quality, Quality Control, and Quality Engineering

Specialty Engineering Benefits

Why? Consider these Headlines

Do some of these sound familiar?

Percent of Life Cycle Cost Incurred

SY S TEM A C QU ISIT ION

Non-recurring (NR) costs

NR costs for disposal

Copyright 2014 Raytheon Company. All rights reserved.

Roles of Specialty Engineers

Reliability and maintainability / life cycle costs and logistics

Program Conceptualization and Capture

Specialty Engineers have roles in all phases of the systems

Copyright 2014 Raytheon Company. All rights reserved.

Design Influence and Interfaces

Through the system engineering process, identify items / processes that

Reliability, Availability, Maintainability (RAM)

Standardization and Interoperability

Environmental (e.g., air, waste, noise)

Corrosion & Non-Destructive Inspection

Hazardous Material (HAZMAT)

Human Factors, Human System Interface &

Relationship with Systems Engineering

Defining technical requirements for specialty engineering, with their assistance.

Where applicable, Specialty Engineers should be consulted for design

One Type of Risk Assessment Chart

Copyright 2014 Raytheon Company. All rights reserved.

Quality, Quality Control, and Quality Engineering

Reliability is the probability that an item will perform its

References: ANSI / GEIA-STD-0009, IEEE 1633

Reliability represents the percent of items that

Hardware Reliability Engineering

(Constant failure rate)

Availability of Service (Reliability is typically high, e.g. 99.9%)

Software Reliability Engineering

termination of a test, software improvements continue in

Copyright 2014 Raytheon Company. All rights reserved.

Software Reliability Engineering