DO Qualification Kit

Simulink Report Generator

Tool Qualification Plan
R2015b, September 2015

How to Contact MathWorks

Latest news:

www.mathworks.com

Sales and services:

www.mathworks.com/sales_and_services

User community:

www.mathworks.com/matlabcentral

Technical support:

www.mathworks.com/support/contact_us

Phone:

508-647-7000

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA 01760-2098
DO Qualification Kit: Simulink Report Generator Tool Qualification Plan

COPYRIGHT 20102015 by The MathWorks, Inc.

The software described in this document is furnished under a license agreement. The software may be used or copied only under
the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written
consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the
federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees
that this software or documentation qualifies as commercial computer software or commercial computer software documentation
as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and
conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification,
reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or
other entity acquiring for or through the federal government)and shall supersede any conflicting contractual terms or conditions.
If this License fails to meet the governments needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a
list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective
holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for more
information.

Revision History
September 2010
April 2011
September 2011
March 2012
September 2012
March 2013
September 2013
March 2014
October 2014
March 2015
September 2015

New for Version 1.3 (Applies to Release 2010b)

Revised for Version 1.4 (Applies to Release 2011a)
Revised for Version 1.5 (Applies to Release 2011b)
Revised for Version 1.6 (Applies to Release 2012a)
Revised for Version 2.0 (Applies to Release 2012b)
Revised for Version 2.1 (Applies to Release 2013a)
Revised for Version 2.2 (Applies to Release 2013b)
Revised for Version 2.3 (Applies to Release 2014a)
Revised for Version 2.4 (Applies to Release 2014b)
Revised for Version 2.5 (Applies to Release 2015a)
Revised for DO Qualification Kit Version 3.0 (Applies to Release 2015b)

Contents
1 Introduction ...................................................................................................................................... 1-1
2 Tool Overview and Identification .................................................................................................... 2-1
2.1 Simulink Report Generator Product Description..................................................................... 2-2
2.2 Simulink Report Generator Product Identifier ........................................................................ 2-2
3 Tool Operational Requirements ....................................................................................................... 3-1
4 Certification Considerations ............................................................................................................. 4-1
4.1 Requirements for Qualification ............................................................................................... 4-2
4.2 Certification Credit .................................................................................................................. 4-4
5 Tool Development Life Cycle Tool User ...................................................................................... 5-1
5.1 Planning................................................................................................................................... 5-2
5.2 Requirements........................................................................................................................... 5-3
5.3 Verification ............................................................................................................................. 5-4
6 Additional Considerations ................................................................................................................ 6-1
6.1 Customer Bug Reporting ......................................................................................................... 6-2
7 Tool Life Cycle Data ........................................................................................................................ 7-1
8 Schedule ........................................................................................................................................... 8-1

1 Introduction
This document comprises the Tool Qualification Plan (reference DO-330 Section 10.1.2) for the
following capability of the Simulink Report Generator verification tool:

System Design Description (SDD) report

Model XML Comparison report

This document is intended for use in the DO-178C and DO-330 tool qualification process for
Criteria 3 TQL-5 tools.
See also the DO Qualification Kit User's Guide, R2015b.

1-2

2 Tool Overview and Identification

2.1 Simulink Report Generator Product Description

Design and generate reports from models and simulations

Simulink Report Generator lets you design and generate richly formatted Microsoft Word,
HTML, and PDF reports from Simulink models and simulations. The report generator lets you
automatically create artifacts for Model-Based Design, such as system design descriptions and
generated code, requirements traceability, and testing reports. You can publish these artifacts in
an interactive web format that colleagues can use without opening the model. You can compare
Simulink models, review comparison results in an interactive XML report, and merge model
differences.
Simulink Report Generator produces artifacts for DO-178, ISO 26262, IEC 61508, and related
industry standards.
Key Features

Automatic capture of simulation results and model specifications

Report formatting based on Word and HTML report templates
Interactive reports for viewing models, generated code, and analysis results in web
browsers
Report designer for creating custom Word, HTML, PDF, RTF, and XML reports
Artifacts for DO-178, IEC 61508, and ISO 26262, including system design, model and
code verification, and requirements documentation
API for forms-based Word and HTML report generation
Model differencing and merging with XML comparison tool

2.2 Simulink Report Generator Product Identifier

Software Tool

Version (Release)

Tool Vendor

Simulink Report Generator

DO Qualification Kit

Version 4.2 (R2015b)

Version 3.0 (R2015b)

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA, 01760-2098 USA

2-2

3 Tool Operational Requirements

The Tool Operational Requirements for the Simulink Report Generator System Design
Description and Model XML Comparison reports are documented in:
Simulink Report Generator Tool Operational Requirements
To access the tool operational requirement document, on the MATLAB command line, type
qualkitdo to open the Artifacts Explorer. The document is in Simulink Report Generator.

3-2

4 Certification Considerations
This section provides certification considerations for the following capability of the Simulink
Report Generator verification tool:

System Design Description (SDD) report

Model XML Comparison report

4.1 Requirements for Qualification

To determine whether a tool must be qualified, you must answer the following questions. If you
answer yes to all three questions, you must qualify the tool.
Question

SDD Report Model XML Comparison Report

Can the tool insert an error into the airborne

Yes1
software or fail to detect an existing error in the
software within the scope of its intended usage?
Will the output of the tool not be verified as
Yes
specified in Section 6 of DO-178C, DO-278A,
DO-331, DO-332 or DO-333?
Are processes of DO-178C, DO-278A, DO-331, Yes
DO-332 or DO-333 eliminated, reduced, or
automated by the use of the tool? Will you use
output from the tool to meet an objective or
replace an objective of DO-178C, DO-278A,
DO-331, DO-332 or DO-333, Annex A or
Annex C?
1 The

Yes1

Yes

Yes

System Design Description report or Model XML Comparison report might fail to detect an error.

Given that the answer to all the preceding questions is yes, the Simulink Report Generator
System Design Description and Model XML Comparison reports must be qualified.
To determine the qualification type (Criteria 1, Criteria 2, or Criteria 3), you must answer the
following questions about the tool.
Question
SDD Report Model XML Comparison Report
1. Is the tool output part of the airborne
No
No
software, such that the output can insert
an error into the software?
2. Could the tool fail to detect an error in
No
No
the airborne software and is the tool also
used to justify the elimination or
reduction of either of the following:

Verification processes other than

that automated by the tool.

3.

Development processes that could

have an impact on the airborne
software.

Could the tool fail to detect an error in

the airborne software?

Yes

Yes

4-2

Because the answers to the first and second questions are no, the Simulink Report Generator
System Design Description and Model XML Comparison reports must be qualified as a Criteria
3 tool.

4-3

4.2 Certification Credit

The following table shows the certification credit (see DO-331 Annex A or Annex C
Objectives), being sought for the Simulink Report Generator System Design Description and the
Model XML Comparison reports. The Model XML Comparison report allows the user to do
incremental reviews of the model based on model changes, instead of reviewing all changes to a
model.

Note You can generate summary and detailed versions of the System Design
Description. The detailed version is a superset of the summary version. The DO
Qualification Kit covers the detailed version only.

Certification Credit for Simulink Report Generator System Design Description and
Model XML Comparison Reports
Annex A
or C
Table
Table
MB.A-3
MB.C-3

Objective

DO-331
Reference

Software high-level
requirements comply
with system
requirements.

Section
MB.6.3.1.a

Table
MB.A-3
MB.C-3

High-level
requirements are
accurate and
consistent.

Section
MB.6.3.1.b

Table
MB.A-3
MB.C-3

High-level
Section
requirements are
MB.6.3.1.c
compatible with target
computer.

Table
MB.A-3
MB.C-3

High-level
requirements are
verifiable.

Section
MB.6.3.1.d

Software or
Assurance
Levels
A, B, C, D
AL1, AL2, AL3,
AL4, AL5

Credit Taken
(in conjunction with other tools)

Full or Partial1 A review of the System

Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and simulation of the model
may be used.
Full or Partial1 A review of the System
A, B, C, D
AL1, AL2, AL3, Design Description or Model XML
Comparison report may be used to
AL4, AL5
satisfy this objective, or a combination
of review, Model Advisor checks and
simulation of the model may be used.
Full or Partial1 A review of the System
A, B
Design Description or Model XML
AL1, AL2
Comparison report may be used to
satisfy this objective, or a combination
of review and Model Advisor checks
may be used.
Full or None1 A review of the System
A, B, C
AL1, AL2, AL3, Design Description or Model XML
Comparison report may be used to
AL4
satisfy this objective, or simulation and
Model Coverage analysis may be used.

4-4

Annex A
or C
Table
Table
MB.A-3
MB.C-3

Objective

DO-331
Reference

High-level
Section
requirements conform MB.6.3.1.e
to standards.

Software or
Assurance
Levels
A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-3
MB.C-3

High-level
requirements are
traceable to system
requirements.

Section
MB.6.3.1.f

A, B, C, D
AL1, AL2, AL3,
AL4, AL5

Table
MB.A-3
MB.C-3

Algorithms are
accurate.

Section
MB.6.3.1.g

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Software low-level
requirements comply
with high-level
requirements.

Section
MB.6.3.2.a

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Low-level
requirements are
accurate and
consistent.

Section
MB.6.3.2.b

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Low-level
Section
requirements are
MB.6.3.2.c
compatible with target
computer.

A, B
AL1, AL2

Table
MB.A-4
MB.C-4

Low-level
requirements are
verifiable.

A, B
AL1, AL2

Section
MB.6.3.2.d

Credit Taken
(in conjunction with other tools)
Full or Partial1 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and Model Advisor checks
may be used.
Full1 A review of the System Design
Description reports Requirements
Traceability section may be used to fully
satisfy this objective. For incremental
changes, the Model XML Comparison
report will identify modified traceability
links.
Full or Partial1 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review, Model Advisor checks and
simulation of the model may be used.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and simulation of the model
may be used.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review, Model Advisor checks and
simulation of the model may be used.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and Model Advisor checks
may be used.
Full or None2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or simulation and
Model Coverage analysis may be used.

4-5

Annex A
or C
Table
Table
MB.A-4
MB.C-4

Objective

DO-331
Reference

Low-level
Section
requirements conform MB.6.3.2.e
to standards.

Software or
Assurance
Levels
A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Low-level
Section
requirements are
MB.6.3.2.f
traceable to high-level
requirements.

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Algorithms are
accurate.

Section
MB.6.3.2.g

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Software architecture Section

is compatible with
MB.6.3.3.a
high-level
requirements.

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Software architecture Section

is consistent.
MB.6.3.3.b

A, B, C
AL1, AL2, AL3,
AL4

Table
MB.A-4
MB.C-4

Software architecture Section

is compatible with
MB.6.3.3.c
target computer.

A, B
AL1, AL2

Table
MB.A-4
MB.C-4

Software architecture Section

is verifiable.
MB.6.3.3.d

A, B
AL1, AL2

Credit Taken
(in conjunction with other tools)
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and Model Advisor checks
may be used.
Full2 A review of the System Design
Description reports Requirements
Traceability section may be used to fully
satisfy this objective. For incremental
changes, the Model XML Comparison
report will identify modified traceability
links.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review, Model Advisor checks and
simulation of the model may be used.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and simulation of the model
may be used.
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review, Model Advisor checks and
simulation of the model may be used.
Full2 A review of the System Design
Description or Model XML Comparison
report may be used to satisfy this
objective.
Full or None2 A review of the
System Design Description or Model
XML Comparison report may be used to
satisfy this objective or simulation and
Model Coverage analysis may be used.

4-6

Annex A
or C
Table
Table
MB.A-4
MB.C-4

Objective

DO-331
Reference

Software architecture Section

conforms to
MB.6.3.3.e
standards.

Software or
Assurance
Levels
A, B, C
AL1, AL2, AL3,
AL4

Credit Taken
(in conjunction with other tools)
Full or Partial2 A review of the System
Design Description or Model XML
Comparison report may be used to
satisfy this objective, or a combination
of review and Model Advisor checks
may be used.

Notes:
1
This credit is taken only if the Simulink and Stateflow models are considered Specification Models for the project.
2
This credit is taken only if the Simulink and Stateflow models are considered Design Models for the project.

4-7

4-8

5 Tool Development Life Cycle

Tool User

5.1 Planning
The Plan for Software Aspects of Certification (PSAC) or Plan for Software Aspects of
Approval (PSAA) designates that the Simulink Report Generator System Design Description
(SDD) and Model XML Comparison reports will be qualified as a Criteria 3 tool.
This document provides the Tool Qualification Plan for the Simulink Report Generator SDD
and Model XML Comparison reports.

5-2

5.2 Requirements

Tool Operational Requirements for the Simulink Report Generator System Design
Description (SDD) and Model XML Comparison reports are in:
Simulink Report Generator Tool Operational Requirements

To access the tool operational requirement document, on the MATLAB command line,
type qualkitdo to open the Artifacts Explorer. The document is in Simulink Report
Generator.

The applicant will:

-

Review the Tool Operational Requirements for applicability to the project under
consideration.
Configure the Tool Operational Requirements in a configuration management system.

User information for the Simulink Report Generator SDD report is available in Simulink
Report Generator System Design Description User Information. To access the user
information document, on the MATLAB command line, type qualkitdo to open the
Artifact Explorer. The document is in Simulink Report Generator.

User information for the Simulink Report Generator Model XML Comparison report is
available in Simulink Report Generator Model XML File Comparison User Information.
To access the user information document, on the MATLAB command line, type
qualkitdo to open the Artifact Explorer. The document is in Simulink Report
Generator.

Instructions for installing the Simulink Report Generator product are available at the
MathWorks Documentation Center, R2015b:
Installation

5-3

5.3 Verification
Requirements-based test cases and procedures will be developed from the:
Simulink Report Generator Tool Operational Requirements
The test cases and procedures will be developed in the form of the Simulink models that
exercise the System Design Description (SDD) and Model XML Comparison reports. The test
cases and procedures will be scheduled using the report files
qualkitdo_xmlcomp_rs.rpt and qualkitdo_rptgenext_tcp.rpt.
The test cases and procedures are documented in:
Simulink Report Generator Test Cases and Procedures
To access the documents, on the MATLAB command line, type qualkitdo to open the
Artifacts Explorer. The documents are in Simulink Report Generator.
The applicant will:

Review the test cases and procedures for applicability to the project under consideration.
Configure the test cases and procedures in a configuration management system.
Execute the test cases and procedures in the installed environment.

5-4

Executing the Simulink Report Generator report files listed in the following table generates
tool verification results in the specified test reports.

Report File
qualkitdo_rptgenext_tcp.rpt

Test Reports
1

qualkitdo_xmlcomp_rs.rpt

qualkitdo_rptgenext_qualificationreport.pdf2
qualkitdo_rptgenext_qualificationreport_model.pdf3
qualkitdo_rptgenext_qualificationreport_model.doc3
qualkitdo_rptgenext_qualificationreport_model.rtf3
qualkitdo_rptgenext_qualificationreport_model.html3
qualkitdo_xmlcomp_qualificationreport.html

Notes:
1
Requires Stateflow and Simulink Verification and Validation licenses.
2
Overview Report.
3
Model level report.

In the above listed test reports, model stands for:

slvnvdemo_fuelsys_docreq,

slvnvdemo_fulesys_htmlreq,

sldemo_mdlref_basic,

SDD_cont,

SDD_discont,

SDD_discrete,

SDD_eml_call_gf,

SDD_logic,

SDD_lookup,

SDD_math,

SDD_other,

SDD_SF_ArithLogic,

5-5

SDD_SF_data_check,

SDD_SF_event,

SDD_SF_exit_substate,

SDD_SF_model,

SDD_signals,

SDD_sources,

SDD_subsys,

SDD_truth_table_model,

SDD_TT_Block,

SDD_Version_Description_Test.

The applicant will:

Review the test results, confirm they are correct.

Explain discrepancies in the test results, if any.
Configure the test results in a configuration management system.

5-6

6 Additional Considerations

6.1 Customer Bug Reporting

MathWorks reports known critical bugs brought to its attention on its bug report system at
www.mathworks.com/support/bugreports. The bug reports are an integral part of the
documentation for each release.
The bug report system provides an interface for customers to view and submit bug reports. Users
can track the status of open bugs. Users can choose to receive notifications for new or updated
bug reports. The bug reports on this web site include internally and externally nominated bugs.
If applicable, bug reports include provisions for known workarounds or file replacements.
Customers can use the bug report mechanism to nominate bugs. These nominations are
processed and evaluated by The MathWorks, Inc. development organization.

6-2

7 Tool Life Cycle Data

The following table shows the life cycle data for the Simulink Report Generator System
Design Description and Model XML Comparison reports. The table maps the documents and
artifacts to DO-330 life cycle data items.
Simulink Report Generator System Design Description and Model XML
Comparison Reports Life Cycle Data
Data
Plan for Software Aspects
of Certification (PSAC) or
Plan for Software Aspects
of Approval (PSAA)
Tool Qualification Plan
Tool Operational
Requirements

Available/
Submit
Submit

DO-330
Documents/Artifacts
Reference
Section
<Insert PSAC or PSAA** reference here.>
10.1.1

Submit*

Section
10.1.2
Section
10.3.1

Available

Simulink Report Generator Tool

Qualification Plan (this document)
Simulink Report Generator Tool
Operational Requirements
Simulink Report Generator System Design
Description User Information
Simulink Report Generator Model XML File
Comparison User Information

Test Cases and Procedures

Available

Test Results

Available

Software Accomplishment
Summary (SAS)

Submit

Section
10.3.3
Section
10.3.4

qualkitdo_rptgenext_tcp.rpt
qualkitdo_xmlcomp_rs.rpt

Section
10.1.16

<Insert reference to SAS** here.>

qualkitdo_rptgenext_
qualificationreport.pdf
qualkitdo_rptgenext_
qualificationreport_model.pdf
qualkitdo_rptgenext_
qualificationreport_model.doc
qualkitdo_rptgenext_
qualificationreport_model.rtf
qualkitdo_rptgenext_
qualificationreport_model.html
qualkitdo_xmlcomp_qualificationrep
ort.html

Data
Tool Qualification
Accomplishment Summary

Available/
Submit
Submit*

DO-330
Documents/Artifacts
Reference
Section
<Insert reference to Tool Qualification
10.1.15
Accomplishment Summary** here.>

Notes:
* Optional for TQL-5 tool qualification
** To be created by applicant

The applicant must deliver data marked Submit to the certification authorities. Data marked
Available must be available at the applicants or tool vendors site for inspection by the
certification authorities.

7-2

8 Schedule
<Insert tool schedule in this section.>