DO Qualification Kit

Simulink Test
Tool Operational Requirements
R2015b, September 2015

How to Contact MathWorks

Latest news:

www.mathworks.com

Sales and services:

www.mathworks.com/sales_and_services

User community:

www.mathworks.com/MATLABcentral

Technical support:

www.mathworks.com/support/contact_us

Phone:

508-647-7000

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA 01760-2098
DO Qualification Kit: Simulink Test Tool Operational Requirements

COPYRIGHT 2015 by The MathWorks, Inc.

The software described in this document is furnished under a license agreement. The software may be used or copied only under
the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written
consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the
federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees
that this software or documentation qualifies as commercial computer software or commercial computer software documentation
as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and
conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification,
reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or
other entity acquiring for or through the federal government)and shall supersede any conflicting contractual terms or conditions.
If this License fails to meet the governments needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a
list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective
holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for more
information.

Revision History
September 2015

New for DO Qualification Kit Version 3.0 (Applies to Release 2015b)

Contents
1 Introduction ...................................................................................................................................... 1-1
1.1 Simulink Test Product Description ......................................................................................... 1-3
2 Operational Requirements ................................................................................................................ 2-1
2.1 Assessment of Simulation and Test Results ............................................................................ 2-2
2.2 Reporting of Simulation and Test Results ............................................................................... 2-5
2.3 Tool Interfaces......................................................................................................................... 2-7
3 Simulink Test User Information ....................................................................................................... 3-1
4 Installation ........................................................................................................................................ 4-1
5 Operational Environment ................................................................................................................. 5-1

vi

1 Introduction
This document comprises the Tool Operational Requirements (Reference DO-330 Section
10.3.1) for the following capabilities of the Simulink Test verification tool:

Comparing simulation results or executable object code test results against expected results
to determine pass/fail simulation or test status.
Generating reports that contain simulation or test results including information on pass/fail
simulation or test status, and requirements tracing.

The Operational Requirements section of this document has the Simulink Test operational
requirements. The requirements are in a tabular form and tagged with unique identifiers that can
be used to trace to the associated tool test cases, procedures and results. Text not included in the
tables does not represent the requirement and is intended to provide supporting information.
The Simulink Test User Information section of this documents identifies the user information
for Simulink Test.
The Installation section of this document specifies the Simulink Test installation procedure.
The Operational Environment section of this document identifies the operational environment in
which the Simulink Test product shall be installed.
This document is intended for use in the DO-330 tool qualification process for TQL-5 tools. The
applicant will:

Review the Tool Operational Requirements for applicability in the project or program
under consideration.
Complete the Tool Operational Requirements with project specific information if
necessary.
Configure the Tool Operational Requirements in the project or program configuration
management system making the document available for certification reviews.

See documentation for the following products at the MathWorks Documentation Center,
R2015b:

DO Qualification Kit (for DO-178)

Simulink Test

1-2

1.1 Simulink Test Product Description

Develop, manage, and execute simulation-based tests

Simulink Test provides tools for authoring, managing, and executing systematic, simulationbased tests. You can create nonintrusive test harnesses to test models and subsystems. Simulink
Test includes a test sequence block that lets you construct complex test sequences and
assessments, and a test manager that lets you manage and execute tests. It enables functional,
baseline, equivalence, and back-to-back testing, including software-in-the-loop (SIL) and
processor-in-the-loop (PIL). You can generate reports, archive and review test results, rerun
failed tests, and debug the component or system under test.
The test harness in Simulink Test lets you test components without creating a separate test
model. You can apply pass and fail criteria that include absolute and relative tolerances, limits,
logical checks, and temporal conditions. Test execution can be automated or customized with
setup and cleanup scripts. Simulink Test stores test cases and their results, creating a repository
for reviewing and investigating failures. You can link requirements to a test case captured in
Microsoft Word, IBM Rational DOORS, and other documents (with Simulink
Verification and Validation).
Key Features

Test harness for subsystem or model testing

Test sequence block for running tests and assessments
Pass-fail criteria, including tolerances, limits, and temporal conditions
Baseline, equivalence, and back-to-back testing
Setup and cleanup scripts for customizing test execution
Test manager for authoring, executing and organizing test cases and their results
Automatic report generation to document test outcomes

For the Simulink Test features that require qualification, see the certification considerations
provided in the Simulink Test Tool Qualification Plan.
Section 2 of this document provides the operational requirements for the features to be qualified.
Figure 1 shows Simulink Test workflow, including three options for developing expected
results, including the inputs and outputs.

1-3

Figure 1: Simulink Test Workflow

1-4

2 Operational Requirements
The following sections provide the Simulink Test capabilities that are supported by the DO
Qualification Kit. The user is responsible for ensuring that the tool features they rely on to
eliminate, reduce or automate the process are sufficiently covered by Tool Operational
Requirements (reference DO-300 Section 6.2.1.aa).

2.1 Assessment of Simulation and Test Results

Requirement ID

Requirements

SLT_ASR_001

Simulink Test shall provide the capability to compare actual simulation results and
executable object code test results to expected results.

SLT_ASR_010

Simulink Test shall determine if the difference between the expected and actual
results of simulation and object code test execution is within (less than or equal to)
a specified absolute tolerance and report the corresponding pass or fail outcome.

SLT_ASR_020

Simulink Test shall determine if the difference between the expected and actual
results of simulation and object code test execution is within (less than or equal to)
a specified relative tolerance and report the corresponding pass or fail outcome.

SLT_ASR_030

If both absolute and relative tolerances are specified, Simulink Test shall apply the
maximum of the two values, based on the expected results value.

SLT_ASR_050

Simulink Test shall provide the capability to compare data of the same data types;
by using the following conversions:
Fixed-point data is converted to double before comparison.
Integer data is converted to double before comparison.
Enumeration data is converted to int32 before comparison.

SLT_ASR_051

Simulink Test shall support comparison of the data of type double.

SLT_ASR_052

Simulink Test shall support comparison of the data of type single.

SLT_ASR_053

Simulink Test shall support comparison of the data of type int8 and uint8.

SLT_ASR_054

Simulink Test shall support comparison of the data of type int16 and uint16.

SLT_ASR_055

Simulink Test shall support comparison of the data of type int32 and uint32.

SLT_ASR_056

Simulink Test shall support comparison of the data of type fixed-point.

SLT_ASR_057

Simulink Test shall support comparison of enumerations.

SLT_ASR_060

Simulink Test shall provide the capability to align time vectors between
corresponding signals of expected results and actual results using the following
synchronization methods:
Union (expected and captured time vectors are merged)
Intersection of expected and captured time vectors

2-2

Requirement ID

Requirements

SLT_ASR_070

Simulink Test shall provide the capability to align data vectors between
corresponding signals of expected and actual results by applying the following
methods of interpolation:
ZOH MATLAB interpolation. By default, applies to discrete signals.
Linear MATLAB interpolation. By default, applies to continuous
signals.

SLT_ASR_080

Simulink Test shall provide the capability to compare the simulation and object
code test results, provided as input signals to the Test Sequence Block, to expected
conditions using the pass/fail conditions specified by the assert MATLAB
construct and when Simulink Test construct for supported logical, relational and
arithmetic operators.

SLT_ASR_090

For assert construct, Simulink Test shall support conditions specified by

arguments of the same data type;

SLT_ASR_091

For assert condition input arguments, Simulink Test shall support the data of type
double.

SLT_ASR_092

For assert condition input arguments, Simulink Test shall support data of type
single.

SLT_ASR_093

For assert condition input arguments, Simulink Test shall support data of type int8
and uint8.

SLT_ASR_094

For assert condition input arguments, Simulink Test shall support data of type
int16 and uint16.

SLT_ASR_095

For assert condition input arguments, Simulink Test shall support data of type
int32 and uint32.

SLT_ASR_096

For assert condition input arguments, Simulink Test shall support data of type
fixed-point.

SLT_ASR_097

For assert condition input arguments, Simulink Test shall support data of type
enumeration.

SLT_ASR_100

For assert conditions, Simulink Test shall support the logical operator AND.

SLT_ASR_101

For assert conditions, Simulink Test shall support the logical operator OR.

SLT_ASR_102

For assert conditions, Simulink Test shall support the logical operator NOT.

SLT_ASR_103

For assert conditions, Simulink Test shall support the logical operator XOR.

2-3

Requirement ID

Requirements

SLT_ASR_110

For assert conditions, Simulink Test shall support the relational operator of
equality (==).

SLT_ASR_111

For assert conditions, Simulink Test shall support the relational operator of
inequality (~=).

SLT_ASR_112

For assert conditions, Simulink Test shall support the relational operator greater
than (>).

SLT_ASR_113

For assert conditions, Simulink Test shall support the relational operator greater
than or equal (>=).

SLT_ASR_114

For assert conditions, Simulink Test shall support the relational operator less
than (<).

SLT_ASR_115

For assert conditions, Simulink Test shall support the relational operator less than
or equal (<=).

SLT_ASR_120

For assert conditions, Simulink Test shall support the scalar addition arithmetic
operator (+).

SLT_ASR_121

For assert conditions, Simulink Test shall support the scalar subtraction arithmetic
operator (-).

SLT_ASR_122

For assert conditions, Simulink Test shall support the scalar multiplication
arithmetic operator (*).

SLT_ASR_123

For assert conditions, Simulink Test shall support the scalar division arithmetic
operator (/).

SLT_ASR_124

For assert conditions, Simulink Test shall support the scalar power arithmetic
operator (^).

SLT_ASR_125

For assert conditions, Simulink Test shall support the scalar absolute value
operator (abs).

2-4

2.2 Reporting of Simulation and Test Results

Simulink Test provides a capability to generate reports with simulation and test results arranged
as per the requirements in the following table. Reports can also contain other auxiliary
information.
Requirement ID

Requirements

SLT_RSR_010

Simulink Test shall provide a capability to generate a report containing:

Title page
Report summary
Subsection(s) with summary results for each test suite
Subsection(s) with detailed information for each test case

SLT_RSR_020

The title page of the report shall include:

Title of the report
Author of the report
Date and time stamp of report generation
Test environment including target platform and MATLAB version

SLT_RSR_030

The report summary shall include the list of test files, test suites, and test cases
included in the report, along with the associated pass/fail status of each entry:
Outcome: total number of test cases, including number passed, failed
and disable
List of test files, test suites and test cases included in the report
Note: The test result summary is provided if there is more than one test case in
the report.

SLT_RSR_040

The report subsection with summary results for test suite shall include:
Header of subsection with name of the corresponding test suite
Parent item
Outcome: total number of test cases, including number passed, failed
and disabled

SLT_RSR_050

The report subsections with test case details shall include:

Header with name of the corresponding test case
Test result information
Test case information
Test case requirements (if traced to the test case)
Baseline comparison information (applies for baseline test cases)
Equivalence comparison information (applies for equivalence test
cases)
Model(s) simulation plot

2-5

Requirement ID

Requirements

SLT_RSR_060

The test result information for test case shall include:

Parent item
Test case start and end time stamp
Outcome: test case status (passed or failed or disabled)
Notes (if provided)

SLT_RSR_070

The test case information shall include:

Name of the test case
Type of the test case (baseline or equivalence or simulation)
Test case description (if provided)

SLT_RSR_080

The test case requirements information shall include the following, for each
requirement traced to the test case:
Description of requirement (if provided)
Hyperlink to the document which contain requirement

SLT_RSR_090

The baseline comparison information shall include:

Table containing a list of signals compared with associated absolute
and relative tolerance applied
Two plots for each signal compared: (1) plot of signal obtained during
simulation/test and its baseline and (2) plot of difference between
simulation/test result and baseline and tolerance applied

SLT_RSR_100

The equivalence comparison information shall include:

Table containing a list of signals compared with the associated applied
absolute and relative tolerances
Two plots for each signal compared: (1) plot of signals obtained during
1st and 2nd simulation/test and (2) plot of difference between 1st and
2nd simulation/test and tolerance applied

SLT_RSR_110

The model simulation information shall include for each model:

Name
Author
Save date
Version
Checksum
Simulation mode
Plot of simulation output (optional)

SLT_RSR_120

Simulink Test shall provide a capability to store generated reports in the

following formats:
PDF
DOCX
HTML, compressed to zip archive

2-6

2.3 Tool Interfaces

Requirement ID

Requirements

SLT_TIR_005

Simulink Test shall generate a warning or error message and set test outcome to
fail if comparison of actual simulation results to expected results cannot be
accomplished.
Simulink Test shall provide the capability to capture the outputs of model
simulation as Simulink.SimulationOutput objects and store the
outputs in a MAT-file to use as expected results for baseline tests. For
information about Simulink.SimulationOutput, see the Simulink
documentation at the MathWorks Documentation Center, R2015b.

SLT_TIR_010

SLT_TIR_011

Simulink Test shall provide a capability to load

Simulink.SimulationData.Dataset objects stored in MAT-files to
use as expected results for baseline tests. For information about
Simulink.SimulationData.Dataset, see the Simulink documentation
at the MathWorks Documentation Center, R2015b.

SLT_TIR_012

Simulink Test shall provide a capability to load

Simulink.SimulationOutput objects stored in MAT-files to use as
expected results for baseline tests.

SLT_TIR_013

Simulink Test shall provide a capability to use the outputs of Simulink model as
expected results in equivalence tests.

SLT_TIR_020

Simulink Test shall generate a warning message if the outputs of reference

model in equivalence tests cannot be captured and set test outcome to fail.

SLT_TIR_030

If expected results cannot be retrieved from the MAT-file specified in the test
case, Simulink Test shall generate an error message and set test outcome to fail.

SLT_TIR_051

For signals, if the expected results do not align with the actual results for
simulation and executable object code tests, Simulink Test shall label the signal
with a failure to align and set test outcome to fail.

2-7

2-8

3 Simulink Test User Information

For information about Simulink Test, see:

Simulink Test Users Guide, R2015b

Simulink Test Reference, R2015b

To access the documents, on the MATLAB command line, type qualkitdo to open the
Artifact Explorer. The document is in Simulink Test.

3-2

4 Installation
To use the Simulink Test product, install the following MathWorks products:
MATLAB
Simulink
Simulink Test
Instructions for installing the products are available at the MathWorks Documentation Center,
R2015b :
Installation

4-2

5 Operational Environment
Simulink Test qualification is supported in the following operational environments:
Personal computer
One of the following operating systems:

Microsoft Windows 32-bit

Microsoft Windows 64-bit
Linux 64-bit

MATLAB
Simulink
Simulink Test
MATLAB Report Generator
Simulink Report Generator