Вы находитесь на странице: 1из 84

Smart Metering Implementation Programme

Interface Testing Approach

Smart Metering Implementation Programme


(SMIP)
Interface Testing Approach Document

Version 1.0
03/10/2014

Page 1 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Document Location
The source of this printed document can be found in Smart DCCs Programme Management Office, or with the author if in an
unapproved status.

Associated Documents
This document is associated with the following other documents:

Ref

Title & Originators Reference

Source

Release
Date

Version

[1]

Glossary of Testing Terms

ISTQB

Oct 2012

v2.2

[2]
[3]
[4]
[5]
[6]
[7]

Joint DSP/CSP Test Strategy


Smart Energy Code Stage 3
DSP Contracts
CSP Contracts (North, Central and South)
Device Selection Methodology
CTSD/SREPTSD (Common Test Scenarios
Document/SMKI and Repository Entry Process
Test Scenarios Document)
Guide for Testing Participants
SEC Definitions (Section A1)
Code of Connection for DCC User Gateway
Interface
Code of Connection for Self Service Interface
Testing Issue Resolution and Disputes Process
DCC User Gateway Interface Specification
(DUGIS)

DSP
DECC
DCC
DCC
DCC
DCC

Feb 2014

v2.3

[8]
[9]
[10]
[11]
[12]
[13]

DCC
DECC
DSP
DSP
DCC
DSP

v .1
v .1
Consultation publication
Consultation publication

Aug 2014
Jan 2014
Sept 2014

Not published
SEC3
v0.9
v0.13
Not published
V0.8

Page 2 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Conten
Table of Figures..................................................................................................................... 7
1

Introduction..................................................................................................................... 8
1.1

Context...................................................................................................................... 8

1.2

Change Forecast.....................................................................................................10

1.3

Approvals and Appeals Process..............................................................................10

1.4

Terminology............................................................................................................. 11

Scope............................................................................................................................. 14
2.1

Overview.................................................................................................................14

2.2

Out of Scope............................................................................................................14

Objectives..................................................................................................................... 15

Test Stage Description.................................................................................................16


4.1

Introduction..............................................................................................................16

4.2

DSP-led testing........................................................................................................16

4.2.1

DCC User Message Gateway...........................................................................17

4.2.2

DCC Self Service Interface...............................................................................17

4.2.3

Additional Functionality Tests...........................................................................17

4.2.4

CSP Regions....................................................................................................17

4.2.5

Non-Functional Testing.....................................................................................17

4.3
5

UEPT....................................................................................................................... 18

Deliverables.................................................................................................................. 19
5.1

Responsibilities for Deliverables..............................................................................19

5.2

Notes on Deliverables..............................................................................................19

Test Procedure.............................................................................................................. 21
6.1

Requirements traceability........................................................................................21

6.2

Test Stage overlap...................................................................................................21

6.2.1

SP UAT Test Stage with Interface Test Stage...................................................21

6.2.2

Solution Test Stage with Interface Test Stage...................................................21


Page 3 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
6.2.3
6.3

Technical Readiness................................................................................................21

6.3.1

Service Providers.............................................................................................21

6.3.2

SEC Parties......................................................................................................22

6.3.3

RDPs................................................................................................................22

6.4

Interface Test Stage with End to End Test Stage..............................................21

Testing Principles.....................................................................................................22

6.4.1

Non-discrimination............................................................................................22

6.4.2

Use of Devices/Device stubs............................................................................22

6.4.3

Testing with one CSP.......................................................................................23

6.4.4

Testing by Region.............................................................................................23

6.5

Release management..............................................................................................23

6.6

Configuration management.....................................................................................23

6.7

Timetable.................................................................................................................24

6.7.1

IT Test Initiation................................................................................................24

6.7.2

IT Test Execution..............................................................................................27

6.7.3

IT Test Completion............................................................................................28

6.8

Dependencies..........................................................................................................30

6.9

Stage Entry and Exit................................................................................................31

6.9.1

Entry Criteria....................................................................................................31

6.9.2

Exit Criteria.......................................................................................................31

Environments, Networks and Test Labs.....................................................................33


7.1

Introduction..............................................................................................................33

7.2

DCC Enterprise environments.................................................................................33

7.3

DSP environment and network................................................................................33

7.4

CSP environments and networks.............................................................................33

7.5

TSP environment.....................................................................................................33

7.6

Parse & Correlate....................................................................................................33

7.7

SEC Parties.............................................................................................................33
Page 4 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
7.8

CSP Test Labs (for devices)....................................................................................34

7.9

De-selection and substitution of Devices.................................................................35

7.10

Summary of test facilities provided..........................................................................36

Test Data........................................................................................................................ 37
8.1

Test Data Plan......................................................................................................... 37

8.2

Test data principles..................................................................................................37

8.3

Responsibilities for data provision...........................................................................39

Testing Issue Management..........................................................................................40


9.1

Principles.................................................................................................................40

9.2

Testing Issue Management tool...............................................................................41

9.3

Issue Review Board.................................................................................................41

9.4

Testing Issue Severities and Priorities.....................................................................42

9.5

Testing Issues and exiting IT....................................................................................43

9.6

Testing Issue reporting.............................................................................................43

10

Test Tracking & Reporting........................................................................................45

10.1

Introduction..............................................................................................................45

10.2

Test Preparation Reporting......................................................................................45

10.2.1

General............................................................................................................. 45

10.2.2

Test Preparation Progress Report.....................................................................45

10.2.3

Test Readiness Report.....................................................................................46

10.3
11

Test Execution Reporting.........................................................................................46


Roles & Responsibilities..........................................................................................48

11.1

General.................................................................................................................... 48

11.2

Roles and Responsibilities.......................................................................................48

11.2.1

Roles of Test Participants.................................................................................48

11.2.2

Reporting structure...........................................................................................51

11.2.3

Roles of Test Management...............................................................................52

11.3

Governance.............................................................................................................52
Page 5 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
11.4

Communication........................................................................................................53

11.4.1

Progress meetings............................................................................................53

11.4.2

Testing Issues and their resolution....................................................................54

11.5

Test Assurance........................................................................................................54

11.5.1

Introduction.......................................................................................................54

11.5.2

Quality Gate Reviews.......................................................................................54

11.5.3

Test Witnessing................................................................................................55

11.5.4

Test Quality Audits............................................................................................56

11.5.5

Product inspections..........................................................................................56

11.5.6

Documentation review......................................................................................56

12

Operational Acceptance Testing..............................................................................57


Appendices..................................................................................................................... 58
A.

Test Scenarios......................................................................................................... 59

i.

Connectivity Test Scenarios........................................................................................59

ii.

Additional Functionality Test Scenarios.......................................................................61

Change of Supplier (On-Demand) Test Scenario for EIS SEC Party Role..........................62
Change of Supplier (On-Demand) Test Scenario for GIS SEC Party Role.........................64
Self Service Interface Test Scenarios.................................................................................66
Billing Test Scenario........................................................................................................... 69
Reporting Test Scenario.....................................................................................................70
Volume Test Scenarios.......................................................................................................71
B.

Documentation RTM................................................................................................73

C.

Functional Coverage RTM...................................................................................74

Page 6 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Table of FiguresY
Figure 1 DCC Solution

Figure 2 Programme Test Documentation Hierarchy

Figure 3 Interface Test Documentation Hierarchy

10

Figure 4 - Abbreviations

13

Figure 5 DCC system

16

Figure 6 RACI Matrix showing Deliverables

19

Figure 7 Timetable for test initiation

27

Figure 8 Timetable for test execution

28

Figure 9 Timetable for test completion

29

Figure 10 Milestones and their dependencies

31

Figure 11 Number of device sets

34

Figure 12 Summary of test facilities provided

36

Figure 13 High-level data types and responsibilities for provision

39

Figure 14 - Testing Issue Priorities

42

Figure 15 - Target testing issue response times

43

Figure 16 Outstanding testing issues and exiting IT

43

Figure 17 Test Preparation Reporting

45

Figure 18 Test Execution Reporting

46

Figure 19 Roles of participating Test Participants

50

Figure 20 IT main roles and structure

51

Figure 21 Roles of Test Management

52

Figure 22 - IT Governance

53

Figure 23 Test Stage Certificates

55

Figure 24 Document RTM

73

Figure 25 Functional coverage RTM

74

Page 7 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Introduction
1.1

Context

This document sets out the manner in which Interface Testing (IT) will be conducted for the
Smart Metering eco-system, which is depicted in the following diagram. Readers are
expected to be familiar with the Joint DSP/CSP Test Strategy1 (Reference [2]).

Figure 1 DCC Solution

This Interface Testing Approach is based on:


the Smart Energy Code (SEC) (Reference [3])
the Joint DSP/CSP Test Strategy (Reference [2])
the DSP and CSP Service Provider Contracts (References [4], [5])
a series of workshops held at the DCC during July September, attended by Service
Users, Service Providers, DECC, SECAS and Energy UK.
Evidence of this documents compliance with the Smart Energy Code can be found in
Appendix A.
The diagram below demonstrates how this Test Approach (enclosed within the red oval) fits in
with the overall hierarchy of test documents that will be produced for the programme.

1________________________ This document will be update in accordance with the


requirements that are set out in SEC3.
Page 8 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Figure 2 Programme Test Documentation Hierarchy

The following diagram illustrates the documentation set for Interface Testing. Note that the
documents in the diagram also apply to E2E Testing and Enduring Testing.

Page 9 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Figure 3 Interface Test Documentation Hierarchy

1.2

Change Forecast

This document will be assessed and, where applicable, updated by the DCC when approved
versions of the following documents are available:
Device Selection Methodology [6] (due mid-November);
Common Test Scenarios Document/SMKI and Repository Test Scenarios Document
(CTSD/SREPTSD) [7] (due end November);
Testing Issue Resolution and Disputes Process [12]; and
further versions of the Smart Energy Code, specifically SEC4.

1.3

Approvals and Appeals Process

Following consultation, the DCC will submit the document to the TAG for review prior to
submission to the SEC Panel for approval. This document will be presented to the SEC
Panel in timescales such that it can be published at least 6 months prior to the start of the
Interface Testing stage.
If the SEC Panel decides not to approve the IT Approach and the DCC is required to do rework, it will follow the process laid out in SEC T3.11.
If the SEC Panel decides to approve the IT Approach, the DCC will follow the process laid out
in SEC T3.12.

Page 10 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

1.4

Terminology

In this document the term Service Provider includes all of the following:
the DSP;
both CSPs;
the Trusted Service Provider (TSP), supplier of the SMKI solution element; and
the DCC Enterprise Service Provider (DCC Enterprise), i.e. the DCC in its role as supplier
of Enterprise systems such as Billing and BI/MI.
The term User Integration Testing (UIT) refers to the test phase that comprises Interface
Testing and End to End Testing.
The term Test Stub means systems and actions which simulate the behaviour of Devices
and User systems.
The term Testing Issue means in respect of any tests (a) anything that is preventing the
execution of the test or (b) once commenced or executed, the test has an unexpected or
unexplained outcome or response.
The term Registration Data Provider (RDP) means
(a) in respect of each Electricity Distributor, the person nominated in writing to the DCC from
time to time by that Electricity Distributor; or
(b) in respect of each Gas Transporter, the person nominated in writing to the DCC from time
to time by that Gas Transporter,
on the basis that more than one Party may specify the same Registration Data Provider, and
that the Electricity Distributor or the Gas Transporter shall be deemed to have so nominated
itself in the absence of any other nomination.
The term RDP Systems means any Systems:
a) which are operated by or on behalf of an Electricity Distributor or Gas Transporter
responsible for providing (or procuring the provision of) Registration Data in
respect of a particular MPAN or MPRN; and
b) which are used wholly or partly for the collection, storage, Back-Up, processing or
communication of that Registration Data prior to, or for the purposes of, its
provision to the DCC over the Registration Data Interface.
This document uses standard testing terminology, a glossary (Reference [1]) of which can be
found on the International Software Testing Qualification Board website, www.istqb.org.
Abbreviations used in this document are listed in the following table.

Page 11 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Abbreviations

Full Title

BI

Business Intelligence

CI

Configuration Item

CoS

Change of Supplier

CPA

Commercial Product Assurance

CSP

Communication Service Provider

CTSD/SREPTSD

Common Test Scenarios Document/SMKI and Repository Test Scenarios


Document

DAB

Design Assurance Board

DCC

Data Communications Company

DCC KI

DCC Key Infrastructure

DECC

Department of Energy and Climate Change

DLMS

Device Language Message Specification

DSP

Data Service Provider

DSM

Device Selection Methodology

DUGIS

DCC User Gateway Interface Specification

E2ET

End to End Testing

GBCS

Great Britain Companion Specification

HAN

Home Area Network

IHD

In Home Display

iGT

Independent Gas Transporter

IRB

Issue Review Board (for Testing Issues)

IT

Interface Testing

MI

Management Information

MPRS

Meter Point Registration System

OAT

Operational Acceptance Testing

PIT

Pre-Integration Testing

RDP

Registration Data Provider

RTM

Requirements Traceability Matrix

SEC

Smart Energy Code

SIT

Systems Integration Testing

SMKI

Smart Meter Key Infrastructure

SP

Service Provider

SREPT

SMKI & Repository Entry Process Test

SSI

Self Service Interface

SSMI

Self Service Management Interface

S/W

Software

TAG

Test Advisory Group

TBDG

Technical Business Design Group

Page 12 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Abbreviations

Full Title

TSP

Trusted Service Provider

UAT

User Acceptance Testing

UEPT

User Entry Process Test

UIT

User Integration Testing


Figure 4 - Abbreviations

Page 13 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Scope
2.1

Overview

In its role as Systems Integrator, the DSP will manage Interface Testing (IT) with support from
SEC Parties, the Registration Data Providers (RDPs) and Service Providers (SPs).
IT will verify that SEC Party, SP and RDP system elements integrate together to form a
working system that meets the agreed functional and non-functional requirements defined in
the SEC and the SP contracts. The system consists of the whole Smart Metering eco-system.
IT will be undertaken on a CSP Region by CSP Region basis, with CSP Regions being tested
in parallel where practicable. It will use the Devices the DCC successfully used as the basis
of testing in SIT. Should Devices not be available, then Device Stubs will be used.
Large Supplier Parties (and Network Parties if so required by the Secretary of State) are
obliged to be ready to begin their User Entry Process Tests (UEPT) during IT. Other SEC
Parties can also conduct their UEPT during IT.

2.2

Out of Scope

The following are outside the scope of IT:

certification of Meter Device Models (energy Suppliers are responsible for ensuring that
any meters they install at consumers premises are SMETS compliant, including a
requirement that they are protocol certified);
certification of Communications Hubs (the CSPs, in conjunction with their Comms Hub
manufacturers, are responsible for this activity);
testing of Meter Device Models, other than the interaction of those Devices that are
installed in CSP Test Labs, for the purpose of conducting User Entry Process Tests, with
the DCC solution (energy Suppliers are responsible for ensuring that any Meter Device
Models that they install are interoperable with the DCC and are SMETS compliant);
testing of the Home Area Network (HAN) other than its interaction with the DCC solution (it
is the responsibility of energy Suppliers to ensure that a SMETS compliant HAN is
established in each consumer premise);
testing of Hand Held Terminals other than their interaction with the DCC solution (energy
suppliers are responsible for ensuring that any Devices that they use are compliant with
the relevant technical specifications);
testing the inter-changeability2 of Devices connected to the Home Area Network (this is not
a requirement under the provisions of the SEC);

testing of Service User Business Processes (Service Users can test their back office
systems against the DCC during the End to End Test stage on a voluntary basis).

2________________________ The ability to exchange one device with another without


affecting the original functionality
Page 14 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Objectives

The objective of Interface Testing is to demonstrate that the DCC and the DCC Systems
together with the Communications Hubs interoperate with User Systems in order that the
DCC is capable of complying with its obligations under Sections E (Registration Data), G
(Security) and H (DCC Services) (in each case) at levels of activity commensurate with
the relevant Volume Scenarios.
This Test Approach document sets out the manner in which the IT objective will be achieved
and includes the requirements that are set out in Section T3 of the SEC.

Page 15 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Test Stage Description

4.1

Introduction

There are two elements to Interface Testing:


DSP-led testing of the interoperability of the DCC solution with SEC Parties systems; and
SEC Parties undertaking their own User Entry Process Testing (UEPT).

4.2

DSP-led testing

The primary focus of IT is on the dynamic interaction between the Service Provider and
Service User systems. The key elements of the testing that will be undertaken are:
the DCC User Message Gateway (item 1 on the diagram below), which was stubbed in
PIT and SIT: integral to this is Parse & Correlate;
the DCC Self Service Interface (item 2), which was tested in PIT and SIT by imitating the
behaviour of Service Users over a local connection.
The other External Interfaces that are shown in the following diagram will already have been
verified during PIT and SIT. Regression testing of these interfaces will be conducted to the
extent necessary as part of the Additional Functionality tests.

Figure 5 DCC system

Page 16 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Testing will be undertaken with Devices if Test Stubs have been used in SIT and if such
Equipment is available in time for Interface Testing.

4.2.1 DCC User Message Gateway


The conduct of UEPT by Parties will extensively test the DCC User Message Gateway as set
out in the CTSD/SREPTSD.

4.2.2 DCC Self Service Interface


The testing undertaken in PIT and SIT will be extended in IT by requiring SEC Parties to
verify that the Self Service Interface works correctly for SEC Parties Identity Providers (used
for authentication and access control).

4.2.3 Additional Functionality Tests


SEC Parties will be required to undertake tests as listed below:
Change of Supplier
Billing;
Reporting;
Self Service Interface; and
Volume test.
Further details are contained in Appendix A - Test Scenarios. The volume test requires the
participation of all SEC Parties.

4.2.4 CSP Regions


The DSP-led testing will be performed on a CSP Region by CSP Region basis, so that
achievement of the IT Objective can be demonstrated for each CSP Region separately. The
technical solution for CSP Regions Central and South is identical 3, and therefore these will be
treated as one Region for purpose of testing, notwithstanding the fact that MPxNs from both
geographical Regions will be included in the testing.

4.2.5 Non-Functional Testing


Performance Testing
Performance testing will be undertaken to the extent possible taking into account the
constraints of the IT test environment. The system will be subject to load (using a stub) to a
proportionate volume to the load used in Systems Integration Testing. A co-ordinated
performance test will then be undertaken involving all SEC Parties that are participating in
Interface Testing (anticipated to be a minimum of 20) during which they will concurrently
submit a Service Request. The metrics from this test will be compared with the metrics for the
equivalent test using the Service User Simulator during SIT to confirm that the introduction of
SEC Party systems has not adversely affected the performance of the DCC solution.

3________________________ CSP Central/South (Telefonica) will be required to provide


evidence to the DCC that the technical solution is the same in each region
Page 17 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Full Performance Testing will take place on the production environments connected together
with the production communications links as part of Operational Acceptance Testing (OAT).
Where practical to do so, this OAT testing will be conducted in the same timescales as IT and
the results provided in the IT Stage Completion Report to the SEC Panel. Otherwise the
results will be reported separately to the SEC Panel for information.

Resilience Testing
As with Performance Testing, full Resilience Testing is not feasible in IT and will instead take
place on the production environments connected together with the production
communications links as part of OAT. SEC Parties are not expected to participate in
Resilience Testing. Where practical to do so, OAT testing will be conducted in the same
timescales as IT and the results provided in the IT Stage Completion Report to the SEC
Panel. Otherwise the results will be reported separately to the SEC Panel for information.

4.3

UEPT

SEC Parties will undertake User Entry Process Testing as described in the CTSD/SREPTSD
[7]. The DCC will facilitate UEPT as described in the CTSD/SREPTSD.
Prior to starting UEPT, the DCC requires SEC Parties to undertake a Connectivity Test in
order to verify that the Service Users system can connect to the DCC. This comprises the
sending of a DCC-only Service Request and the receipt of a Response.

Page 18 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Deliverables

5.1

Responsibilities for Deliverables

The responsibilities of each Test Participant with respect to each IT deliverable are set out in
Figure 6 RACI Matrix showing Deliverables.
Key:R Responsible
A
Accountable
C
Consulted
I
Informed
Test Participant
Deliverable

DCC

DSP

CSP

DCC
Enterprise

TSP

RDP

SEC
Party

Interface Test Approach


document

Test Infrastructure
(hardware &
Communication)

Test Environments
(software & configuration)

Test Labs Premises +


Communication Hubs

Test Labs - Devices

A/R

Interface Test Plan

Test Data Plan document

Test Data Provision

Test Tools

Executed Tests

Support for Test Execution

IT Preparation Progress
Reports

IT Execution Progress
Reports

IT Stage Completion
Report

Figure 6 RACI Matrix showing Deliverables

5.2

Notes on Deliverables

The DSP, in its role as systems integrator, is responsible for the overall planning,
management and delivery of Interface Testing. The DCC, in its overarching assurance role, is
accountable.
The DCC (together with Device manufacturers) is responsible for provision of the Devices for
use in CSP Test Labs and any related technical support which is necessary. These Devices
will be selected in accordance with the Device Selection Methodology [6]. The CSPs are
responsible for provision of the physical environment of each Test Lab, the communications
Page 19 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
hubs and related technical and testing support. The Device manufacturers will install Devices
in the CSP Test Labs, and the CSPs (together with the Device manufacturers) will verify that
they function in connection with the communications hubs.
The Test Tools row in the above table refers to any tools which need to be used for Interface
Testing. The CSPs will provide the stubs used in SIT, if Devices are not available for IT.
These stubs will provide a platform for the receipt and transmission of commands. IT does
not test the functionality of the Device and the use of stubs should therefore provide an
adequate level of assurance that a DCC user can communicate with a Device (noting the
criteria that are set out in the SIT Approach Document for verifying that a test stub is suitable
for use). However, Energy Suppliers will be invited to re-perform an install and commission
test when actual Devices are available.
The DSP may need to provide a stub to simulate the activity of a SEC Party, in order to carry
out some of the test preparation and specific test execution activities (e.g. Change of
Supplier).

Page 20 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Test Procedure

6.1

Requirements traceability

The Requirements Traceability Matrix (RTM) output from SIT will be updated with tests added
in IT and the revised version will be supplied to the DCC.

6.2

Test Stage overlap

6.2.1 SP UAT Test Stage with Interface Test Stage


As part of the programme re-plan in February 2014, the decision was taken to overlap the SP
UAT Test Stage of SIT with the Interface Test Stage in order to minimise the impact of Great
Britain Companion Specification (GBCS) delays on the go-live date. This overlap introduces a
risk that any significant defects found in SP UAT will cause re-work and delay in Interface
Testing, however this risk is small given that SP UAT is merely a repeat of a subset of the
tests run in the Solution Test Stage. This risk will be mitigated by scheduling the high risk
tests for the start of SP UAT.

6.2.2 Solution Test Stage with Interface Test Stage


Given that SIT will be conducted on a CSP Region by Region basis, the DCC may, after
consultation, recommend to the Secretary of State that Interface Testing for one or more
Regions is started before Solution Testing is completed.
The DCC will publish any such proposal and consultation results on its website.
Where the Secretary of State agrees with the DCCs recommendation, Interface Testing shall
commence from the time recommended for the identified Regions.

6.2.3 Interface Test Stage with End to End Test Stage


End to End Testing is currently scheduled to follow Interface Testing but the DCC, after a
period of consultation, may recommend to the SEC Panel that End to End Testing starts
before the end of Interface Testing.
The DCC will publish any such proposal and consultation results on its website.

6.3

Technical Readiness

6.3.1 Service Providers


All SPs will undergo a Technical Readiness assessment before participating in IT, which will
confirm:
the impact on IT of any outstanding Solution Test defects, and whether the dates in the
Work-Off Plan are acceptable
that the relevant IT environments have been:
o

built

configured
Page 21 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
o

loaded with the required solution elements (including Test Stubs, tools and
devices)

connected up (e.g. firewalls opened up and validated)

smoke tested

that the base test data required to commence IT has been loaded to the relevant data
stores
that the Test Management tool has been set up and made available to the relevant
personnel
that guidelines for the Test Management tool have been developed and that the relevant
SP and supporting SEC Party personnel have been made familiar with its use.

6.3.2 SEC Parties


All SEC Parties will undergo a Technical Readiness assessment as defined in the
CTSD/SREPTSD.

6.3.3 RDPs
The Technical Readiness assessment will consist of verifying that each has supplied the data
necessary to support the testing.

6.4

Testing Principles

6.4.1 Non-discrimination
The DCC will support SEC Parties performing UEPT in IT and the DCC will facilitate the
concurrent testing of these SEC Parties to the extent reasonably practicable. To facilitate this,
SEC Parties are required to notify the DCC of their intention to participate in IT five months
ahead of the start of IT execution so that Test Lab facilities and support resources can be
expanded where necessary.
Should unexpected problems be encountered, Energy Suppliers will be given priority to test in
their gas supplier and electricity import supplier roles in order that IT can complete in a timely
manner. See Section 7.1 - Introduction, for details on current assumptions made regarding
the number of Users who will participate.

6.4.2 Use of Devices/Device stubs


IT will be conducted with physical Devices rather than stubs. However, if Devices are not
available at the start of IT, stubs will be used. Should Devices become available in sufficient
volume during the conduct of IT, they will be introduced into IT. However, the DCC will first
verify that these Devices can interoperate with the DCC, which may require testing in the SIT
environment or against some other suitable test tool. Before introducing Devices into IT, the
DCC will determine if by doing so it introduces significant delay (i.e. because of the need for
retesting) to the completion of IT or adversely impacts any SEC Party that is conducting
UEPT. In choosing to use stubs as the basis of IT, the DCC will first confirm that the stub will
provide a suitable basis for the conduct of UEPT.

Page 22 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

6.4.3 Testing with one CSP


Each SEC Party will be expected to conduct their UEPT tests with a single CSP, as allocated
by the DCC. Every effort will be made to accommodate a preference where one is expressed.
For Network Operators, the CSP allocated will be in the geographical area covered by that
SEC Party.
Having completed the tests with one CSP, the SEC Party will not be expected to repeat them
with the other CSP.

6.4.4 Testing by Region


The SEC enables IT to start in one Region in advance of another Region.
As set out in Section 6.4.3, the DCC will make every effort to allocate SEC Parties to the
Region of their choice for the purpose of conducting UEPT.
However, where the difference in the start date for the two Regions is more than 2 weeks,
SEC Parties will be allocated to the first available Region. At the point at which the second
Region becomes available, those SEC Parties who have already completed UEPT (or are still
undertaking UEPT) on the first Region will be invited to conduct selected tests on the second
Region. SEC Parties yet to start their UEPT will be allocated across both Regions, subject to
capacity constraints.

6.5

Release management

Releases of fixes and configuration changes will be scheduled for each week during IT,
unless otherwise agreed by the DCC (frequency to be reviewed in the light of experience).
The UIT Issue Manager will agree the contents of these releases with the SP IT Managers
and the DCC Service User Integration Test Manager. Should an emergency Release be
required (e.g. because testing is seriously impeded or halted), the UIT Manager will decide on
the timing after consulting with the SP IT Managers and the DCC Service User Integration
Test Manager.
Each Release into the IT environment will be accompanied by a Release Note describing the
contents of the Release. All fixes and configuration changes included in a Release will
undergo PIT and SIT with the relevant SPs, which includes an appropriate degree of
regression testing according to the principles laid out in the PIT and SIT Approaches. Once a
Release has been installed in the IT environment, it will be subject to:
a smoke test, that will be undertaken by the DCC;
testing of the constituent fixes and configuration changes, that will be undertaken by the
relevant SPs and SEC Parties; and
an IT regression test, based on assessment of the risks that the new release will cause
features previously working in IT to stop working that will be undertaken by the DCC.

6.6

Configuration management

The DSPs Configuration Manager owns the master Configuration Plan which defines a) the
Configuration Items (CIs) comprising the DCC Solution and b) the inter-dependencies
between these CIs. This configuration management process is part of the test preparation
activities. See Figure 7 Timetable for test for the timeline of this.
Page 23 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Page 24 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

6.7

Timetable

The high level timetable for IT can be divided into the following sections:
Test Initiation
Test Execution
Test Completion.
The tables below list the activities that must be undertaken in each section. They do not duplicate the information relating to UEPT activities in
the CTSD/SREPTSD but describe additional activities.

IT Test Initiation

The table below sets out the steps that must be undertaken by either the DCC or relevant Test Participant seeking to undertake IT and the
timeframes within which such steps must be completed.
Ref

By When

Action

From

To

23/07/2014

Publish P&C Software


Architecture
Specification

DCC

SEC Parties

6 months prior to
the start of IT

Publish IT Approach

DCC

SEC Parties

SEC Panel approval

DCC website

6 months prior to
the start of IT

Publish
CTSD/SREPTSD

DCC

SEC Parties

Incorporated into the SEC as a subsidiary document

DCC website

6 months prior to

Publish

DCC

SEC Parties

SMKI

Information Required

Method
DCC website

DCC website

Page 25 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Ref

By When

Action

From

To

the start of IT

Interface Specification

6 months prior to
start of IT

Publish
Interface
Testing Plan

DCC

SEC Parties

DCC website

6 months prior to
start of IT

Publish Test Data Plan

DCC

SEC Parties

DCC website

6 months prior to
start of IT

Publish
Plan

DCC

SEC Parties

By secure email

5 months prior to
the start of IT

Confirm intention to
participate in IT

SEC Parties

DCC

Confirmation of intention, and supporting plan

By email

6 months prior to
the start of IT

Order
Connection

SEC Parties

DCC

Completed Order Form

email

10

4 months prior to
start of IT

P&C software
available

V2

DCC

SEC Parties

DCC website

11

2 months prior to
start of IT

Publish Testing Issue


Management
Tool
Guide

DCC

SEC Parties

DCC website

12

2 month prior to
start of IT

Publish Schedule of IT
execution

DCC

SEC Parties

DCC Website

13

2 months prior to
start of IT

Publish Testing Issue


Resolution
and

DCC

SEC Parties

DCC website

Environment

Network

Information Required

Method

Page 26 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Ref

By When

Action

From

To

DCC

SEC Parties

Information Required

Method

Disputes Process
14

2 months prior to
start of IT

Publish IRB Terms of


Reference

15

30 working days
prior to the start of
IT

Set up Individual test


environments

16

30 working days
prior to the start of
IT

Supply meters for Test


Labs

17

25 working days
prior to the start of
IT

Configure
and
connect Interfaces

18

1 month prior to the


start of IT

19

DCC website

ALL

Completed Checklist

email

DCC

Inventory list

email

DCC

ALL

Completed Checklist

email

Populate
test
environments with test
data

SEC Parties

DCC

Confirmation

email

1 month prior to the


start of IT

Develop Test Stubs (if


needed)

SEC Parties

DCC

Confirmation

email

20

1 month prior to the


start of IT

Set up Testing Issue


Management Process
& Tool

DCC

SEC Parties

Completed Checklist

email

21

1 month prior to the


start of IT

Set up Configuration
Management Process

DCC

SEC Parties

Completed Checklist

email

Page 27 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Ref

By When

Action

From

To

Information Required

Method

DCC

SEC Parties

Completed Checklist

email

DCC

SEC Parties

Completed Checklist

email

SEC Parties

DCC

High level test design, test scripts, regression test pack

By email

DCC

ALL

Completed Checklist

email

DCC

ALL

Completed Checklist

email

SEC Parties

DCC

Quality Gate checklist

by email

& Tool
22

1 month prior to the


start of IT

Provision
Connection

Network

23

1 month prior to the


start of IT

Set up Test Labs

24

1 month prior to the


start of IT

25

15 working days
prior to the start of
IT

Validate eco-system

26

5 working days
prior to the start of
IT

Complete Readiness
Verification

27

1 working day prior


to the start of IT

Satisfy
Criteria

28

Start of IT

Commence IT

Deliver Test Artefacts

IT

Entry

ALL

Figure 7 Timetable for test initiation

One of the key events in the above section is Confirm intention to participate in IT (Ref 8), which is when SEC Parties planning to participate
in IT must declare their intention to do so. Declaration of intention to test must be accompanied by a plan, against which the Users
preparation progress will be monitored. The date of declaring intention to test is when the Service Providers need to know whether or not the
Page 28 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

planned Test Lab and support facilities will be adequate for the expected demand: for further details of this and the assumptions made, see
Section 7 - Environments, Networks and Test Labs.

IT Test Execution

The table below sets out the steps that must be undertaken during test execution by either the DCC or Testing Participant and the timeframes
within which such steps must be complete.
Ref

By When

Action

From

To

Information Required

Method

[10 working days]


after the start of IT

User
Connectivity
Validated
(first
2
users)

DCC

[1 month] after the


start of IT

North
Region
:
Connectivity
Test
completed

SEC Parties

DCC

Test Completion Report

email

[2 months] after the


start of IT

North Region : UEPT


completed

SEC Parties

DCC

Test Completion Report

email

[2 months] after the


start of IT

North
Region:
Additional
Functionality
Tests
completed

DCC

Test Completion Report

email

[30 working days]


after the start of IT

Central
&
South
Region : Connectivity
Test completed

SEC Parties

DCC

Test Completion Report

email

[50 working days]


after the start of IT

Central
&
Region:
completed

SEC Parties

DCC

Test Completion Report

email

South
UEPT

Page 29 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Figure 8 Timetable for test execution

Another key milestone is User Connectivity Validated (first 2 users) (Ref 1). This is where it has been demonstrated that two SEC Parties in
each Region can successfully connect to the DCC System, by the successful sending of a Service Request and subsequent receipt of a
Response. (All SEC Parties will conduct Connectivity Testing, but for the purposes of milestone planning, the DCC will assume that user
connectivity is validated when two SEC Parties have demonstrated they can successfully connect to the DCC system.)

IT Test Completion

The table below sets out the steps that must be undertaken during test completion by either the DCC or Testing Participant seeking to
undertake IT and the timeframes within which such steps must be completed.

Page 30 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Ref

By When

Action

From

To

Information Required

Method

10 working days
before the end of IT

Draft IT Completion
Report

DCC

SEC Panel

As described in Joint Test Strategy [2]

email

End of IT

Final IT Completion
Report

DCC

SEC Panel

As described in Joint Test Strategy [2]

DCC website

Figure 9 Timetable for test completion

Page 31 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

6.8

Dependencies

There are a number of dependencies within IT preparation and execution, which are
described in the following tables. In each case, the milestones and activities listed in the
tables are taken from the above Timetables: Figure 7 Timetable for test initiation and Figure
8 Timetable for test execution.
Milestone
Interface Test Approach
Published

Dependent on

Notes

Completion of industry
consultation
SEC Panel approval

Interface Test Plan Published

Interface Test Approach

Test Data plan Published

Definition of data

Test Data Ready

Issuing of Test Data Plan


Population of data
Verification of data

Test Tools Ready

Development of test stubs


Set-up of Testing Issue
Management tool

Parse & Correlate

Parse & Correlate v2 S/W being


Available
Provision of network connection
Set-up of test labs
Set-up/configuration of individual
environments
Configuration of interfaces
Verification of eco-system smoke test, penetration test

Test Artefacts Delivered

Development of detailed test


scripts
Development of other test
artefacts

Interface Test Entry Criteria


Satisfied
Interface Test Started

See Section 6.9.1 - Entry Criteria


Publication of Interface Test
Approach

The Interface Test Approach


must be published 6 months
before the start of IT

Publication of Interface Test Plan


Declaration of Intention to Test
Test Data being Ready

Intention to Test must be


declared 5 months before the
start of IT

Test Tools being Ready


Test Infrastructure &

Page 32 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Environments being Ready


Interface Test Entry Criteria being
Satisfied
User Connectivity Validated

Completion of Connectivity Test

Interface Test Complete

See Section 6.9.2 - Exit Criteria

The Connectivity Test must be


successfully completed by two
Large Suppliers of each energy
type in at least one CSP Region
to pass this milestone.

Figure 10 Milestones and their dependencies

6.9

Stage Entry and Exit

6.9.1 Entry Criteria


General entry criteria for the Stage:
Interface Test Approach (this document) published 6 months previously;
Authority to Proceed certificate issued by DCC;
Interface Test Specifications (and supporting test data) prepared;
Technical Readiness Assessments (see Section 6.4) completed for each SP and RDPs
have provided the data necessary;
Service Provider test environments and Test Labs prepared;
Service Provider resources are available to support the testing; and
confirmation that SPs have implemented the security controls defined in the Code of
Connections for the DCC User Gateway Interface [10] and Self Service Interface [11],
where the IT test environment differs from that used for SIT.

For SEC Parties participating in UEPT:


confirmation from the DCC Service User Integration Test Manager that the SEC Party has
met their Entry Criteria, as defined in the CTSD/SREPTSD [7];
SEC Party resources are available to support DSP-led testing.

6.9.2 Exit Criteria


For the Interface Test Stage
SIT has successfully completed for both CSP North and CSP South/Central;
SIT Phase Completion Certificate issued;
Connectivity Test and UEPT successfully completed by at least two large suppliers of gas
(who are not affiliates of one another) in their role as gas supplier and two large suppliers
of electricity (who are not affiliates of one another) in their role as import supplier, for each
Page 33 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
of CSP North and CSP South/Central (successful Connectivity Test is defined as all
planned tests run; successful UEPT is defined as issue of Completion Certificate);
Connectivity Test and UEPT successfully completed using the DCC-supplied Parse &
Correlate software by at least one supplier (successful is defined as confirmation from
the UIT Manager that the suppliers evidence of the use of Parse & Correlate is
acceptable);
Where the Secretary of State has required the Network Parties to be ready to participate in
IT, Connectivity Test and UEPT successfully completed by at least one network party in
their role as electricity distributor or at least one network party in their role as gas
transporter, for each of CSP North and CSP South/Central;
All planned Additional Functionality Tests run, or any exceptions documented and agreed
with the DCC;
The number and severity of outstanding testing issues are within agreed thresholds and
the work-off plan has been published (see Section 9.4 - Testing Issue Severities);
DCC-produced IT Stage Completion Report issued; and
DCC-produced IT Stage Completion Certificate issued.

For SEC Parties participating in UEPT:


confirmation from the DCC Service User Integration Test Manager that the SEC Party has
met their Exit Criteria, as defined in the CTSD/SREPTSD [7].

SEC Panel approval


Having consulted with the RDPs and the SEC Parties, the DCC will present the IT Stage
Completion Report and the IT Stage Completion Certificate to the SEC Panel for approval
when the DCC considers that the Exit Criteria have been met. The DCC will clearly
demonstrate how each exit criterion has been met and the level of assurance that has been
achieved. If the Panel rejects the recommendation, then the DCC will undertake further tests
as directed by the SEC Panel and re-submit the evidence.
Should one Region complete IT before the other, then the DCC will present an IT Region
Completion Report to the SEC Panel. The contents of the report will be the same as the IT
Stage Completion Report.

Page 34 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Environments, Networks and Test Labs

7.1

Introduction

The DCC test environment for IT comprises a set of networked environments provided and
supported by each Service Provider, and Test Labs at each of the CSPs premises furnished
with communications hubs and Devices.
The environments, Test Labs and support have been designed on an assumed number of
testing participants (a SEC Party conducting testing for a particular Role) taking part in
Interface Testing. The assumption is that there will be 120 testing participants, of which 90 will
be assigned to test in CSP Central/South and 30 assigned to CSP North.
Should the number of SEC Parties wishing to test simultaneously exceed these numbers,
then the requirements will be assessed, the technical feasibility validated, the relevant
approvals sought, the delivery planned and then the Test Labs and technical support can be
increased accordingly. The long lead times involved mean that an early date for notification of
intention to test has to be given (see section 1 - for the relevant date).

7.2

DCC Enterprise environments

DCC Enterprise will provide a test environment which contains Billing and Reporting
functionality.

7.3

DSP environment and network

The DSP will provide a test environment containing all the DSP-delivered components,
including the Self Service Interface (SSI), as well as the network connections used by the
SEC Parties.

7.4

CSP environments and networks

Both CSPs will provide a test SMWAN network, together with test systems connected to the
DSP. If SIT Solution Test has exited with Device stubs and physical smart Devices are still
not available in time to use in IT, CSPs will also provide the Device stubs used in SIT.

7.5

TSP environment

The TSP will provide a test environment containing all the TSP-delivered components.

7.6

Parse & Correlate

The Parse & Correlate software will be provided to the SEC Parties, for installation on their
test environments.

7.7

SEC Parties

SEC Parties will provide and support their own test environments for use in IT, and are
responsible for establishing connectivity with the DSP network. The DCC will provide support
to the SEC Parties as requested and as appropriate.

Page 35 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

7.8

CSP Test Labs (for devices)

Each CSP will provide a secure physical environment in which to house the relevant number
of communications hubs and Devices. The CSP will also provide the communications hubs
(Devices will be provided by the DCC in collaboration with the meter manufacturers as
described in the Device Selection Methodology [6]);
Each Test Lab will contain sets of devices, with a set consisting of a minimum of:
one communications hub
one electricity meter
one gas meter,
and other Devices or stubs as required to complete UEPT.
Network operators will be allocated 2 Device sets and all other SEC Parties will be allocated
5 device sets.
Details of support provided by the CSPs during testing are yet to be finalised 4, and the
intention is that the type and level of support offered by the two CSPs will be similar. It is
expected to include support for checking the Devices during testing, as well as facilities which
will allow SEC Parties to house a limited number of their own staff on site and allow them
controlled access to the test labs. The CSP-provided support for checking the Devices will be
via a call-logging system, which will operate two different support methods, using:
requests for support at a specified time in the future (following working day onwards); and
requests for immediate support.
Once a CSP support team member has started working on a particular request, then contact
will be directly between the CSP staff member and the relevant SEC Party staff member,
either by phone or by email. Email contact between the Test Participants will be secured by
TLS or an equivalent method.
Figure 11 Number of device sets shows the numbers of devices available:
Number of Device Sets

CSP North (Arqiva)

CSP Central/South (Telefonica)

Initial Provision *

75

225

Expansion Capability (number


that can be added)

25

150

Total Possibly Available

100

375

Figure 11 Number of device sets

4________________________ The level of support will be discussed with Test Participants


via the TAG and TDEG. Any comments will be taken into account to the extent appropriate in
the final version of this document and the results of the discussions and any issues raised will
be presented to the TAG when it conducts its final review on the document prior to making a
recommendation to the SEC Panel on whether this document should be approved.
Page 36 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
*A portion of the Initial Provision (final percentage to be confirmed, current estimate is 10%),
will be used for Systems Integration Testing, that occurs in parallel with IT (and
therefore is not available for IT)
Devices will be available for electricity export testing.
The devices will be subject to load simulation to the extent necessary to support UEPT, noting
that the functionality of the devices will not be tested.
Sufficient numbers of spare devices will be held in reserve so that faulty devices can be
replaced immediately, as laid out in the Device Selection Methodology [6]. This will ensure
that the number of operative Devices is kept at the level shown in Figure 11 Number of
device sets.
It is intended that where possible, Devices and communications hubs will be re-used by
different SEC Parties, as some SEC Parties complete their testing and possibly others begin
their testing. This may be done by using a simple Change of Supplier process. In addition,
each Energy Supplier will need some devices provided in an initial state as delivered by the
meter manufacturer (in order to perform an Install and Commission Test) and this will be
catered for.

7.9

De-selection and substitution of Devices

Where it becomes evident that a Device that is used in IT for the purpose of UEPT has a
defect, the DCC will de-select that Device in accordance with the Device Selection
Methodology [6].
In doing so, the DCC will first determine if the defect relates to one specific Device or to all
Devices of that specific model. Where the defect relates to one Device, that Device will be
substituted with one of the same type. If the defect relates to all Devices of that model, the
DCC will substitute Devices of that model type with another Device model according to the
Device Selection Methodology. In so doing, the DCC will undertake whatever testing it
deems appropriate to prove that the substitute Device model can communicate with the DCC
in respect of all relevant messages. If it is not possible to substitute the defective Device with
either another of the same model or of a different model, then the DCC will use a test stub in
place of the defective Device.

Page 37 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

7.10 Summary of test facilities provided


Standard facilities are as described in the table below. Reasonable requests for out-of-hours support will be considered on a case-by-case
basis.

Hours of support availability/


access to labs

DCC Enterprise
Systems

DSP Systems/
DCC User
Gateway

TSP System

CSP Systems/
SMWAN

Parse &
Correlate

Test Labs CSP

Test Labs
DCC/meter
manufacturer

08:00-18:00 on
working days

08:00-18:00 on
working days

08:00-18:00 on
working days

08:00-18:00 on
working days

08:00-18:00 on
working days

08:00-18:00 on
working days

08:00-18:00 on
working days

Figure 12 Summary of test facilities provided

A working day is defined as Monday-Friday, excluding England & Wales Bank Holidays.

Page 38 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Test Data

8.1

Test Data Plan

Full details of the test data to be used will be specified as part of a separate exercise and
documented in the Test Data Plan (see Section 6.8.1 for the timeline for production of this
Plan).
The Test Data Plan will cover:
Principles
o

How data is allocated to each SEC Party

How data allocation is communicated

How co-ordination will ensure no overlap/conflict between SEC Parties when


testing (and between DSP-led tests and SEC Parties doing UEPT)

Responsibilities of each SP/SEC Party


Timetable, including
o

When data must be provided

Dependencies
Test Data Management
o

Process

Tool

Data Specification - for each item of data needed for system set-up:
o

Test Participant responsible for providing/maintaining

How it relates to other data in the DCC Systems

Source and content of the data

Quantity of data required

The production of this Test Data Plan and subsequent provision of test data for IT will be a
collaborative exercise between all SPs and the SEC Parties participating in TDEG. This will
ensure that SEC Parties have adequate and correctly-functioning data with which to test.

8.2

Test data principles

The principles stated in the Test Data Plan will include the following:
Data provided will be adequate to support the planned number of participating SEC Parties

Page 39 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Data provision and maintenance will be co-ordinated and managed by the DSP, who may
instruct an SP or SEC Party to carry out an update or correction to the data for which they
are responsible
Test data will comprise a cut of old Live data if necessary, with the proviso that the
Supplier to an MPxN is not divulged to another Supplier
Customer names will not be included in the data provided
Some test data will be reserved for DSP-led testing
Each SEC Party will be allocated data with which to test (which will be enough to cover the
number of Device sets they are using for IT); this will be done by consultation between the
DCC and the SEC Party
The data one SEC Party is using for testing will not be revealed to another SEC Party; that
SEC Party will have exclusive use of the allocated data during IT (e.g. meter points
allocated)
Data will comply with the provisioning and management guidelines in the Security section
(Section G) of SEC [3].
Data will be reviewed before testing starts
Use of data will comply with the UK Data Protection Act
Data will be able to be restored (if needed) at some point during the testing.

Page 40 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

8.3

Responsibilities for data provision

Figure 13 High-level data types and responsibilities for provision shows the main types of
test data in the system and which Test Participant has the responsibility for defining, providing
and maintaining this data.
Data Type

Accountable Party

Service Requests/Responses

n/a

Registration

RDPs

SMS Inventory

DSP

SMKI Repository:

TSP

Organisation Certificates

Device Certificates

Certificate Revocation Lists

SSI/SSMI User Security Credentials

DSP

CSP network coverage

CSP

RDP list

DSP

Mapping:

Post code to CSP region

User role to Service Request

GBCS Use Case device capability to Service


Request

DSP

DCC Reference (manufacturers, device types, models)

DSP

User Manuals and FAQs (for SSI)

DSP

Network configuration:

User Gateway

DSP

SMWAN

CSPs

SMKI

TSP

System configuration:
DCC Enterprise Provider

Billing

Reporting
Figure 13 High-level data types and responsibilities for provision

Page 41 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Testing Issue Management

9.1

Principles

The DCC will provide licences for HP Quality Centre (HP QC), for use by all SEC Parties and
SPs participating in Interface Testing, for logging and resolving testing issues. A single
licence will be provided to each SP and SEC Party and they will be required to record testing
issues directly on the HP QC database.
HP QC is a test management tool, which has facilities to create and manage the following:
Business requirements
Test scenarios and detailed test scripts
Test Phase and Cycle execution details
Testing Issues.
It allows links to be established between the items listed, which facilitate the production of
quality assurance documentation such as Requirements Traceability Matrices (RTM) and test
execution progress reports. Its set-up will be done as shown in Figure 7 Timetable for test .
HP QC will be used to:
manage DSP-led testing
demonstrate traceability between SP Requirements, test scripts and issues.
SEC Parties will use HP QC only for logging testing issues.
The HP QC Issue Repository will comprise several different (and segregated) Projects:
Interface Testing Issue Repository (IT IR) testing s relating to DCC Systems uncovered
in any of the tests conducted as part of IT (i.e. Connectivity, UEPT, Additional Functionality
Tests)
SEC Party Issue Repository (SEC Party IR) one for each participating SEC Party,
containing testing issues relating to that SEC Partys systems (i.e. testing issues which the
SEC Party needs to resolve before exiting their UEPT)
Device Issue Repository (Device IR) one for each manufacturer with a device being used
in IT.
This segregation ensures that one SEC Party cannot see testing issues relating to the
systems of another SEC Party, and that one Device manufacturer cannot see testing issues
relating to the Devices of another manufacturer. However, all Test Participants involved in the
testing will have access to IT IR.
IT IR will be used to record only testing issues attributed to the DCC Systems, and the testing
issues in IT IR will be used to determine whether or not the IT Exit Criteria have been met.

Page 42 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
If a testing issue recorded in IT IR is subsequently attributed to a SEC Party or Device
manufacturer it will be closed on IT IR and a new testing issue opened on the appropriate IR.
The UEPT for an individual SEC Party will be managed using their SEC Party IR, and the
testing issues in this repository will be used for determining whether or not the SEC Party has
met its UEPT Exit Criteria.

9.2

Testing Issue Management tool

The HP QC instance will be hosted and administered by the DSP. The DCC will ensure that
sufficient licences are available for use by the various Test Participants involved in IT. This HP
QC instance must be used for recording testing issues.
SEC Parties may be using their own HP QC systems for managing their testing issues
internally. In order to support this, IT IR will be set up so as to accept input of new testing
issues by either:
Online input - by being typed into the tool online directly by SEC Party/Service Provider, or
Excel import a template will be provided by the DSP to SEC Parties, to enable them to
capture testing issues once only in their own system, and then have them imported direct
into the DSPs central HP QC system (these templates will be transferred between the
SEC Party and the DSP via secure email).

9.3

Issue Review Board

A Terms of Reference5 will be created for the IRB, which will meet twice daily during test
execution to review testing issues recorded on the IT IR. Each new testing issue will:
be classified as one of:
o

testing issue:

that prevents execution of a test

that causes an unexplained or unexpected outcome or response to a


test

not a testing issue (e.g. a misunderstanding)

duplicate

change

need more information.

have its Severity and Priority set


be assigned to the relevant resolver group.

5________________________ This will include who can attend, what will be discussed, who
will be expected to attend (with reference to technical triage), what is the process including
outcome, what information is expected to be presented at the meeting to enable successful
triage and how confidential information is treated.
Page 43 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
The IRB will also review:
the outstanding Severity 1 and Severity 2 testing issues to ensure they are being resolved
at the requisite speed
outstanding actions from previous IRBs.
The IRB will be chaired by the UIT Issue Manager and attended by
SP Issue Managers
DCC Service User Integration Test Manager (and relevant SEC Party representative as
required)
SP Design Authorities
Meter Manufacturer SMEs
UIT Manager
DCC Test Assurance
SP Test Managers as required.
Where any testing issues are identified that impact the SEC or any subsidiary documents,
they will be managed in accordance with the Issue Resolution Procedure set out in the SEC
and the conclusions that are reached through this process will be incorporated into a Release
Note where appropriate.

9.4

Testing Issue Severities and Priorities

Each testing issue raised will be assigned a Severity and a Priority.


The Severity classifications are defined in the Joint DSP/CSP Test Strategy.
The Priority classifications are defined in Figure 14 - Testing Issue Priorities, below, lists the
standard Testing Issue Priorities:
Testing Issue
Priority

Description

All test progress is blocked by the testing issue.

Testing not completely blocked by the testing issue but the impact on test progress is
significant.

Testing can proceed but the work-around for the testing issue has moderate impact on
test progress.

Testing can proceed and the testing issue has little/no impact on test progress.
Figure 14 - Testing Issue Priorities

The following table lists the planned target response times for testing issues, to be measured
from the point at which the testing issue is logged in HP QC.
Page 44 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Page 45 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Priority

Initial
response
completed

Triage
completed

Assessed by
resolver group

Fix time
assessed

Target Release
identified

1 hour

4 hours

5 hours

13 hours

17 hours

1 hour

Next IRB

Next IRB +
1 hour

Next IRB +
8 hours

Next IRB +
8 hours

N/A

Next IRB

Next IRB +
1 hour

Next IRB +
16 hours

Next IRB +
8 hours

N/A

Next IRB

Next IRB +
1 hour

As required to
meet defect
thresholds

As required to
meet defect
thresholds

Figure 15 - Target testing issue response times

9.5

Testing Issues and exiting IT

Figure 16 Outstanding testing issues and exiting IT, below shows the maximum number of
testing issues of each severity which are allowed in order to exit IT. Note that the numbers
are given on a per SEC Party per Service Provider basis.
Severity

Number of Outstanding Testing Issues per SEC Party


per Service Provider

15

30
Figure 16 Outstanding testing issues and exiting IT

9.6

Testing Issue reporting

Reporting of testing issues by SEC Parties in relation to UEPT will be undertaken as specified
in the CTSD/SREPTSD [7].
Reporting of testing issues relating to the DCC Systems will be done by the UIT Manager and
based on the information contained in the IT IR and also in the Device IR. The reports will
contain as a minimum:
Current Testing Issue Status:
o

Count of testing issues by status (total since beginning of test)

Count of testing issues by Severity (total since beginning of test)

Page 46 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Count of Open testing issues by Severity.

Testing Issue Trend:


o

For Severities 1+2 combined, the trend over time of number of testing issues,
shown by status:
o

Total of all testing issues

Separately for each Service Provider (showing testing issues raised against
that Service Provider)

For all Severities combined, the trend over time, of number of testing issues,
shown by status:
o

Total of all testing issues

Separately for each Service Provider (showing testing issues raised against
that Service Provider)

Page 47 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

10

Test Tracking & Reporting

10.1 Introduction
Details on Test Tracking and Reporting for both test preparation and test execution activities
are described below.
SPs and SEC Parties responsible for documentation deliverables listed in Section 5 Deliverables will report on progress from the point at which this Interface Test Approach
document is published. During the last month of Preparation, the frequency of reporting
changes from monthly to weekly.
The progress report from each Test Participant will be addressed to the UIT Manager. The
UIT Manager will consolidate the information into a single report, which will be made available
to the DCC and all Test Participants (information will be anonymized where necessary). The
DCC will also make this report available to the SEC Panel. The report will be distributed via
email and its classification will be assigned according to the DCC Classification Standard; the
classification is likely to be either DCC CONFIDENTIAL or DCC CONTROLLED.

10.2 Test Preparation Reporting


10.2.1

General

During this part of the Stage, reporting is as follows:


Report Name

Frequency

From

To

Method

Test Preparation
Progress Report

Monthly

SEC Parties
SPs
RDPs

DCC

TBC

SEC Parties
SPs
RDPs

DCC

TBC

(weekly during the


final month)
Test Readiness
Report

Weekly
From 40 working
days prior to start of
test execution

Figure 17 Test Preparation Reporting

Test Preparation takes place from the date of approval of this document until test execution
commences.
This means that in the final 2 months before test execution starts, SEC Parties and the DSP
will need to provide both of these reports. There will be minimal overlap, given that the Test
Preparation Report is at a higher level than the Test Readiness Report.

10.2.2

Test Preparation Progress Report

The Test Preparation Progress Report will be delivered on the last Friday of each month. It
relates to the Test Participants preparation for the testing. The Test Participant may not be

Page 48 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
executing any tests itself (e.g. a CSP) but will have responsibilities in the run-up to the start of
testing.
The report will contain:
Executive Summary, including overall RAG status
Key Risks, Assumptions, Issues and Dependencies
Key Achievements this Period
Key Plans for next Period
Deliverables:
o

Date Planned

Date Forecast

RAG Status.

A simple Excel spreadsheet will be made available to facilitate and standardise this reporting.
Each report will be discussed in a short (30 minute) bilateral meeting in the first week of each
month, chaired by the UIT Manager and attended by the reporting Test Participant as well as
the DCC. Any issues or risks of relevance to other SEC Parties that require escalation will
then be identified and discussed at the next TDEG. At this point, the TDEG meetings will be
held monthly.

10.2.3

Test Readiness Report

The Test Readiness Report is as described in the CTSD/SREPTSD [7].


For the DSP-led testing, the DSP will submit a Test Readiness Report in the same way as
SEC Parties.
Test Readiness Reports relating to UEPT will be submitted to the relevant DCC Service User
Integration Test Manager.

10.3 Test Execution Reporting


Report Name

Frequency

From

To

Method

Test Execution
Dashboard

Daily

SEC Parties
SPs
RDPs

DCC

TBC

Test Support
Progress Report

Weekly

DCC

DECC

By email and
published on DCC
Website

Test Completion
Report

End of Stage

DCC

DECC

By email and
published on DCC

Page 49 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Website
Figure 18 Test Execution Reporting

The Test Execution Dashboard is as described in the CTSD/SREPTSD [7]. It is delivered to


the relevant DCC Service User Integration Test Manager, who will make it available to the UIT
Manager. The DSP will also submit this report for its IT tests to the UIT Manager. The UIT
Manager will produce a consolidated report once per week, which will not contain any SEC
Party-specific information and will be available to all Test Participants involved in the testing
and to the SEC Panel. This report will include a summary of test progress, number of testing
issues raised (by Severity), number of testing issues outstanding (by Severity) and any risk to
the timely completion of testing.
The Test Support Progress Report will be delivered each Thursday by CoB, enabling a
consolidated report to be produced each Friday (which will be redacted where necessary). Its
contents will be the same as for the Test Preparation Progress Report.
The Test Completion Report is as described in the CTSD/SREPTSD [7]. It is first provided in
draft form 10 working days ahead of the planned end of the testing.

Page 50 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

11

Roles & Responsibilities

11.1 General
All parties involved in IT shall:
Comply with the SEC and follow Good Industry Practice when participating in IT i.e. the
exercise of that degree of skill, diligence, prudence and foresight which would reasonably
and ordinarily be expected from a skilled and experienced person engaged in a similar
type of undertaking as that Party under the same or similar circumstances
take all reasonable steps to facilitate achievement of the IT Objective.

11.2 Roles and Responsibilities


11.2.1

Roles of Test Participants

Test Participant
DSP (Systems
Integrator role)

Role
Responsible for management and delivery of Interface Testing, (including its
planning, control and tracking)
Provision of test execution and environment usage schedules
Provision of Interface Test Plan
Provision of Test Data Plan
Management of testing issue resolution, provider of HP Quality Centre
Operation of the master Configuration Management Plan
Operation of the Environment Plan
Provision of test scenarios and test scripts for Connectivity and Additional
Functionality Tests
Provision of resource to carry out Additional Functionality Tests
Maintenance of IT RTM

DCC

Provision of test assurance activities (see Section 11.5)


Production of CTSD/SREPTSD
Provision of Devices for test labs, working with Meter Manufacturers (see
below)
Provision of progress reports to the SEC Panel

DSP (SP role)

Provision of DSP component of UIT test environment and related data


Provision of test stubs to simulate SEC Party input (e.g. to support CoS
testing)
Provision of support for DSP solution elements and DSP component of UIT
test environment
Provision of fixes required in DSP solution elements

Page 51 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Provision of resource to support SEC Parties testing


CSPs

Provision of CSP component of UIT test environment and related data


Provider of support for CSP solution elements and CSP component of UIT test
environment
Provision of fixes required in CSP solution elements
Provision of test lab facility and communications hubs
Provision of Device stubs (if necessary)
Provision of support for test lab
Provision of resource to support SEC Parties testing
Provision of support to the DSP as Systems Integrator in:

SEC Parties

management and delivery of IT

design and creation of test scenarios, test scripts, test data and test
environments

preparing test execution and environment usage schedules

triaging testing issues

creation and operation of the master Configuration Management Plan

creation and operation of the master Release schedule

creation and maintenance of the Environment Plan

Provision of detailed Connectivity Test and UEPT test scripts


Provision of SEC Party test environment and related data
Provision of resource to carry out the UEPT, Connectivity and Additional
Functionality tests
Provision of resource to triage testing issues
Provision of any fixes required in SEC Parties systems
Provision of Communication links to UIT test environment
Provision of test evidence as set out in the CTSD/SREPTSD

RDP

Provision of initial set of registration data to be used in testing


Provision of resource to triage testing issues

TSP

Provision of SMKI component of UIT test environment and related data,


including test certificates
Provision of support for SMKI solution elements and SMKI component of test
environment
Provision of fixes required in SMKI solution elements
Provision of resource to support SEC Parties testing
Provision of support to the DSP as Systems Integrator in:

management and delivery of IT

design and creation of test scenarios, test scripts, test data and test
environments

Page 52 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

DCC Enterprise

preparing test execution and environment usage schedules

triaging testing issues

creation and operation of the master Configuration Management Plan

creation and operation of the master Release schedule

creation and maintenance of the Environment Plan

Provision of billing/reporting component of UIT test environment and related


data
Provision of support for billing/reporting component of test environment
Provision of fixes required in billing/reporting solution elements
Provision of resource to support SEC Parties testing
Provision of support to the DSP as Systems Integrator in:

Critical Software

management and delivery of IT

design and creation of test scenarios, test scripts, test data and test
environments

preparing test execution and environment usage schedules

triaging testing issues

creation and operation of the master Configuration Management Plan

creation and operation of the master Release schedule

creation and maintenance of the Environment Plan

Provision of support for standard DCC Parse & Correlate software


Provision of fixes required in Critical Software solution elements

Meter manufacturer
(together with DCC)

Supplier of meters to furnish the test labs, working with the DCC
Provision of support for Devices both in labs and when returned to factory
Provision of resource to triage testing issues

SEC Panel

Approval of IT Approach Document


Approval of IT Stage Completion Report
Recipient of escalated testing issues from DCC
Notify testing issues to the Secretary of State
Resolve of testing issues where directed by the Secretary of State
Recipient of progress reports

DECC

Resolve testing issues notified by the SEC Panel


Recipient of progress reports where provided by the SEC Panel

Ofgem

Resolve disputes on IT Approach Document unless otherwise directed by SEC


Panel
Figure 19 Roles of participating Test Participants

Page 53 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Large Supplier Parties have an obligation to take all reasonable steps to be ready to start
UEPT at the beginning of Interface Testing. Network Parties have an equivalent obligation, if
directed to do so by the Secretary of State.
Each Test Participant participating in test execution activities is responsible for ensuring that
the staff it provides are adequately trained and have adequate facilities to enable them to
perform the tests.

Page 54 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

11.2.2

Reporting structure

The reporting structure for IT is shown in Figure 20 IT main roles and structure, below:

Figure 20 IT main roles and structure

The UIT Manager will maintain a list of the contact details of the person filling each role for each Test Participant.
Page 55 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

11.2.3

Roles of Test Management

Role
User Integration Test Manager

Responsibility
The User Integration Test Manager is the person responsible for overall
planning, preparation and execution of Interface Testing. This role
encompasses co-ordination, control, management and
tracking/progress reporting for the testing. It includes directing the
Connectivity and Additional Functionality Tests.
The UIT Manager is provided by the DSP, in its role as overall systems
integrator and reports via the Integration and Acceptance Test
Manager to the DCC Test Programme Manager.

User Integration Testing Issue


Manager

The User Integration Testing Issue Manager is the person responsible


for overall management of testing issues. This includes
responsibility for the repository used to manage testing issues, and
chairing the Issue Review Board meetings.
The UIT Issue Manager reports to the UIT Manager and is provided by
the DSP, in its role as overall systems integrator.

DCC Service User Integration


Test Manager

The DCC will provide the DCC Service User Integration Test Manager
to manage UEPT. This role is described in the CTSD/SREPTSD
[7].

SP and SEC Party IT Manager

The SEC Party IT Manager is the person who has overall responsibility
for their Test Participants planning, preparation and conduct of the
testing, including responsibility for the successful and timely
completion of the testing.
The SP IT Manager is the person who undertakes the planning and
provisioning, and is responsible for providing support to those
executing the tests.
The SP and SEC Party IT Managers also have responsibility for the
tracking and management of all testing issues relating to their
organisation (alternatively they may delegate this to an Issue
Manager). This person attends the daily Issue Review Board and is
responsible for ensuring that testing issues observed by their
organisation are recorded in a timely manner and resolved as
agreed at the IRB.

SP SME/SEC Party SME

The Subject Matter Expert (SME) from the SP and from the SEC Party
is available to support IT Manager in resolving testing issues. The
SMEs are able to pin-point the cause of a testing issue and to
collaborate with their counterparts in the Service Providers as well
as other SEC Parties in order to determine responsibility for a
testing issue (whether or not it is a testing issue relating to their own
system), as well as the best way to resolve it.
Figure 21 Roles of Test Management

11.3 Governance
The DCC will provide overall governance of IT and liaise with the following stakeholders:

Page 56 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
DECC
SEC Panel
SEC Parties
Energy UK
Energy Networks Association
Ofgem.
The Forums and Boards relevant to IT are shown in the following diagram.

Figure 22 - IT Governance

The Test Design and Execution Group (TDEG) consists of Service Providers, SEC Parties,
Ofgem and DECC. The UIT Manager will report plans and progress at each TDEG meeting,
thus keeping all stakeholders informed and receiving their input.

11.4 Communication
The DCC Programme Test Manager will report plans and progress at each TDEG meeting to
ensure that SEC Parties, SPs, Ofgem and DECC are kept informed. Communication of plans
and progress to the wider community will take place via a report at each Implementation
Management Forum meeting (presented by the DCC Implementation Manager).
The DCC will hold bilateral meetings with SEC Parties as part of the communication process.
Page 57 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

11.4.1

Progress meetings

A regular progress meeting for DSP-led testing will be held by telephone, chaired by the UIT
Manager and attended by SP IT Managers and DCC Service User Integration Test Manager.
This meeting will review the consolidated progress report and discuss any testing issues
arising. The frequency will match the frequency of the progress reports:
Monthly during most of test preparation (in the first week of each month)
Weekly from one month prior to the start of testing and during the test execution itself
(each Friday).
Progress meetings for UEPT will be as described in CTSD/SREPTSD [7].
For the further details and the contents of the progress reports, see section 10 - Test Tracking
& Reporting.

11.4.2

Testing Issues and their resolution

The testing issue management process is described in general in Section 9 - Testing Issue
Management.
However, there may be urgent testing issues which require immediate communication with
the Test Participants involved in testing. For this reason, all SEC Parties and SPs are
expected to be available on email or the phone at all times during the hours of testing. If, for
example, the system has to be taken out of service, then this will be communicated by the
UIT Manager to the SPs and to the DCC Service User Integration Test Manager by both email
and text message. The email will be marked High Importance (with !) and will be sent to an
email address designated in advance. The text message will be sent to the designated
phone number. A response is required within 30 minutes (at least an acknowledgement) to
such communications. If there is no response, then the UIT Manager will place a phone call
to the designated phone number. If there is still no response, then the necessary action will
be implemented regardless. The UIT Manager will have a dedicated phone for this purpose.
Email exchange will be secured by TLS or an equivalent method.

11.5 Test Assurance


11.5.1

Introduction

The UIT Manager will assure the test scripts for Connectivity and Additional Functionality
Tests.
The DCC will assure IT as described in the Joint DSP/CSP Test Strategy [2], using the
following methods:
Quality Gate Reviews
Test Quality Audits
Product inspections

Page 58 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Document Review.

11.5.2

Quality Gate Reviews

There will be a Quality Gate Review between the Solution Test Stage of SIT and the Interface
Test Stage. This Review will be chaired by the DSP, approved by the DCC and attended by
the Service Providers.
As approver, the DCC will set the outcome of the Review as one of the following:
Solution Test Stage can close, Interface Test Stage can start, only minor (if any) remedial
actions required, or
Solution Test Stage cannot close until remedial actions have been completed, Interface
Test Stage can start, or
Solution Test Stage can close, Interface Test Stage cannot start until remedial actions
have been completed, or
Solution Test Stage cannot close, Interface Test Stage cannot start, until remedial actions
have been completed.
The Quality Gate Review meeting will be a short, checklist-driven event at which previously
assembled and validated evidence relating to the Exit and Entry Criteria is considered and
decisions made to close Solution Test and start Interface Test. It is expected that Quality Gate
Review meetings will be dry-run to enable testing issues to be identified and resolved in a
timely manner, and thereby avoid impacting the start date for Interface Test.
The Solution Test Stage will complete (and achieve its Milestone) on attainment of its Exit
Criteria. The Interface Test Stage will commence (and achieve its Milestone) on attainment of
its Entry Criteria.
Test Stage Complete certificates will be issued by the DCC as follows:

Item
No.

Test Stage Exit Criteria

Circumstance in which Test Stage Exit


Criteria achieved

Resulting Certificate

Interface Testing (North


Region) Stage Exit
Criteria

Test Success Criteria achieved in respect


of Interface Testing in North Region with
two (2) Large Supplier Parties in Great
Britain; and one (1) Network Party if
Network Parties are obliged to be ready
to commence testing from the start of IT;
and
all other Stage Exit Criteria achieved

Interface Testing (North)


Test Stage Complete
Certificate

Interface Testing (Central


Region) Stage Exit
Criteria

Test Success Criteria achieved in respect


of Interface Testing in Central Region
with two (2) Large Supplier Parties in
Great Britain; and one (1) Network Party
if Network Parties are obliged to be ready
to commence testing from the start of IT;

Interface Testing
(Central) Test Stage
Complete Certificate

Page 59 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

and
all other Stage Exit Criteria achieved
3

Interface Testing (South


Region) Stage Exit
Criteria

Test Success Criteria achieved in respect


of Interface Testing in South Region with
two (2) Large Supplier Parties in Great
Britain; and one (1) Network Party if
Network Parties are obliged to be ready
to commence testing from the start of IT;
and
all other Stage Exit Criteria achieved

Interface Testing (South)


Test Stage Complete
Certificate

Figure 23 Test Stage Certificates

11.5.3

Test Witnessing

The DCC will agree with the DSP a witness execution schedule for the Connectivity and
Additional Functionality Tests.
The protocol for this witnessing is described in the Joint Test Strategy.

11.5.4

Test Quality Audits

By prior agreement with the Service Providers on the timing, duration and scope, the DCC
may perform Test Quality Audits of Interface Testing.
The protocol for conducting these Audits and addressing queries/issues is described in the
Joint DSP/CSP Test Strategy [2].
Such Audits will include reviewing that the Exit Criteria for Interface Testing have been met.
Reviewing that Exit Criteria have been met may include a review of the IT Stage Completion
Report and the testing issues raised during IT. Upon issue of the IT Stage Completion
Report, the DCC will review it within 10 working days, or within a mutually-agreed period.

11.5.5

Product inspections

By prior agreement with the Service Providers on the timing, duration and scope, the DCC
may conduct on-site product inspections.
The protocol for conducting these product inspections and addressing queries/issues is
described in the Joint DSP/CSP Test Strategy [2].

11.5.6

Documentation review

The DCC may undertake a review of any documents (including Design documents) used in
testing by the Service Providers.
The protocol for conducting these reviews and addressing queries/issues is described in the
Joint DSP/CSP Test Strategy [2].

Page 60 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

12

Operational Acceptance Testing

Operational Acceptance Testing (OAT) is a separate testing activity and will have its own
scope, Test Approach and Test Plan documents. Its purpose is to verify that the solution:
can be installed and configured in the production environment
can be operated by the Service Management function under normal and exceptional
conditions
complies with the non-functional requirements
will meet its Service Level Agreements.
Such verification includes but is not limited to:
installation and configuration testing
end to end security testing, including penetration testing and the Security Operation
Centres
service monitoring and reporting
BCDR testing:
o

Business Continuity Planning testing

resilience and failover testing of solution components (e.g. equipment,


networks, databases)

Disaster Recovery testing

performance testing
testing of the Service Management processes
functional testing of service management tools
backup and restore testing.
OAT will be performed in the production environment before go-live, in parallel with the SIT
and IT phases. Where practical, the test results will be provided in the SIT and IT Completion
Reports, otherwise they will be reported separately to the SEC Panel for information
purposes.
.

Page 61 of 84
03/10/14 V1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Appendices

Page 62 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

A. Test Scenarios
i.

Connectivity Test Scenarios

For EIS, EES, GIS, RSA SEC Party roles:


Test Scenario
Title:

Connectivity to the DCC for EIS, EES, GIS or RSA SEC Party roles

Prerequisite:

Energy Relevant Party holds the role of EIS, EES, GIS or RSA

Connection to DCC Test Laboratory

Appropriate data (i.e. device details provided by the DCC)

SEC Party completion of Registration and Enrolment of Organisations and their representatives to the DCC Registration Authority following the processes
described in the Registration Authority Policies and Procedures (RAPP) in order to acquire access to the SMKI using the SMKI Portal and also to the SMKI
Repository.

Step
s

Description

Objective

Device
Prenotification

Populate
Inventory

Smart

Metering

Actions

Acceptance Criteria

1.

SEC Party will receive the success I0 response code as


defined in DUGIDS.

The SEC Party will receive the read the response that
will contain all data item in the SMI for the device
performed in Step 1.

Complete the following Service Request to


support connectivity of device:
DUGIDS SR 12.2 - Device Pre-notification

Read Inventory

Check device is registered on the


Smart Metering Inventory

1.

Complete the following Service Request to


support connectivity of device:

Page 63 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

DUGIDS SR 8.2 Read Inventory


3

Test verification

Verify that Service Request has


been performed

1.

Query Service Audit via the SSI for the


Service Requests performed in steps 1 and
2:
i.

DUGIDS SR 12.2 - Device Prenotification

ii.

DUGIDS SR 8.2 Read Inventory

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

Page 64 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

For ENO, GNO or OU SEC Party roles:


Test Scenario
Title:

Connectivity to the DCC for ENO, GNO or OU SEC Party roles

Prerequisite:

Energy Relevant Party holds the role of ENO, GNO or OU

Connection to DCC Test Laboratory

Smart Metering Inventory populated with a Device in own region

Step
s

Description

Objective

Actions

Acceptance Criteria

Read Inventory

1.

The SEC Party will receive the read the response that
will contain all data item in the SMI for the device(s).

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

Check device is registered on the


Smart Metering Inventory

Complete the following Service Request to


support connectivity of device:
DUGIDS SR 8.2 Read Inventory

Test verification

Verify that Service Request has


been performed

1.

Query Service Audit via the SSI for the


Service Request performed in steps 1:
DUGIDS SR 8.2 Read Inventory

Page 65 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

ii.

Additional Functionality Test Scenarios

Page 66 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Change of Supplier (On-Demand) Test Scenario for EIS SEC Party Role
Test Scenario
Title:

Change of Supplier (On-Demand) for EIS SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of EIS.

Connection to DCC Test Laboratory.

Dummy DCC Supplier setup as old supplier for take on the specified Devices.

Appropriate data (i.e. device details provided by the DCC).

Registration Test Data setup to match test scenario for the CoS date.

Step
s

Description

Change
Supplier
request

of

Objective

Actions

Acceptance Criteria

1.

Acknowledgement received for the DUGIDS SR 6.23


Update Security Credentials (CoS) Service Request by
requesting SEC Party.

Response is then delivered to the new supplier and the


old (losing) supplier is notified of completion of the
process via a DCC Alert N27.

Change of Supplier for each


specified Electric meter devices:
i.

ii.

Electric Smart Meter (3x


certificate updates)
HAN
Connected
Auxiliary Load Control
Switch (1x certificate
update)

Complete the following Service Request to


initiate the Change of Supplier process for
each specified device updating the required
certificates:
DUGIDS SR 6.23
Credentials (CoS)

Update

Security

Page 67 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

Update tariff

1.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 1.1.1


Update Import Tariff (Primary Element) Service Request
by requesting SEC Party.

DUGIDS SR 1.1.1 Update Import Tariff


(Primary Element)

Response is received for Update Import Tariff by


requesting SEC Party.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 6.8


Update Device Configuration (Billing Calendar) Service
Request by requesting SEC Party.

DUGIDS
SR
6.8
Update
Configuration (Billing Calendar)

Device

Response is received for the Update


Configuration by requesting SEC Party.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 4.4.1


Retrieve Debt and Credit Billing Data Log Service
Request by requesting SEC Party.

DUGIDS SR 4.4.1 Retrieve Debt and Credit


Billing Data Log

Response is received for the Debt and Credit Billing


Data Log by requesting SEC Party.

1.

Query Service Audit via the SSI for the


Service Requests performed in steps 1, 2, 3
& 4.

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

2.

Save screen print of audit trail query.

Update billing
calendar

Retrieval
of
Debt and Credit
Billing Data Log

Test verification

Update of import tariff (ondemand) for the specified Electric


Smart Meter device post CoS.

Update of billing calendar for the


specified Electric Smart Meter
device post CoS.

Retrieve Debt and Credit Billing


Data Log post CoS.

Verify that Service Requests have


been performed

1.

1.

Device

Page 68 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Change of Supplier (On-Demand) Test Scenario for GIS SEC Party Role
Test Scenario
Title:

Change of Supplier (On-Demand) for GIS SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of GIS.

Connection to DCC Test Laboratory.

Dummy DCC Supplier setup as old supplier for take on of the specified Devices.

Appropriate data (i.e. device details provided by the DCC).

Registration Test Data setup to match test scenario for the CoS date.

Step
s

Description

Change
Supplier
request

of

Objective

Actions

Acceptance Criteria

1.

Acknowledgement received for the DUGIDS SR 6.23


Update Security Credentials (CoS) Service Request by
requesting SEC Party.

Response is then delivered to the new supplier and the


old (losing) supplier is notified of completion of the
process via a DCC Alert N27.

Change of Supplier for each


specified Gas meter devices:
i.

ii.

Gas Smart Meter (3x


certificate updates)
Gas Proxy Meter (2x
certificate updates)

Complete the following Service Request to


initiate the Change of Supplier process for
each specified device updating the required
certificates:
DUGIDS SR 6.23
Credentials (CoS)

Update

Security

Page 69 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

Update tariff

1.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 1.1.1


Update Import Tariff (Primary Element) Service Request
by requesting SEC Party.

DUGIDS SR 1.1.1 Update Import Tariff


(Primary Element)

Response is received for Update Import Tariff by


requesting SEC Party.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 6.8


Update Device Configuration (Billing Calendar) Service
Request by requesting SEC Party.

DUGIDS
SR
6.8
Update
Configuration (Billing Calendar)

Device

Response is received for the Update


Configuration by requesting SEC Party.

Complete the following Service Request to


support verification of completion of
Change of Supplier:

Acknowledgement received for the DUGIDS SR 4.4.1


Retrieve Debt and Credit Billing Data Log Service
Request by requesting SEC Party.

DUGIDS SR 4.4.1 Retrieve Debt and Credit


Billing Data Log

Response is received for the Debt and Credit Billing


Data Log by requesting SEC Party.

1.

Query Service Audit via the SSI for the


Service Requests performed in steps 1, 2, 3
& 4.

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

2.

Save screen print of audit trail query.

Update billing
calendar

Retrieval
of
Debt and Credit
Billing Data Log

Test verification

Update of import tariff (ondemand) for the specified Gas


Smart Meter device post CoS.

Update of billing calendar for the


specified Gas Smart Meter device
post CoS.

Retrieve Debt and Credit Billing


Data Log post CoS.

Verify that Service Requests have


been performed

1.

1.

Device

Page 70 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Self Service Interface Test Scenarios

1. A SEC Party wishes to request the Postcode Coverage Report via the SSI using the DCC IDP (Identity Provider).
Test Scenario
Title:

Self Service Interface CSP SMWAN Network Coverage Request for EIS, GIS, EES, ENO, GNO, RSA or OU SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of EIS, GIS, EES, ENO, GNO, RSA or OU.

Connection to DCC Test Laboratory.

CSP Coverage Database to be populated with appropriate test data for each region.

SEC Party SSI login authentication via DCC IDP.

Step
s

Description

Objective

Actions

Acceptance Criteria

Login via DCC


IDP

Authenticate via DCC IPD.

1.

Open the web service for SSI logon and


complete SEC Party login via DCC IPD.

Login success and SEC Party will be presented with


Self Service Interface.

CSP SMWAN
Network
Coverage
Request

CSP SMWAN Network Coverage


Request via SSI.

1.

Select Planning & Diagnostics tab and


access the CSP WAN Coverage form.

SEC Party will receive a message of successful


coverage for the postcode district.

2.

Enters a valid postcode district and presses


a Check Coverage button.

SEC Party will be presented with a button that can be


pressed to initiate the download of a CSV file containing
the details for each postcode within the district specified.

Page 71 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

Test verification

1.

Verify that SEC Party is able to


perform CSP SMWAN Network
Coverage Request.

Press the button to download the CSV file


containing the details.

The SEC Party will be required to provide the download


CSV file to the DCC via email.

2. A SEC Party wishes to request the Postcode Coverage Report via the SSI using their own IDP (Identity Provider).
Test Scenario
Title:

Self Service Interface CSP SMWAN Network Coverage Request for EIS, GIS, EES, ENO, GNO, RSA or OU SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of EIS, GIS, EES, ENO, GNO, RSA or OU.

Connection to DCC Test Laboratory.

CSP Coverage Database to be populated with appropriate test data for each region.

SEC Party SSI login authentication via their own IDP.

Step
s

Description

Objective

Actions

Acceptance Criteria

Login via own


IDP

Authenticate via own IPD.

1.

Open the web service for SSI logon and


complete SEC Party login via own IPD.

Login success and SEC Party will be presented with Self


Service Interface.

CSP SMWAN
Network

CSP SMWAN Network Coverage

1.

Select Planning & Diagnostics option and


access the CSP WAN Coverage form.

SEC Party will receive a message of successful

Page 72 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Coverage
Request

Request via SSI.

Test verification

Verify that SEC Party is able to


perform CSP SMWAN Network
Coverage Request.

Actions

2.

1.

Acceptance Criteria

Enters a valid postcode district and presses


a Check Coverage button.

Press the button to download the CSV file


containing the details.

coverage for the postcode district.

SEC Party will be presented with a button that can be


pressed to initiate the download of a CSV file containing
the details for each postcode within the district specified.

The SEC Party will be required to provide the download


CSV file to the DCC via email.

3. A SEC Party wishes to confirm that they can access the Ordering of Communications Hubs and Auxiliary Equipment webpage via the SSI:
Test Scenario
Title:

Self Service Ordering of Communications Hubs and Auxiliary Equipment Request for EIS or GIS SEC Party Roles

Prerequisite:

Energy Relevant Party holds the role of EIS or GIS.

Connection to DCC Test Laboratory.

SEC Party SSI login authentication via either DCC IDP or own IDP.

CSP web based ordering system webpage is available.

Page 73 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

Login to SSI

Authenticate via IPD.

1.

Open the web service for SSI logon and


complete SEC Party login via IPD.

Login success and SEC Party will be presented with


Self Service Interface.

Access
to
Ordering
of
Communication
s Hubs and
Auxiliary
Equipment
webpage via the
SSI

The Ordering of Communications


Hubs and Auxiliary Equipment
webpage via SSI can be
displayed for all 3 CSP regions
(North, Central and South).

1.

Select Planning & Diagnostics option and


access the Forecasting and Ordering of
Communications Hubs and Auxiliary
Equipment option.

Selecting
the
Forecasting
and
Ordering
of
Communications Hubs and Auxiliary Equipment option
will present SEC Party with all 3 CSPs buttons.

2.

Select each CSP button in turn and check


that the relevant CSPs web based ordering
system is displayed.

Webpage for each 3 CSP regions (North, Central and


South) are displayed successfully.

Test verification

1.

Save screen print of each of the 3 CSP


region Ordering of Communications Hubs
and Auxiliary Equipment webpages.

The SEC Party will be required to provide the screen


prints of each of the 3 CSP regions Ordering of
Communications Hubs and Auxiliary Equipment
webpages to the DCC via email.

Verify that the SEC Party is able


to access each of the 3 CSP
region
Ordering
of
Communications
Hubs
and
Auxiliary Equipment webpages
via SSI.

Page 74 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Billing Test Scenario

1. DCC Enterprise System to receive for the billing for the Enrolled Smart Meter Report via the DCC Enterprise Systems Interface.
Test Scenario
Title:

DCC Enterprise System to receive for the billing for the Enrolled Smart Meter Report via the DCC Enterprise Systems Interface.

Prerequisite:

Completion on Install and Commission of Devices by SEC Parties.

SMI to be populated with Installed and Commissioned Devices.

DSP to pass scheduled report to DCC Enterprise System via the FTP Server

FTP folder available on the Enterprise System

Step
s

Description

Objective

Actions

Acceptance Criteria

DSP
report

1.

DSP to trigger scheduled Enrolled Smart


Meter Report.

Sending of Enrolled Smart Meter Report completing


successfully via FTP server.

2.

Send Enrolled Smart Meter Report via FTP


Server to DCC Enterprise Systems.

trigger

DSP
to
initiate
scheduled
Enrolled Smart Meter Report

DCC to access
report

Confirmation that DSP initiated


scheduled Enrolled Smart Meter
Report received

1.

DCC to access Enrolled Smart Meter


Report in the relevant FTP folder.

DCC is able to view the Enrolled Smart Meter Report.

Test verification

Verify the creation of the Enrolled


Smart Meter Report.

1.

Download report and convert the report into


PDF format

PDF is created of the Enrolled Smart Meter Report.

Page 75 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Page 76 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Reporting Test Scenario

1. The DCC creates the Monthly Service Provider Performance Statistics Report.
Test Scenario
Title:

The DCC creates the Monthly Service Provider Performance Statistics Report.

Prerequisite:

The CSPs to capture and then send performance data to the DSP via the CSP Management Interface.

The DSP to pass the performance data to the BI/MI System via the ESI.

Step
s

Description

Objective

Actions

Acceptance Criteria

Access BI/MI

DCC to login to BI/MI

1.

Open the BI/MI logon and complete DCC


login.

Login success and DCC presented with BI/MI.

Producing the
Monthly Service
Provider
Performance
Statistics
Report

Creation of the Monthly Service


Provider Performance Statistics
Report.

1.

Run Monthly Service Provider Performance


Statistics Report.

Report will be successfully displayed on-screen.

Test verification

Verify the creation of the Monthly


Service Provider Performance
Statistics Report.

2.

Download report and convert report into


PDF format

PDF is created of the Monthly Service Provider


Performance Statistics Report.

Page 77 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Volume Test Scenarios

1. Generating a volume of traffic over the DCC System, performed by EIS, GIS, ENO and GNO SEC Party Roles.
Test Scenario
Title:

Volume Test for EIS, GIS, ENO and GNO SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of EIS, GIS, ENO, or GNO.

Connection to DCC Test Laboratory.

All allocated Devices Installed and Commissioned.

SEC Parties have test evidence that they are able to perform the DUGIDS SR 4.1.1 Read Instantaneous Import Registers Service Request on the specified role.

Time and date provided to participating SEC Parties by the DCC for start and duration of Performance Test. (the test duration is constructed to allow for
consecutive parallel execution of Service requests by each SEC Party role).

SEC Party resource allocation confirmed for supporting the performance testing.

System to be subject to load (using a stub) to a proportionate volume to the load used in SIT.

Step
s

Description

Objective

Actions

Acceptance Criteria

Read
Instantaneous
Import
Registers

1.

Complete the following Service Request on


all allocated Device devices at the start of
Performance Test*:

Acknowledgement received for the DUGIDS SR 4.1.1


Read Instantaneous Import Registers Service Request
by requesting SEC Party.

DUGIDS SR 4.1.1 Read Instantaneous


Import Registers.

Response is received for the DUGIDS SR 4.1.1 Read


Instantaneous Import Registers Service Request by

To support the synthesising of


performance
test
data
by
performing
multiple
Service
Request by all SEC participating
for allocated Device sets for a
specified start and duration.

Page 78 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

Acceptance Criteria

*Service Requests are required to be


performed in a timely manner and repeated
across the allocated Device devices until
notified by the DCC.
2

Service
Request
verification

Verify that Service Requests have


been performed

1.

At the end of Performance Testing the SEC


Party queries Service Audit via the SSI for
the Service Requests performed in steps 1.

2.

Save screen print of audit trail query.

requesting SEC Party.

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

Page 79 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

2.Generating a volume of traffic over the DCC System, performed by EES, RSA and OU SEC Party Roles.
Test Scenario
Title:

Volume test for EES, RSA and OU SEC Party Roles.

Prerequisite:

Energy Relevant Party holds the role of EES, RSA or OU.

Connection to DCC Test Laboratory.

All allocated Devices Installed and Commissioned.

SEC Parties have test evidence that they are able to perform the DUGIDS SR 6.2.4 Read Device Configuration (Identity Exc MPxN) Service Request on the
specified role.

Time and date provided to participating SEC Parties by the DCC for start and duration of Performance Test. (the test duration is constructed to allow for
consecutive parallel execution of Service requests by each SEC Party role).

SEC Party resource allocation confirmed for supporting the performance testing.

System to be subject to load (using a stub) to a proportionate volume to the load used in SIT.

Step
s

Description

Objective

Actions

Acceptance Criteria

Read
Instantaneous
Import
Registers

1.

Complete the following Service Request on


all allocated Devices at the start of
Performance Test*:

Acknowledgement received for the DUGIDS SR 6.2.4


Read Device Configuration Service Request by
requesting SEC Party.

DUGIDS
SR
Configuration

Response is received for the DUGIDS SR 6.2.4 Read


Device Configuration Service Request by requesting

To support the synthesising of


performance
test
data
by
performing
multiple
Service
Request by all SEC participating
for allocated Device sets for a

6.2.4

Read

Device

Page 80 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach
Step
s

Description

Objective

Actions

specified start and duration.

Service
Request
verification

Verify that Service Requests have


been performed.

Acceptance Criteria

*Service Requests are required to be


performed in a timely manner and repeated
across the allocated Devices until notified
by the DCC.
1.

At the end of Performance Testing the SEC


Party queries Service Audit via the SSI for
the Service Requests performed in steps 1.

2.

Save screen print of audit trail query.

SEC Party.

The SEC Party will be required to provide a screen print


of the audit trail query to the DCC via email.

Page 81 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

B. Documentation RTM
This RTM will be revised each time a revision is made to this document.
The RTM is contained in the following Excel document:

Microsoft Excel
97-2003 Worksheet

Figure 24 Document RTM

Page 82 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

C. Functional Coverage RTM


The document attached below lists each statement regarding functionality from the
relevant SEC sections (E and H) and states where it is being tested. The testing
identified is in one or more of:
PIT
SIT
IT:
o

UEPT

Additional Functionality

OAT.
If a statement is relevant to IT and is not covered elsewhere, then an Additional
Functionality test scenario is described for it. These scenarios are summarized in
Appendix A - Test Scenarios.
Section G Security is wide-ranging and is verified in different ways. During IT, a duty
of care with regard to the test environment will be exercised, to ensure that there are no
breaches of security. Some security-related testing will be carried out in PIT, which will
include, but is not limited to:
recording system activity in audit logs
ensuring backed up data is protected
checking that expired data is securely deleted
detecting Anomalous Events.
Other tests, which can only be carried out on the Production environment, will be done
during OAT and these include:
detecting unauthorised connections and software
identifying deviations from expected system configuration
identifying
service

unauthorised

network

port/protocol/communication/application/network

detecting attempts to access resources


detecting data leaving the system
detecting denial of service events.

Page 83 of 84
03/10/14 Version 1.0
DCC Public

Smart Metering Implementation Programme


Interface Testing Approach

Microsoft Excel
97-2003 Worksheet

Figure 25 Functional coverage RTM

Page 84 of 84
03/10/14 Version 1.0
DCC Public