HSE

Health & Safety

Executive
Anevaluationofcurrentlegislative
requirementsforverificationof
elementscriticaltothesafetyof
offshoreinstallations
Prepared by BOMELLimitedfor the
Health and Safety Executive 2005
RESEARCH REPORT 397
HSE
Health & Safety
Executive
Anevaluationofcurrentlegislative
requirementsforverificationof
elementscriticaltothesafetyof
offshoreinstallations
BOMELLimited
Ledger House
Forest Green Road
Fifield
Maidenhead
Berkshire SL6 2NR
Thisreportpresentsanddiscussesresearchundertakentoevaluatetheoutcomesoftheamendment
made in 1996 to the Offshore Installations (Safety Case) Regulations (1992) which introduced a
requirement for the Verification of the Safety-Critical Elements of an offshore installation by an
IndependentandCompetentPerson(ICP).PriortotheintroductionofVerification,offshoreinstallations
were required to have a Certificate of Fitness and the present work, by using the supposed
continuation of that Certification regime as a counterfactual, attempted to determine the difference, in
cost / benefit terms, between the two regimes. However, in the event, there proved to be insufficient
cost data to allow a meaningful quantitative comparison, nor was it possible to isolate the effects of
Verification from other confounding factors. Therefore, a revised methodology was implemented and
qualitativedatawascollected,bystakeholderinterviews,andanalysedtoallowsubjectivecomparison
ofthetworegimes.Fromtheanalysisoftheevidencecollected,recommendationsforimprovementof
theVerificationprocesswereidentifiedforconsiderationbytheHSE.
This report and the work it describes were funded by the Health and Safety Executive (HSE). Its
contents,includinganyopinionsand/orconclusionsexpressed,arethoseoftheauthorsaloneanddo
notnecessarilyreflectHSEpolicy.
HSE BOOKS
Crown copyright 2005
First published 2005
All rights reserved. No part of this publication may be
reproduced,storedinaretrievalsystem,ortransmittedin
any form or by any means (electronic, mechanical,
photocopying, recording or otherwise) without the prior
writtenpermissionofthecopyrightowner.
Applicationsforreproductionshouldbemadeinwritingto:
LicensingDivision,HerMajesty'sStationeryOffice,
StClementsHouse,2-16Colegate,NorwichNR31BQ
orbye-mailtohmsolicensing@cabinet-office.x.gsi.gov.uk
ii
ACKNOWLEDGEMENTS
The time and information kindly given by the various HSE personnel, ICPs, Duty Holders and
Trade Union Officers who assisted with this study is gratefully acknowledged.
iii
iv
CONTENTS
Page
EXECUTIVE SUMMARY xiii
1 INTRODUCTION 1
1.1 BACKGROUND TO, AND PURPOSE OF THE EVALUATION 1
1.2 STRUCTURE OF THE REPORT 2
2 INFORMATION SOURCES 4
3 METHODOLOGY 6
3.1 INITIAL METHODOLOGY 6
3.2 REVISED METHODOLOGY 8
4 THE COUNTERFACTUAL AND CONFOUNDING INFLUENCES 9
4.1 THE COUNTERFACTUAL 9
4.2 CONFOUNDING FACTORS 9
5 DATA PRESENTATION AND DISCUSSION OF EVIDENCE COLLECTED 11
5.1 DISCUSSION OF EVIDENCE COLLECTED 11
6 COMPARISON OF OUTTURN TO THE COUNTERFACTUAL 16
6.1 COSTS TO THE HSE 16
6.2 COSTS TO THE EXCHEQUER 16
6.3 SECTORS AFFECTED AND COMPLIANCE COSTS 16
6.4 TOTAL COSTS TO SOCIETY 17
6.5 HEALTH AND SAFETY BENEFITS 17
6.6 OTHER BENEFITS 18
6.7 TOTAL BENEFITS 18
6.8 WEIGHING THE BALANCE 18
v
6.9 FINDINGS IN RESPECT OF THE EVALUATION OF THE
VERIFICATION REGULATIONS 18
7 UNCERTAINTIES 22
8 CONCLUSIONS, LESSONS AND RECOMMENDATIONS 23
8.1 CONCLUSIONS, LESSONS AND RECOMMENDATIONS
REGARDING THE EVALUATION OF THE REQUIREMENTS FOR
VERIFICATION 23
8.2 CONCLUSIONS, LESSONS AND RECOMMENDATIONS
REGARDING THE PRESENT RESEARCH 24
9 REFERENCES 26
APPENDIX A ANALYSIS OF EVIDENCE
APPENDIX B TENDER SPECIFICATION
APPENDIX C BLANK QUESTIONNAIRE
APPENDIX D DEFINITIONS, ABBREVIATIONS AND ACRONYMS
vi
LIST OF FIGURES
Page
Figure 1 Number of Respondents by Organisations Role A.2
Figure 2 Response Distribution for Q 1.6: Do you have experience of Certification as well as
Verification? A.4
Figure 3 Response distribution for Criterion (a): Moving responsibility from the CA to Duty
Holders A.5
Figure 4 Response distribution for Criterion (b): Requiring risk-based targeting with the
concept of SCEs. A.6
Figure 5 Response distribution for Criterion (c): Requiring independence of the Verification
body. A.6
Figure 6 Response distribution for Criterion (d): Removing the link between the Regulator and
the Certification / Verification body. A.7
Figure 7 Response distribution for Criteria (e): Other. A.7
Figure 8 Response distribution to Q 2.2: How do you view the HSEs involvement with
Verification? A.8
Figure 9 Results of keyword analysis of the responses to Q 2.3: What is HSEs principal
contribution; what more / less should be done; with what benefits / cost
(by organisational role) A.10
Figure 10 Results of keyword analysis of the responses to Q 2.5 What health and safety
benefits do you see of Verification over Certification (by organisational role). A.11
Figure 11 Response distribution for Q 2.7: If, on a scale of 1 to 5, current practices with
Verification deliver a safety level of 3 for SCEsif Verification requirements were now
removed (as in Question 2.6) where would you expect the safety of those elements currently
classed as safety-critical to rate in 3 years time given the scenario you envisage? A.13
Figure 12 Response distribution for Q 2.8: If on a scale of 1 to 5, current practices deliver a
safety level of 3 for SCEslooking back to around 1996 and the Certification regime, where
would you rate safety for those elements now classed as safety-critical? A.15
Figure 13 Results of keyword analysis of the responses to Q 2.9: Why do you think there is a
general preference for DHs to appoint external ICPs? (by organisational role) A.16
Figure 14 Response distribution to Q 2.10: If you are a Duty Holder, have you considered
appointing an ICP who was not previously an established CA? A.17
vii
Figure 15 Response Distribution for Q 2.11: If you are a Duty Holder and a non CA ICP was
considered, did you proceed with such an appointment? A.17
Figure 16 Response distribution to Q 2.12: Why do you think there was a general preference
for Duty Holders to appoint external ICPs? A.18
Figure 17 Response distribution for Q 2.13: If you are a Duty Holder, has 2nd Party
Verification (i.e. independent in-house Verification) been considered? A.19
Figure 18 Response distribution for Q 2.14: If you are a Duty Holder and 2nd party
Verification was considered, was this implemented? A.19
Figure 19 Response distribution of positive responses to Q 2.15: If you are a Duty Holder, do
you have more than one Verification body covering the complete range of work? (showing the
number of bodies used). A.20
Figure 20 Response distribution for Q 2.16: Are you satisfied with the legal relationship
between the Duty Holders and the HSE in the context of Verification? A.23
Figure 21 Response distribution for Q 2.17: Are you satisfied with the legal relationship
between the Duty Holder and ICPs? A.24
Figure 22 Response distribution for Q 2.18: Is the Duty Holder / ICP commercial relationship
conducive to rigorous independent scrutiny? A.24
Figure 23 Response distribution for Q 2.19: Are you satisfied with the legal relationship
between the ICPs and HSE? A.25
Figure 24 Keyword analysis of the responses to Q 2.21: Please identify examples of risk
control benefits / safety improvements brought about by Verification during operations? A.27
Figure 25 Response distribution to Q 2.22 showing the percentage assigned to the option
Satisfactory. A.28
Figure 26 Response distribution to Q 2.22 showing the percentage assigned to the option Slow
but low risk (i.e. tolerable) A.28
Figure 27 Response distribution to Q 2.22 showing the percentage assigned to the option Slow
high risk issues (Unacceptable) A.29
Figure 28 Results of keyword analysis of the responses to question Q 2.23 (by organisational
role). A.29
Figure 29 Respondents estimates (%) of the change to risk attributable to SCR /
Verification. A.31
Figure 30 Respondents estimates (%) of the change to risk attributable to PFEER. A.32
Figure 31 Respondents estimates (%) of the change to the DCR Wells Examination
Scheme. A.32
viii
Figure 32 Response Distribution for Q 3.1: Degree of agreement with AUPEC statement that a
key benefit was the risk assessment process, because of the increased focus and awareness of
higher risk categories, prioritisation of mitigation measures, identification of cost savings and
allocation of funds to those areas most in need. A.33
Figure 33 Response Distribution for Q 3.2: Degree of agreement with AUPEC statement that
Building on structured decision making processes, led to the targeting of risk reduction
efforts. A.34
Figure 34 Response Distribution for Q 3.3: Degree of agreement with AUPEC statement that
an early benefit of DCR (e.g. in relation to well ops?) was from identification of SCEs and
development of performance standards. A.34
Figure 35 Response distribution for Q 4.1: The benefits of Verification on risk and safety are
consistent with those experienced from the original SCR A.35
Figure 36 Response distribution for Q 4.2: Beyond the initial implementation of Verification,
inspection programmes have been the primary means of assuring the SCEs meet their
performance standards. A.36
Figure 37 Response distribution for Q 4.3: The scope and breadth of the CAs approval /
inspection process under Certification diluted the attention paid to elements now classed as
SCEs. A.36
Figure 38 Response distribution for Q 4.4: Certification provided a safety check in relation to
major accident precursors which may not be identified by the Verification process. A.37
Figure 39 Response Distribution for Q 4.5: The formal identification of SCEs and the
Verification process have achieved the intended regulatory purpose of allowing safety
management effort / resources to be appropriately focused on those items with the greatest
contribution to safety of the installation. A.38
Figure 40 Response Distribution for Q 4.6: For fixed installations, the schedule of SCEs
originally identified with the introduction of Verification has not changed significantly over the
subsequent 6-8 years. A.38
Figure 41 Response distribution for Q 4.7: Verification has been beneficial in highlighting the
importance of particular safety elements to those involved in their design, operation,
maintenance, verification, modification and future decommissioning. A.39
Figure 42 Response distribution for Q 4.8: Freedom for the Duty Holder in appointing ICPs
has improved the expertise Duty Holders draw on. A.39
Figure 43 Response distribution: for Q 4.9: The free market in procuring ICP services has
increased the value for money / cost effectiveness of the services. A.40
Figure 44 Response Distribution for Q 4.10: The effectiveness of Verification in relation to
major accident prevention could be improved A.41
ix
Figure 45 Response Distribution for Q 4.11: By placing responsibility on Duty Holders away
from CAs, Verification has improved standards of installation safety. A.41
Figure 46 Response Distribution for Q 4.12: ICPs have been over cautious in the definition of
SCEs and controls because safety-criticality is open to interpretation. A.42
Figure 47 Response Distribution for Q 4.13: Under Verification, inadequate attention is now
placed on assessing the suitability and condition of elements of installations which are not
safety-critical. A.43
Figure 48 Response Distribution for Q 4.14: The lack of prescription under Verification has
enabled cost-saving reductions in the scope and \ or frequency of physical inspections to be
implemented by the industry which may be open to debate from a rigorous risk-based
perspective. A.44
Figure 49 Response Distribution for Q 4.15: Over the period Verification has been in force,
cost savings have been achieved in the assurance of SCE integrity without compromising
safety. A.44
Figure 50 Response distribution for Q 4.16: Through life experience from Verification is
being fed back effectively to improvements for new design. A.45
Figure 51 Response distribution for Q 4.17: The safety benefits of Verification for major
accident prevention outweigh the costs of ongoing Verification activity. A.46
Figure 52 Response distribution for Q 4.18: The costs of establishing and implementing the
Verification regime were small compared with the benefits achieved over Certification. A.47
Figure 53 Keyword analysis of the responses, by organisational role, to Q 6.1: Please add any
additional comments you may have about your experiences that you consider are relevant to this
evaluation of the impact of the Verification regulations? A.48
Figure 54 Response Distribution for Q 7.1: Were the questions clear and straightforward to
answer? A.51
x
LIST OF TABLES
Page
Table 1 Responses to Question 2.6: If the legal requirement for Verification were now
removed, which one of the following scenarios would you envisage? A.12
Table 2 Distribution of Responses to Q 2.20: Overall within the industry, how well is the
concept of SCEs and the importance of their suitability / condition in preventing / controlling
major accidents understood by personnel within the following groups? A.26
xi
xii
EXECUTIVE SUMMARY
The Safety Case Regulations (SCR), introduced in 1992, were amended by the 1996 Offshore
Installations and Wells (Design and Construction, etc.) Regulations (DCR) to introduce
requirements for the Safety-Critical Elements (SCEs) of an offshore installation to be Verified
as suitable by an Independent and Competent Person (ICP).
Evaluation of regulations, post-implementation, is a formal part of the regulatory process and in
April 2004 the Health and Safety Executive (HSE) commissioned BOMEL Limited to carry out
research to evaluate the outcomes of the current legislative requirements for the Verification of
Elements Critical to the Safety of Offshore Installations (SCEs), hereafter the Regulations.
Prior to the introduction of Verification, offshore installations were required to have a
Certificate of Fitness by the Offshore Installations (Construction and Survey) Regulations (SI
1974/289), which were revoked in 1998.
The initial aims of the evaluation were:
1) To look back and evaluate the achievements and costs (benefits and dis-benefits) of
Verification compared with the previous Certification regime.
2) To look forward and make recommendations for improving the effectiveness of Verification
as an input to the [then] current review of the Safety Case Regulations.
Key questions underpinning the evaluation were whether the Regulations have succeeded in
maintaining or improving standards of safety on the installations they cover and whether the
associated net costs are proportionate. It was also important to establish whether the
Regulations could be made more effective or less burdensome.
The research entailed seeking evidence to allow comparison of the standards and costs of
Verification with those of Certification while giving consideration to various confounding
factors which affect the making of a like for like comparison. It was initially intended to make
the comparison by using the supposed continuation of the Certification regime as a
counterfactual and to thereby attempt to determine the difference, in quantitative cost / benefit
terms, between the two regimes. However, in the event, there proved to be insufficient cost data
to allow a meaningful quantitative comparison, nor was it possible to isolate the effects of
Verification from other confounding factors. Therefore, a revised methodology was
implemented and qualitative data collected, by stakeholder interviews. This data was analysed
to allow subjective comparison of the two regimes and to identify opportunities for potential
improvement to the Verification process for consideration by the HSE.
This report details the findings of the evaluation in regards to both the prevention of Major
Accidents and the implications of the Regulations for the various stakeholders associated
therewith. These are principally the HSE, the Duty Holders, the Independent Competent
Persons (ICPs) (responsible for undertaking the independent Verification of the Duty Holders
Verification Schemes) and the Offshore Workforce.
xiii
It is concluded that, as far as the limited cost data permits, the ongoing costs of Verification to
the HSE and the offshore industry are not significantly different from those associated with the
previous Certification regime, although the costs of the transition to Verification were,
apparently, substantial to both the industry and the ICPs. There is no direct evidence to
demonstrate that the requirements for the Verification regime have prevented, or may in the
future prevent, Major Accidents over and above the prevention that might have been associated
with continuation of the previous requirement for a Certificate of Fitness. However, there is
evidence to suggest that the overall changes to the regulatory regime post-Cullen, which
includes the requirement for Verification, have resulted in improved risk management and,
particularly, to improved planned maintenance systems and practices which implicitly indicate
an improved level of safety.
It is supposed that the costs of Verification to Duty Holders represent a very small component
of their operating budgets and the qualitative evidence collected showed that the majority of
stakeholders consider the benefits of the process outweigh the costs.
Areas identified for consideration by the HSE for implementation to improve the Verification
process include:
Intervention to raise the profile and understanding of the Verification process and
reinforce its added value to Duty Holders.
Clarification of the HSEs role in Verification and whether the HSE adds most value
by further offshore inspection to provide a further level of audit of the condition of
SCEs, seen by some as a duplication of the ICPs activities, or by undertaking
Inspections of the Duty Holders Verification systems and the competency of the ICPs.
Identifying and exploring options to enhance the exchange, evaluation, and distillation
of information and abstraction of learnings from Verification activities and HSE
Inspections and to improve dissemination of those learnings to all parties.
That an improved process is required to ensure Duty Holders respond in an
appropriate and timely manner to the findings presented to them by the ICPs. Options
for such improvement include giving the ICPs the powers to issue provisional
Improvement Notices or introducing a facility for the ICPs to bring deficiencies
directly to the attention of the HSE so that appropriate enforcement action can be
implemented.
Further consideration could be given to the effects that the commercial arrangements
between the Duty Holders and the ICPs might have on their independence and
whether an alternative arrangement might be beneficial.
Provide further guidance on the HSEs expectations for the workings of the
Verification process, specifically to attempt to achieve higher levels of consistency
and standardisation in its application but without introducing prescriptive measures
and thereby losing the flexibility associated with the present approach that allows the
Verification scheme to be tailored to the specific needs of each installation.
xiv
However, it is recommended that to make significant modifications to the requirements at this
point in time would be inappropriate as it is considered that after a long learning curve the
process is beginning to bed down and show benefit.
xv
xvi
1 INTRODUCTION
1.1 BACKGROUND TO, AND PURPOSE OF THE EVALUATION
The Piper Alpha disaster and the subsequent inquiry by Lord Cullen led to a review and
modification of the regulatory framework for offshore hydrocarbon exploration and production.
It essentially eliminated prescription and replaced it with a goal-setting regime requiring a risk-
based approach to managing safety with the Safety Case being the Duty Holders central
demonstration of the safety of his facilities and operations. The Safety Case Regulations (SCR)
were introduced in 1992 and were amended, by the 1996 Offshore Installations and Wells
(Design and Construction, etc.) Regulations (DCR), to introduce requirements for the Safety-
Critical Elements (SCEs) of an offshore installation to be Verified as suitable by an Independent
and Competent Person (ICP). The Offshore Installations (Construction and Survey)
Regulations (SI 1974/289), under which the prior Certification regime was established, were
finally revoked by DCR in 1998.
The objective of the SCR is the prevention of major accidents. Major Accidents are specified in
the regulations as:
a) a fire, explosion or the release of a dangerous substance involving death or serious injury to
persons on the installation or engaged in an activity on or in connection with it;
b) any event involving major damage to the structure of the installation or plant affixed thereto
or any loss in the stability of the installation;
c) the collision of a helicopter with the installation;
d) the failure of life support systems for diving operations in connection with the installation,
the detachment of a diving bell used for such operations or the trapping of a diver in a
diving bell or other subsea chamber used for such operations; or
e) any other event arising from a work activity involving the death or serious personal injury
to five or more persons on the installation or engaged in an activity in connection with it.
The overall aim of the requirements for Verification (as stated in the Guide to the Installation
Verification and Miscellaneous Aspects of Amendments by DCR to SCR) was to set in place
independent and competent scrutiny of the identification and condition of those parts of an
installation which are critical to safety, thereby obtaining assurance of the satisfactory condition
of such items for their intended purpose.
Whereas, previously, the Duty Holders appointed, from a list of bodies approved by the
regulator, Certification Authorities to inspect and award Certificates of Fitness for installations,
Verification shifted responsibility onto the Duty Holder to operate a Verification scheme. The
specific link between Verification and Safety-Critical Elements (defined as those associated
with the causation, prevention or mitigation of major accidents) was also intended to provide
more opportunity for the Duty Holders to make Verification arrangements customised to each
installation and to encourage effective targeting of resources at the areas of greatest risk. Thus a
1
regime that is risk and Major Accident prevention focused and subject to Verification
replaced the periodic, condition-monitoring based regime of Certification.
Evaluation of regulations, post-implementation, is a formal part of the regulatory process and in
April 2004 the Health and Safety Executive (HSE) commissioned BOMEL Limited to carry out
research to evaluate the outcomes of the current legislative requirements for the Verification of
Elements Critical to the Safety of Offshore Installations (SCEs), hereafter the Regulations.
The initial aims of the evaluation were:
1) To look back and evaluate the achievements and costs (benefits and dis-benefits) of
Verification compared with the previous Certification regime.
2) To look forward and make recommendations for improving the effectiveness of Verification
as an input to the [then] current review of the Safety Case Regulations.
Unfortunately, in the event, although certain information was available as input to the SCR
review, the present project was not completed in time to allow its full conclusions to be
available for that review.
In order to achieve the above objectives, it was not considered sufficient just to obtain measures
of value for money (VFM) but to also understand and take into account the ways in which the
regulations are effective and the interdependencies between these and the changing shape of the
industry.
Key questions underpinning the evaluation were whether the Regulations have succeeded in
maintaining or improving standards of safety on the installations they cover and whether the
associated net costs are proportionate. It was also important to establish whether the
Regulations can be made more effective (or more appropriate to the current and future industry)
or less burdensome.
The research entailed seeking evidence to allow comparison of the standards and costs of
Verification with those of Certification. However, it was also essential to give consideration to
various confounding factors which affect the ease with which a like for like comparison can be
made. This aspect is fully discussed in Section 4 below.
This report details the findings of the evaluation in regards to both the prevention of Major
Accidents and the implications of the Regulations for the various stakeholders associated
therewith. These are principally the HSE, the Duty Holders, the Independent Competent
Persons (ICPs) (responsible for undertaking the independent Verification of the Duty Holders
Verification Schemes) and the Offshore Workforce. Conclusions are drawn from the research
on the implementation and workings of the Verification process and recommendations are
presented for consideration for its continuous improvement.
1.2 STRUCTURE OF THE REPORT
Following this Introduction, Section 2 of the report discusses the rationale for the sources of the
information used for the research. Section 3 discusses the rationale for the methodology
2
adopted for the work and the need to revise this in the light of the outcome of a very poor return
of the Questionnaire used to survey the Duty Holders.
Section 4 addresses the adoption of the counterfactual and the confounding influences
impacting on the evaluation.
Section 5 discusses the evidence collected and presents the conclusions made from its analysis.
Section 6 describes the comparison of the outcome with the counterfactual, discusses the costs
and benefits associated with the regulatory change, the nature of the cost /benefit obtained. It
also highlights the findings on the specific areas of interest to the HSE as given in the Tender
Specification.
Section 7 discusses the uncertainties associated with the work and the conclusions drawn.
Section 8 presents the Conclusions, Lessons and Recommendations arising both from the
evaluation and from the research processes used.
Section 9 details References.
The Appendices A through D contain respectively the detailed Data Analysis, The Tender
Specification; the electronic Questionnaire and a table of the Definitions, Abbreviations and
Acronyms used in the report.
3
2 INFORMATION SOURCES
A prime consideration for the design of the research methodology was to understand on whom
the Regulations impacted and what the influencing factors might be in forming the various
perspectives of these parties (stakeholders) both now under Verification and in relation to the
previous arrangements under Certification. The key sources of evidence were identified as:
1. The Duty Holders and their Trade Associations
Although some commonality of perspective is to be expected, it is important to recognise that
there will be different issues facing individual Duty Holders for different installations (e.g. type:
fixed / floating / jack-up; installation function: drilling / production / accommodation; hazard
profile; age, etc.) and such issues as policy; personnel continuity / discontinuity; take-overs /
mergers, etc. Superimposed on this is the need to consider organisational changes, such as the
shift of responsibilities from central corporate functions to asset-led teams, which further affects
the determination of reference baselines and continuity of operation.
Data on Duty Holders, their installations and their ICPs were taken from a number of offshore
installation databases that BOMEL maintains for the HSEs Offshore Safety Division (OSD).
The following data were extracted:
The Offshore Data Browser indicated that of nearly 250 fixed installations /
complexes submitting safety cases, around one quarter are operated by new-entrants
to the UKCS. Around one sixth were installed post-1996 and have therefore always
fallen under the Verification provisions of SCR. About half of that number were
installed in the period 1992-1995 and, for these, all the Duty Holders are non-original
merged organisations (e.g. Conoco-Phillips, BP Amoco, etc.). Of the 115 facilities
installed pre-1992, which are still in operation, around one third indicate some
continuity in that they are still operated by Shell, whereas all the others are operated
by merged companies (or are otherwise categorised as operated by new entrants).
In the case of Floating Production, Storage and Offloading (FPSO) installations, none
of the vessels installed pre-1992 now remain in operation; however, there has been a
rapid expansion in FPSO numbers during the Safety Case era, with 4 installed in 1992,
a further 9 by 1996 and more than 15 thereafter. There is, therefore, no basis for a
time based regime comparison for FPSO installations; nevertheless the issues
associated with Verification for these important facilities were important to the study.
The final BOMEL database covers jack-up installations that have Safety Cases
submitted for actual, or potential, operation in UK waters. For the 52 jack-ups data-
based the ICPs are: ABS in 33 cases; DNV in 14 cases; and Lloyds Register in 5
cases. For this category of installations, with predominantly overseas owners (and
operated on a global basis) the requirements for Class are often the primary
demonstration of safety. Although no database information specific to semi-
submersibles was available, it is believed that the distribution of ICPs for semi-
submersibles would be broadly in the same proportion as that for jack-ups.
4
This data highlights both the variety of Duty Holder perspectives to be encompassed and the
need to recognise the limitations constraining historic comparisons.
2. The Independent Competent Persons (ICPs)
The databases also highlighted that it is the former Certifying Authorities (CAs) who dominate
as ICPs and that the continuity of their involvement puts them in a strong position to compare
both the onshore and offshore activities undertaken by CAs and ICPs under the previous and
present regimes respectively. Alongside this is the specific position taken by Shell who initially
adopted a second party ICP Verification route.
3. The HSE
The HSE is a stakeholder both in terms of the regulatory structure and the safety and integrity
outcomes that Verification is intended to promote. Because of the range of roles, and
disciplines within the organisation (e.g. Policy, Specialists and Inspection Management Teams),
the HSE provides an opportunity to obtain evidence from a variety of perspectives.
4. The Workforce and Trade Unions
It was considered essential that the views of the offshore workforce on the benefits, or
otherwise, of the change to the Verification regime was investigated. Because of the
impracticality of obtaining evidence by interviewing the work force directly, it was decided to
seek evidence by proxy, interviewing Officers from various Trade Unions that included offshore
workers within their membership. Although this might not be as effective as interviewing the
primary sources, it was considered that it would provide a balanced representation of the overall
views of the workforce.
The use of the above broad set of information sources was perceived to allow triangulation of
evidence from the different sources which in turn, allows more weight to be ascribed to
conclusions drawn from interpretations of the information.
5
3 METHODOLOGY
3.1 INITIAL METHODOLOGY
This section describes the rationale for, and the methodology initially adopted for the execution
of, the research and how it was necessary to modify the approach in the light of the very low
levels of completed returns of the electronic questionnaire issued to Duty Holders.
Evaluation research has been characterised as a form of applied research which aims to
produce information about the implementation, operation and ultimate effectiveness of policies
and programmes designed to bring about change. In contrast to what might be termed basic
research, the primary objective is not the discovery of new knowledge but rather the study of the
effectiveness with which existing knowledge is used to inform and guide practical action.
(Clarke and Dawson, 1999).
Evaluation research has no distinguishing methodology but utilises established social science
methodologies as considered most apposite to the particular study under consideration. In
discussing the methodological options available to evaluation researchers Clarke and Dawson,
1999 cite Patton (1986) that:
In evaluation the classic deductive approach is measuring relative attainment of predetermined
clear goals in a randomised experiment that permits precise attribution of goal attainment to
identifiable program treatments. In contrast, the classic inductive approach is goal-free
evaluation, in which the evaluator gathers qualitative data on actual program impacts through
direct observations of program activities and in-depth interviews with participants, all without
regard to stated, predetermined goals.
The un-stated goal of the introduction of the requirement for Verification is presumed to be the
elimination, or reduction in the probability of, the occurrence of Major Accidents. As such
events are, fortunately, statistically rare and the Verification regime has been in place only a
relatively short time, research to determine the extent to which such a goal has been met is not
considered to be statistically meaningful and the success or otherwise of the intervention must,
therefore, be assessed by consideration of proxies for the ultimate goal. It must also be
recognized that Major Accidents can occur regardless of the benefits, or otherwise, of the
Verification of Safety-Critical Elements as defined in the regulations as such definition does not
embrace management systems; Major Accidents can occur as a result of failure of such systems
and also by human factors.
The design of the research methodology for the present work had to recognise that the
regulations applied across the entire industry affecting all stakeholders and it would not be
possible to use randomised samples as a control group to establish a quantitative counterfactual
for comparison purposes. Furthermore, there was no pre-existing Verification specific base-line
work against which the present conditions could be evaluated. This suggests that an inductive
approach might be a preferred, if not the only appropriate, methodology for the present study.
Although there might be a preference for a predominately quantitative methodology, there was
recognition by the researchers that there might be significant difficulties in obtaining hard data
6
and that a qualitative approach might provide deep data that was more appropriate and
valuable for the present evaluation. Similarly, the disparate nature of the various stakeholders
suggests that adopting a constructivist approach might better identify and reflect the range of
perspectives held by such stakeholders. Again, whereas the use of a quantitative methodology
might imply (and arguably achieve) a more objective outcome from the research, this benefit
might be outweighed by the improved level of insight that might be obtained from a qualitative
approach involving significant interaction between the researcher and the subjects.
Furthermore, such an approach might be preferred as The researcher actually getting close to
the data in order to understand the actors point of view obtains social knowledge (Clarke and
Dawson, 1999).
However, rather than considering the quantitative and qualitative paradigms as mutually
exclusive, Cooke and Reichart, (1979) cited in Clarke and Dawson, 1999 state:
There is no need to choose a research method on the basis of a traditional paradigmatic stance.
Nor is there any reason to pick between two polar-opposite paradigms. Thus there is no need
for a dichotomy between the method-types and there is every reason (at least in logic) to use
them together to satisfy the demands of evaluation research in the most efficacious manner
possible.
Considered in cost / benefit terms, the present evaluation requires data regarding two relatively
distinct aspects of the implications of the requirement for Verification. The first of these relates
to the costs associated with its introduction and ongoing implementation and how these
compared with those associated with the previous Certification regime. This represents
relatively hard data and, if the necessary data records were available, could be readily
determined and is, thus, likely to be conducive to a quantitative approach. In contrast, the
second relates to the benefits of Verification. Here, there is likely to be an absence, or paucity,
of evidence of the causality that Verification has, per se, specifically eliminated or reduced the
probability of a Major Accident occurring; hence, a qualitative approach would appear to be
more appropriate.
It was therefore decided to adopt a contingency approach and to source evidence by survey
using a mixed methodology that adopts primarily quantitative techniques to address the costs of
Verification and part quantitative and part qualitative techniques to address the benefits / dis-
benefits in respect of the outcomes of Verification on the prevention / reduction of Major
Accidents. Likert scale responses were extensively used as the researchers have found these to
be very successful in surveys of this nature, to determine not only what the issues are but how
they rank in peoples judgements; often such information can not be provided in response to
open questions. The questionnaire was carefully crafted to map to the evaluation needs. For
example, where interviewees may not have directly comparable experience under Certification
the questions were framed to identify deficiencies in, or potential improvements to, Verification
and their possible outcomes in terms of cost and potential risk reduction.
The survey strategy was considered and it was concluded that the optimal approach, to ensure
there was sufficient weight in the findings, was to survey as complete a range as possible of
Duty Holders by means of an electronic questionnaire, followed up with a limited number of
interviews, to provide clarification and additional data, as considered necessary. Data from the
HSE would be primarily sought by face-to-face structured interviews with an appropriate range
of personnel as would evidence from the ICP organisations and from Trade Union Officers, the
7
latter as representative proxies of the perceptions of the workforce. The interviews would be
based on the questions in the electronic questionnaire but adapted to the specific nature of the
interviewees. The HSE made recommendations as to which of their staff might be willing to
participate and advised of two Trade Union Officers who might be willing to be interviewed. In
view of the limited number of ICPs, it was decided to try to interview representatives from all
the major players. Because the various offshore Trade Associations (e.g. UKOOA, BROA, etc.)
have high level safety management contacts within the Duty Holders, it was elected to distribute
the electronic questionnaire via their good offices. The HSE made initial contacts with each of
the associations to seek their agreement to assist and BOMEL followed these contacts up to
make detailed arrangements.
A key step was to pilot the questionnaire and test the analysis for postulated responses not only
to test for clarity but also to test the degree of interpretation possible.
Questionnaires were prepared for electronic completion by the respondents to allow automated
analysis of both categorised and free text data. Interview data was similarly coded and
analysed. This facilitated factor analysis and allowed the distributions of opinion within
different groups to be presented graphically, thus permitting ready examination of the range or
commonality of the responses.
3.2 REVISED METHODOLOGY
There was an extremely low response rate to the electronic questionnaire from Duty Holders.
This is attributed to its completion being a low priority item for the respondents; a general
industry saturation with questionnaires; difficulties in abstracting historic costs data as a result
of the lengthy time since the ending of Certification and the introduction of Verification this
latter being further aggravated by the changes in ownership that have occurred in the industry
during the interim and, possibly, the questionnaire being directed by the Trade Associations to
individuals who might not have been the optimum source for the evidence requested.
It was therefore decided, in conjunction with the HSE, to gain evidence from Duty Holders by
further interviews. Time and cost constraints meant that the number of interviews would, by
necessity, be limited. It was concluded that approximately ten interviews with carefully
selected Duty Holders would produce a data set that, although not as comprehensive as would
have been obtained from receipt of a significant number of the electronic questionnaires, would,
nevertheless be sufficiently broad to allow meaningful conclusions to be abstracted from the
data.
These additional interviews with Duty Holders were arranged and conducted and the results
added to those of the interviews with HSE Inspectors, ICPs and Trade Union Officers to form
the data set used for the analysis.
8
4 THE COUNTERFACTUAL AND CONFOUNDING
INFLUENCES
4.1 THE COUNTERFACTUAL
The principal objective of the study was essentially to determine the impact and efficacy (in the
widest sense) of the requirement for Verification, by ICPs, in eliminating / reducing Major
Accidents since its introduction, at present, and potentially in the future. More specifically, it
was to compare this with that achieved under the previous Certification regime. This required
the adoption of a counterfactual against which the degree of success and relative costs could be
measured. It was recognised that industry records were unlikely to be available to generate an
absolute Certification base-line and that an alternative approach would be needed. Following
much detailed consideration of this issue it was decided to adopt a counterfactual based on the
following methodology.
Current benefit would be estimated in terms of the value of avoidance of major accidents
through the Verification measures that are to be applied to Safety-Critical Elements and the
costs, to all parties, associated with these arrangements. It was hoped that data from both Duty
Holder and ICP records could be obtained and used to triangulate the rate of accident
precursors, the extent of ICP interventions etc. and to provide information for both the cost of
Major Accidents and the value of accidents avoided.
Taking the present day as the reference basis, structured questions were therefore devised to
address the difference in value offered by Certification when looking backwards. It was
anticipated that in the case of ICPs (former CAs) some hard data on relative costs may be
available given their continuity of involvement and this could be used to help check the Duty
Holders judgements. Similarly in looking to the future, questions could take the present day
position as a reference basis for any additional benefits or potential costs that may accrue.
An advantage of this approach, i.e. adopting a present day counterfactual, is that respondents
should be able to accommodate this and therefore give robust responses. This approach should
also help to ensure that there is a sound baseline provided for any future evaluation of
legislative or interpretive changes.
4.2 CONFOUNDING FACTORS
As noted in Section 1, Introduction, to this report, very significant organisational and economic
changes affecting the industry have had a major influence on a number of the factors that
Verification is seeking to address. In addition, Section 2, Information Sources, has illustrated
the many variations in United Kingdom Continental Shelf (UKCS) Duty Holders and their
individual installations to which the Verification arrangements apply. It is therefore very
important that attempts are made to differentiate the contribution, costs and success of the
actions flowing from the Regulations from other factors which may be reinforcing or detracting
from them. These factors have affected Duty Holders differently depending on type; life-cycle
phase; hazard potential; etc. of their various installations.
9
Such factors include the following:
That Verification and the concept of Safety-Critical Elements were introduced once
the risk based safety management philosophy of SCR was already well embedded
within the offshore industry. From the major accident focus of the Safety Case, the
identification of SCEs and their required performance standards (PSs) are a natural
development of the risk based approach and not a fundamental shift in philosophy.
Hence, the requirement for Verification forms an integral part of the overall offshore
safety regime and it was necessary to try to isolate the specific effects of moving from
Certification to Verification of Safety-Critical Elements from those associated with
the fundamental change to a permissioning regime and to the risk management
principles that have replaced prescription.
That the structure of the industry has altered radically since the introduction of
Verification. There have been many changes in ownership and the emergence of new
entrant Duty Holders to the UK Continental Shelf (UKCS). Other considerations
include: reductions in manning levels both on and offshore; fewer in-house technical
specialists; severe financial constraints affecting investment; increased use of sub-sea
developments and the use of Floating Production Storage and Offloading installations
(FPSOs) for deepwater and marginal field development; ageing fixed installations
(some now operating well beyond their design life); much reduced levels of offshore
modification (a key focus under Certification / Verification); advanced technology
developments with extensive use of IT; the inversion of risk and reliability
techniques to argue the case for reduced intervention; and other initiatives on a range
of safety matters including specific campaigns from industry and the HSE on reducing
hydrocarbon releases and improving integrity.
Recognition of these confounding factors is essential as it demonstrates the degree of
pragmatism that was necessary in designing the project to ensure that resources were optimally
directed to obtain rich, wide ranging and robust evaluation evidence that is fit for purpose for
the study.
Clearly industry stakeholders perspectives, and the manner in which duties are discharged in a
goal-setting regime, are not homogeneous. Nevertheless it is important that the evaluation took
both an holistic view (as the Regulations apply universally and are anticipated to have a net
positive effect in aggregate) whilst also providing insight into those areas of the industry where
it is more, or less, suited and where, for example, specific guidance might be beneficial.
Finally, it is important to recognise that there are situations in the industry where the same Duty
Holder uses different ICPs, where a Duty Holder has a range of installation types and where
ICPs work with different Duty Holders with similar installations, etc. There is, therefore a need
to identify, isolate and analyse key factors and where the use of statistical techniques is not
appropriate, to make interpretations from the qualitative evidence.
10
5 DATA PRESENTATION AND DISCUSSION OF EVIDENCE
COLLECTED
Appendix A to this report presents the detailed analysis, by response to each question asked, of
the evidence elicited by the research and notes the observations made on that analysis. Where it
is thought that the verbatim responses would be of particular interest these are also given. This
Section presents a more general discussion of the evidence, draws general conclusions in respect
of evaluation of the requirements for Verification and identifies potential opportunities for
system improvement.
5.1 DISCUSSION OF EVIDENCE COLLECTED
This Section provides discussion of the evidence collected where the questions are considered to
be of particular value as the responses provide evidence-based insight into the way the
regulations work in practice and how / where they might be improved. This discussion forms
the basis of the conclusions, lessons and recommendations presented in Section 8, below.
Because of the large amounts of data collected and the wide range of views expressed by the
respondents, the discussions are intended to highlight items that were raised by the responses
and perceived to be of key interest rather than to indicate any validity of the views expressed or
robustness of the information provided.
5.1.1 Question 2.3: What is the HSEs principal contribution; what more / less
should be done; with which parties; with what benefit / cost?
Some key issues emerged in the responses to this question.
Firstly, there was a clear divergence of views as to whether the HSEs resources should be
primarily utilised for physical inspection of offshore installations and thus provide a second
level of audit to the findings of the ICPs or whether the resources are better spent auditing the
Duty Holders Verification schemes and the ICPs. There was also a strong view that the ICPs
findings should be subject to a much higher level of scrutiny and review by the HSE such that
more learnings could be abstracted and communicated to the industry. This latter would, in
turn, require more communication of ICP findings to HSE personnel. A requirement for formal
reports from the ICPs to the HSE, perhaps annually, was often endorsed. An associated item
was seen to be the need for the HSE to be aware of deficient response to the ICPs comments, by
the Duty Holder, so as to provide an early opportunity for enforcement action in these cases.
Conversely, there was a perceived benefit in the HSE sharing the findings of their inspections
with the ICPs.
A further issue was that of consistency and standardisation in the whole Verification process.
For instance, the provision of more guidance to the ICPs and Duty Holders of the HSEs
expectations was suggested. Consistency of the HSE Inspectors approaches to Verification
was also raised. However, there is obviously a fine line here in addressing these issues without
moving back towards prescription.
11
5.1.2 Question 2.5: What health and safety benefits do you see of Verification
over Certification?
Responses to this and to Questions 2.7 and 2.8 suggest that there is generally a perception of
little change to overall safety as a result of the change from Certification to Verification.
Furthermore it is practically impossible to extricate any benefits and specifically attribute them
as an outcome of Verification rather than the overall changes resulting from risk based and
goal-setting regimes embodied in the entire post-Cullen regulatory regime. There is, however,
a consensus that these changes, including the requirement for Verification, has led to
significantly more structured, visible and rigorous systems for inspection, testing and
maintenance of SCEs. Although this change must be considered positive, there is, however,
apparently no explicit evidence that it has, in itself, improved safety in respect of Major
Accidents.
5.1.3 Question 2.21: Please identify examples of risk control benefits /safety
improvements brought about by Verification during operations.
As in the responses to Question 2.5, discussed above, the examples provided of the main
improvements in safety were seen as process improvements using the risk based approach with
improved planned maintenance systems. Although the requirement for Verification must have
enhanced this change, other factors such as the requirement for Safety Cases and the
requirements of PFEER and DCR will have also contributed. Isolating specific improvements
and attributing them to Verification is, apparently and as noted above, practically not possible.
5.1.4 Question 6.1: Please add any additional comments you may have about
your experiences that you consider are relevant to this evaluation of the
impact of the Verification regulations?
Responses to this question generally create the impression that Verification is seen as a valuable
adjunct to a risk based regime. It was thought to promote confidence that the right things were
being done and being done correctly and to improve understanding of the importance of SCEs
across the workforce. The goal-setting approach also allows flexibility for risk based
assessment of options when performance standards are not met. Although it was considered
that the ICPs provide a valuable contribution to safety, concern was expressed as to a perception
of declining ICP surveyor competence (although a late telephone contribution to the study, from
a Duty Holder, expressed the totally opposite opinion), more focus on paperwork than
hardware and reduced continuity in both employment of an ICP by Duty Holders and of ICP
dedication of particular personnel to a Duty Holder.
As noted above, the keyword analysis of the responses to this question indicated four areas of
significant concern viz: the implementation of the regulations, the ICP \ Duty Holder power
balance, utilisation of information from the Verification systems and communication.
It was evident from the responses from all stakeholders, but predominantly by Regulators, that
the implementation process for the requirements for Verification was a key issue. The structure
of the regulations stemming from the Cullen Report (The Hon. Lord Cullen, 1990) was found to
be complex and it was suggested that the regime could be simplified and clarified by the issue
of one set of regulations to replace SCR, PFEER, DCR etc. It was noted, by one Regulator, that
the HSE had assigned a very limited number of Inspectors to the implementation process and
12
that, although this promoted consistency, it was perceived to have induced a restricted world-
view of the requirements.
There were recommendations that the HSE should issue more guidance on their expectations for
the requirements for Verification or as this would not only assist the Duty Holders but would
improve the perceived lack of consistency in the approaches of the HSE Inspectors and the
ICPs.
A relatively weak implementation by the HSE, with the consequential poor understanding by
all stakeholders leading to a long learning curve, was considered to have led, in some cases, to
senior management seeing no value in the process. The problems with poor understanding
were reported to have led to poor Verification schemes and to their poor implementation by the
Duty Holders, this latter reducing the effectiveness of the process. However, there is also a
developing perception of optimism stemming from evidence that the system was now bedding
down and starting to work effectively.
Concern was also expressed (again predominantly by the Regulators) as to relative power of
the ICPs and the Duty Holders. Although there was recognition by some respondents that the
power available to be exercised by the CAs under Certification was perhaps excessive, it was
now, perhaps, too weak as the ICPs have no sanctions available if their recommendations are
not followed up or are given low priority with delayed implementation; this leads to ICP
frustration. It was mentioned that this was aggravated by the commercial pressure on the ICPs
from the Duty Holders such as to limit the ICPs scope of work and a perceived unwillingness
to take enforcement action by the HSE, despite their issuing frequent letters to Duty Holders
expressing HSE dissatisfaction.
Many respondents, particularly the Regulators, ICPs and a TU Officer, were concerned that the
findings from Verification were subject to too little review and analysis by all parties but
especially by the HSE who should be positioned to take an overview of the information, that
valuable lessons were thereby not identified and best practice not being transferred broadly
across the industry. This question of the perceived best use of the HSE resources links with the
discussion relating to Q 2.3, above, and the fundamental aspect of the overall HSE vision of
the process discussed in Section 17 below.
As so often, many respondents promoted a call for more and better communication by all
stakeholders. This was related to improved guidance on Verification from the HSE; sharing of
findings from Verification and HSE Inspection amongst all parties; and more abstraction and
distribution of both issues and best practice.
5.1.5 Question 6.2: What steps would you recommend to improve the
effectiveness of the impact of Verification and what benefits would these
bring?
Many suggestions for improving the verification process were offered by the interviewees. A
fundamental suggestion was to rewrite the offshore safety legislation to simplify and clarify the
requirements. There was also a suggestion that the ICPs should be government funded to
ensure their independence.
13
It was considered that there was still a need to promote the process and improve Duty Holders
understanding of the process so that its value was fully recognised. A further key issue related
to the use of the findings of the ICPs viz:
The quality of reporting the findings could be improved to enhance Duty Holder
understanding of their significance.
There should be scrutiny and analysis of the findings by the HSE. This could be
facilitated by regular reports from the ICPs to the HSE, perhaps annually.
There was a need to ensure the ICP findings were actioned in a timely manner by the
Duty Holders. This might be achieved by an increase in ICPs powers (e.g. allowing
them to issue provisional improvement notices) or by the ICPs advising the HSE of
Duty Holder deficiencies so that HSE could take enforcement action.
The benefits of improved communication were emphasised. Improvements suggested included:
Clearer HSE remit
Better HSE Guidance to ICPs
Mutual secondments between the HSE and ICPs
HSE publication of the outcomes of the Verification process and associated actions.
Suggestions for process improvements included:
More HSE inspections of Duty Holders, particularly audit of Duty Holders
Verification systems
Less documentation and less documentation review
More examination and testing
Early consideration of SCEs in design
Ensured HSE and ICP competency (there was a concern expressed over reduced
surveyor competence and experience).
Associated benefits that were considered potential outcomes from the above improvements
included:
Improved safety and consequential improved performance
Better understanding of the process by all stakeholders (including the Grass Roots
workforce)
Improved understanding of the HSEs role and expectation
14
ICP independence
Clarity, standardisation and consistency
Improved HSE awareness of the process and its outcomes
Improved Duty Holder buy-in, responsiveness and self-regulation.
15
6 COMPARISON OF OUTTURN TO THE
COUNTERFACTUAL
As noted in Section 3, it was initially intended to estimate current cost / benefit in terms of the
value of avoidance of Major Accidents attributable to the Verification measures that were to be
applied to Safety-Critical Elements. This was to be achieved by obtaining data from both Duty
Holder and ICP records and using it to triangulate the rate of accident precursors, the extent of
ICP interventions etc. and to thus provide information for both the cost of Major Accidents
and value of accidents avoided. In the event this methodology was not effective because of
lack of forthcoming data and the impossibility of isolating the benefit, if any, accruing from the
requirements for Verification from the other confounding factors, particularly those stemming
from the previous change to the goal- setting regime and the initial Safety Case Regulations.
Therefore, making meaningful comparisons of the relative cost / benefits of the Verification and
previous Certification regimes was not achievable. However, the qualitative data obtained
suggests that there has been net benefit in safety terms as a result of the regulatory change and
that, although transition costs were significant, ongoing costs associated with Verification are
probably similar to those under the Certification regime (in real terms) and that therefore an
improved, but non-quantifiable positive cost / benefit has been achieved. The responses to
Question 4.18 illustrated in Figure 5.52 in Appendix A tend to support this conjecture.
6.1 COSTS TO THE HSE
The HSE provided information on the costs to the organisation that were associated with the
requirements for Verification. It was reported that, although time writing records were not
available, two to three personnel were assigned to the management of the Verification
requirements and that this was similar to the resources previously applied to the Certification
regime. It would also appear that the introduction of Verification had little effect on the
resources employed by the Inspection Management Teams. It is concluded that the change from
Certification to Verification had a limited effect on the HSEs overall costs and it is expected to
do so in the future.
6.2 COSTS TO THE EXCHEQUER
No costs to the Exchequer have been identified other than the direct costs incurred by the HSE
(see Section 6.1 above) and the suspected minimal loss of Corporation Tax associated with the
possibly increased costs of the Verification to industry marginally reducing the Duty Holders
profits..
6.3 SECTORS AFFECTED AND COMPLIANCE COSTS
No Sectors were affected other than the Offshore Industry. The level of financial data obtained
from the survey is insufficient to allow any statistically meaningful statements to be made
regarding the compliance costs to industry of implementing and providing continuing
compliance with the requirements for Verification. However, from the little cost data that was
obtained, the following information may be of interest.
16
Only three respondents offered cost information in respect of Section 5 of the questionnaire.
One further data point was established by interview. The information received can not be
considered typical for the industry at large because of the small sample size and the fact that the
principal data received was in respect of MODUs (where the cost of Class requirements is a
confounding factor) there being no data received from Duty Holders of fixed installations.
6.3.1 Preparation costs of Verification Schemes
A Duty Holder having 10 semi-submersibles estimated preparation costs for all these units at
315,000 (+/-25%) with a requirement of 1250 mandays.
An ICP put its preparation costs at 250,000 (+/-25%).
6.3.2 Implementation Costs
The semi-submersible Duty Holder put their implementation costs at 100,000 with a
requirement of 650 mandays.
6.3.3 Ongoing costs
At interview, one Fixed Structures Duty Holder put ongoing Verification costs at 40,000 per
installation per annum. This is not considered to be significantly different from the costs of the
previous Certification regime.
No responses were received to Question 5.17: What cost savings or increases have been
experienced in the overall processes and activities for assuring integrity in the period since
Verification replaced Certification.
6.3.4 Other cost information
Where MODUs were Classed and the classification contributed to Verification responses
indicated that the costs of class were in the order of 70% to 80% of the total Verification costs.
6.4 TOTAL COSTS TO SOCIETY
There were, apparently no costs to society other than the direct costs to the HSE, the Duty
Holders and the ICPs. However, as discussed further below, the benefits to society in costs
saved to Government, Corporations, and Individuals and the associated human costs is,
potentially, very significant if the requirements for Verification have avoided, or may in the
future avoid, one or more significant Major Accidents such as the Piper Alpha tragedy.
Whether the Verification requirements have to date avoided such an event is not known nor can
the likelihood of such future prevention by Verification be determined or meaningfully
estimated.
6.5 HEALTH AND SAFETY BENEFITS
There was no intention to provide health benefits as a result of the introduction of the
requirements for Verification. The only possible health benefit provided by the initiative that
has been identified is a possible reduction in the stress levels of offshore workers that
17
Verification might induce by providing greater assurance against the possible occurrence of a
Major Accident.
Although, as has been stated above, it is not possible to isolate and attribute specific safety
benefits to Verification (and there is even some limited opinion that Verification provides less
safety assurance than the previous Certification regime) there are, nevertheless some indicators
that an improved safety regime has emerged from the post-Cullen regulatory regime and that
Verification has contributed to this (but probably in a limited manner). Such improvements
tend to be attributed to the risk based perspective now taken in respect of facility safety and the
improved planned maintenance systems now in place that provide a more structured and visible
approach to inspection, testing and maintenance. Enhanced reporting and documentation
systems also allow more thorough analysis and use of the data gathered and thereby enhances
learning and understanding of the facilities performance and early identification of potentially
dangerous trends.
6.6 OTHER BENEFITS
No specific other benefits were identified as part of the evaluation of the introduction of the
requirements for Verification over and above those discussed in Section 6.5 above.
6.7 TOTAL BENEFITS
The total benefits of the introduction of the requirements for Verification are seen as limited to
the health and safety benefits discussed in Section 11 above.
6.8 WEIGHING THE BALANCE
As discussed above, it did not prove to be possible to make a meaningful quantitative
assessment of the benefits and related costs of the introduction of the requirements from
Verification. This resulted from the lack of information from the Duty Holders as to the cost
associated with its introduction and ongoing resource needs and the numerous confounding
factors. For similar reasons it is not possible to make a meaningful cost / benefit comparison
with the preceding Certification regime.
Nevertheless, there is qualitative evidence, that it is the opinion of many of the respondents
(although there are dissenting voices), that the introduction of Verification represents an
improvement over Certification.
6.9 FINDINGS IN RESPECT OF THE EVALUATION OF THE VERIFICATION
REGULATIONS
The tender specification (Appendix A) asked for the work to address a series of specific issues
in support of the overall aims (P2). These have been addressed throughout the report but are
summarised here for convenience and to demonstrate completeness.
The move from Certification to Verification was intended to improve standards of installation
safety by removing responsibility from the Certifying Authority onto Duty Holders. Has this
worked? If not, why not?
18
The response to Question 2.1 indicates that respondents generally gave a high weight to this
change in assuring offshore safety, although one regulator and one TU Officer thought this was
a detrimental move. However, the responses to Question 4.1 present a more complex picture.
Here there is a marked contrast in the responses of most Duty Holders, who expressed
Agreement and most Regulators and ICPs who either expressed Neutrality or
Disagreement. A preference for the Certification regime by the previous CAs is perhaps not
unexpected but the disagreement by the Regulators is harder to rationalise.
Has the concept of identifying Safety-Critical Elements led to improvement in risk control?
On the assumption that additional safety management effort leads to improved risk control, the
response to Question 4.5 indicates that there is generally agreement that the concept of
identifying Safety-Critical Elements has led to significant improvement in the control of risk.
What factors determine the choice of in-house or external Independent Competent Persons to
carry out the Verification arrangements?
The response to Question 2.13 indicates that the principal factors affecting this aspect are
insufficient in-house resources; comfort using a 3
rd
Party; poor industry experience with 2
nd
Party approach; and difficulty demonstrating independence.
Why is there an apparent preference for the use of an external Independent Competent Person?
Similar to the above.
How realistic were the original cost and benefit estimates in the Compliance Cost Assessment
and have they been achieved?
Insufficient cost data were obtained to answer this question and the provenance of the data used
in the Compliance Cost Assessment was not known. Therefore, making a meaningful
comparison was not possible. However, it appears that transition costs were probably
significantly more than estimated in the Compliance Cost Assessment and there has been little
reduction in costs to the Duty Holders and possibly a marginal increase. Benefits appear to be
in accordance with expectation.
Is the customer / contractor relationship between the Duty Holder and the Independent
Competent Person conducive to rigorous independent scrutiny e.g. do commercial pressures
and competition for work compromise the depth and competence of scrutiny?
The responses to Q 2.18 indicate difficulties with the commercial relationship between the Duty
Holders and the ICPs as the pressure on the ICPs to minimise costs and the ability of the Duty
Holders to prescribe scope might conflict with ICPs acting fully independently in their
activities. All of the Regulators, and all but one of the ICP responses, expressed dissatisfaction
with the arrangements; however, all but one of the Duty Holders expressed satisfaction with this
relationship. Please also see the extensive comments received on this issue and reported in
Appendix A.
How do senior management in industry see the Value of Verification or does it add value?
19
The answer to this question is perceived to vary significantly with the Duty Holder involved but
the responses to Question 2.4 show that 13 of the 19 respondents, to this question, believed
senior management thought it added some value but not necessarily good value for money; 8
saw it adding cost-effective value or high value and only 3 believed senior management saw no
value in the process.
Are the current arrangements effective in identifying deficiencies in Duty Holders
arrangements for maintaining the integrity of Safety-Critical Elements?
The evidence collected overall suggests that the arrangements are generally thought to be
effective in this context; however, the response to Question 4.10 gives a strong indication that
there is much room for further improvement in the process.
Could / should the arrangements be more effective and if so how?
As noted above many respondents saw opportunities for improvement in the process. Please see
the response to Question 6.2 in Section 5, above, and the recommendations from the present
work in Section 17, below.
Is the relationship between the HSE and the Independent Competent Persons appropriate?
It was generally thought, particularly by the Regulators and the ICPs that this process could be
improved. There was recognition, from both parties, that improved communication including
mutual exchange of findings from HSE Inspections and from the ICP Verification activities
would be beneficial. Please also see the analysis of the responses to Question 2.19 in Section 5
above.
Does the requirement for the competent persons to be independent add value to the process?
How?
A significant number of respondents questioned this as can be implicitly seen from the
responses to Question 2.6 where 11 of the 25 responses indicate that, if the requirements for
Verification were withdrawn, it was expected that Duty Holders would not employ independent
Verifiers, although some Duty Holders thought the independence added a further layer of
confidence that the management of SCEs was appropriate.
Are there any intervening variables apart from the HSEs contribution? For example the state
of the economic cycle and the structure of employment; working and management practices
(e.g. the use of subcontract labour); production techniques, - and the extent to which hazardous
practices or pieces of equipment have been engineered out; the state of scientific knowledge.
As discussed in Section 4 above, there are many confounding factors that make it essentially
impossible to isolate the benefits, or otherwise, resulting from the requirements for Verification.
What is the contribution of the HSE to the achievement of the outcomes (i.e. excluding other
stakeholders and intervening variables)?
As discussed above there is significant difficulty in assigning all, or part, of any outcomes
specifically to Verification and it is as hard, if not harder to then attribute that part of any
outcome to any one cause. However, it is known that the HSEs help in implementation of the
20
requirements made a significant contribution (although there was some adverse comment on the
implementation process) and subsequent Inspections have resulted in Improvement Notices
being issued to Duty Holders when their arrangements have been perceived to be deficient. Not
directly related to Verification, but seen to achieve significant safety improvements was the
HSEs KP1 programme on hydrocarbon releases but it is not feasible to definitively attribute a
reduction of releases to specific initiatives.
Were any problems encountered with the study? How many of these problems can be avoided
in future studies?
There were significant problems encountered in executing the work, most prominently the low
return of the electronic questionnaire by the Duty Holder as discussed in Section 3 above.
Research of this kind is dependent on the goodwill and contributions of the stakeholders and is
generally out-with the control of the researcher. The problems encountered with the project and
recommendations for avoiding such problems in the future are detailed in subsection 8.2 below.
21
7 UNCERTAINTIES
As the available cost data are insufficient to make any meaningful quantitative analyses,
statistical uncertainty is not a consideration for this report.
Although the offshore industry is generally consistent in its technical activity it can be
considered significantly diverse in its individual corporate cultures. It is not, therefore,
surprising that there was often a strong divergence in the responses received to the questions
asked in the interviews, as illustrated in the graphics presented in Appendix A. This divergence
was not only between the different stakeholder roles but also from individuals within those
roles. Added to any uncertainty introduced by this cultural divergence is the further uncertainty
introduced by the researcher in interpretation of the responses.
It is important to give due consideration to the above in respect of the conclusions drawn in this
report and the recommendations made from those conclusions.
22
8 CONCLUSIONS, LESSONS AND RECOMMENDATIONS
8.1 CONCLUSIONS, LESSONS AND RECOMMENDATIONS REGARDING THE
EVALUATION OF THE REQUIREMENTS FOR VERIFICATION
8.1.1 Conclusions
It is concluded that, although a reduction in potential for (or occurrence of), Major Accidents
since the introduction of the requirements for Verification (or that they may do so in the future)
can not be definitively attributed to those requirements, there is qualitative evidence that the
Verification process has contributed to improved risk management, has raised standards of
Planned Maintenance systems (planning, inspection, testing, maintenance recording and review)
and has focused activities on critical systems and components. This suggests an implicit
improvement in safety.
It is also concluded that, although there were significant costs to the industry when preparing for
and implementing the requirements for Verification, ongoing costs are not significantly
different from those of the previous Certification regime and consume a small component of the
Duty Holders overall operating budgets.
Therefore, it would appear, from the views expressed, that (although it can not be positively
identified and quantified) the change from Certification to Verification improved the cost /
benefit over that previously experienced.
8.1.2 Lessons
The principal lesson appears to be the time it takes for changes to regulations to bed down and
become effective. To minimise future learning curves associated with such changes it is
important to apply appropriate resources to ensure the stakeholders hold an aligned perspective
with the HSEs expectations from the revised regulations and that the expected value to society,
the industry, and the individuals that the revised regulations are expected to have (and why) is
clearly communicated to those affected so as to obtain early understanding and buy-in.
Piloting of revised regulations, although understandably difficult, could show benefits in ironing
out issues prior to general application of the modified requirements.
8.1.3 Recommendations
The following recommendations are offered to the HSE for their consideration as potential ways
to improve the outcomes of the requirements for Verification:
Intervention to raise the profile and understanding of the Verification process and
reinforce its added value to Duty Holders.
Clarification of the HSEs role in Verification and whether the HSE adds most value
by further offshore inspection to provide a further level of audit of the condition of
23
SCEs, seen by some as a duplication of the ICPs activities, or by undertaking
Inspections of the Duty Holders Verification systems and the competency of the ICPs.
Identifying and exploring options to enhance the exchange, evaluation, and distillation
of information and abstraction of learnings from Verification activities and HSE
Inspections and to improve dissemination of those learnings to all parties.
That an improved process is required to ensure Duty Holders respond in an
appropriate and timely manner to the findings presented to them by the ICPs. Options
for such improvement include giving the ICPs the powers to issue provisional
Improvement Notices or introducing a facility for the ICPs to bring deficiencies
directly to the attention of the HSE so that appropriate enforcement action can be
implemented.
Further consideration could be given to the effects that the commercial arrangements
between the Duty Holders and the ICPs might have on their independence and
whether an alternative arrangement might be beneficial.
Provide further guidance on the HSEs expectations for the workings of the
Verification process, specifically to attempt to achieve higher levels of consistency
and standardisation in its application but without introducing prescriptive measures
and thereby losing the flexibility associated with the present approach that allows the
Verification scheme to be tailored to the specific needs of each installation.
However, it is recommended that to make significant modifications to the requirements at this
point in time would be inappropriate as it is considered that after a long learning curve the
process is beginning to bed down and show benefit.
8.2 CONCLUSIONS, LESSONS AND RECOMMENDATIONS REGARDING THE
PRESENT RESEARCH
8.2.1 Conclusions
It is concluded that the present research, although not meeting the initial objectives in:
1) Being complete, in time, to make a contribution to the review of the Safety Case Regulations,
and
2) Not providing a more quantitative indication of the cost / benefits associated with the
changed regime,
has, nevertheless, obtained and analysed appropriate data to provide a useful overview of the
requirements for Verification, to highlight perceived difficulties with its present workings and to
make recommendations for consideration by the HSE on opportunities for potential
improvements to the system.
24
8.2.2 Learnings
The following learnings have been identified from the research activity:
It is considered that the key learning to emerge from the work was in respect of the
unrealistic expectations of the willingness and /or ability of the stakeholders to provide
data in the quantity and quality anticipated.
It is important to ensure questions are unambiguous and each question addresses a
single aspect (see the analysis of the response to Question 7 in Section 5, above).
More extensive piloting of questionnaires might identify difficulties in obtaining data
at an earlier stage in the project and thereby allow more time to identify, examine,
select and implement alternative approaches.
It appears that many stakeholders, who might not otherwise complete a survey
questionnaire, are often willing to provide evidence by face to face interview. Such
interviews have the added advantages of the opportunity to clarify questions and
thereby assist the interviewee to provide the information required.
8.2.3 Recommendations
The following recommendations are offered for consideration for implementation in future
related studies:
If it is anticipated that an impact evaluation of a programme is to be undertaken, to
make baseline studies of the existing regime prior to making changes and to pre-
advise the stakeholders of the proposed future study so that there is an higher
possibility of their collecting and maintaining data for future use.
Hold initial discussions with stakeholders to obtain information on the availability of
data and the likelihood of it being made available to the researchers.
Ensure appropriate piloting of questionnaires and allow sufficient programme time for
this activity.
Make early considerations of contingency provisions for alternative methodological
approaches to the research should the initial proposals fail to meet their objectives.
25
9 REFERENCES
AUPEC Ltd. (1999) Evaluation of the Offshore Safety Legislative Regime, Aberdeen
Clarke, A. and Dawson, R. (1999) Evaluation Research Sage Publications Ltd, London
Cullen, The Hon. Lord (1990) The Public Inquiry into the Piper Alpha Disaster. HMSO London
26
APPENDIX A
ANALYSIS OF EVIDENCE
A.1
This Appendix presents the evidence collected, in response to each question asked, and notes
the observations made in respect of the analysis of that evidence.
In the graphical presentations of the data the abscissa axis represents the number of respondents
within a given class or returning a particular response.
Where the ordinate axis is in percentages the respondents were asked to ensure their responses
totalled 100%. In these instances it is considered that detailed distribution of the individual
responses is of less significance than the overall distribution of the data. Where a respondent
was in disagreement with the sense of a given statement negative values arise.
QUESTIONNAIRE SECTION 1: RESPONDENT PROFILES
A total of 27 sources of evidence were surveyed and Figure 1, below, shows the distribution of
these data sources by the role of the organisation providing the response. The balance between
Duty Holders and other participants does not reflect the overall numerical distribution of the
various stakeholders in that the Duty Holders are significantly underrepresented numerically as
a result of the very low returns of the electronic questionnaire received from them.. It is
nevertheless considered that the Duty Holders interviewed were a good representative sample of
the Duty Holders by such characteristics as role, size, and their time in the UK offshore industry
and that the evidence collected is acceptable to allow conclusions to be drawn there-from in
respect of the Duty Holders perspectives of the comparison of the Certification and
Verification regimes etc..
12
10
8
6
4
2
0
Duty Holder - Duty Holder -
mobile permanent
installation(s) installation(s)
ICP Regulator W orkforce
Representing
Body
Figure 1 Number of Respondents by Organisations Role
Figure 2 shows the response distribution to Q 1.6: Do you have experience of Certification as
well as Verification? The vast majority of the respondents did have experience of both
regimes and were therefore well positioned to make comparisons between them. All but one of
A.2
the respondents, having experience with both regimes, had that experience of both regimes with
the same organisation. Examination of the job titles and roles of the respondents indicates that
they were appropriate individuals to provide evidence for the present study. Specific
experiences noted by the respondents as to their experience relative to Certification and
Verification included:
Previous experience of working with the CA regime when employed by a Duty
Holder.
Experience from both an operations perspective as an OIM and in present role as
Verification Engineer.
Active member of the consultative industry groups set up to discuss the SCR and
DCR.
Presently leading a comprehensive overview of Verification for the Regulator.
Background in major hazards and risk assessment.
Employed with a CA / ICP since 1991; caught the 'tail end' of Certification and the
transition to Verification.
Experience both with the Regulator and a Duty Holder.
Now a HSE director for a Duty Holder, previous experience with the Regulator and
another Duty Holder.
Worked as a Duty Holder with CA Surveyors and very familiar with Verification.
Involved since 1975 with Certification and with the subsequent transition to and
implementation of Verification.
Worked for a CA for 12 years and dealt with the Certificate of Fitness regime, now
working for the Regulator.
Previously worked with a contractor under the Certification regime, now works for the
Regulator.
Worked with a contractor under the CA regime.
As a Duty Holder worked closely with a CA under SI 289 and has lived' with
Verification since its conception.
Now a Regulator, worked on facility design projects under the CA regime.
Experience with the requirement for Verification demonstration of initial suitability
of SCEs in respect of old Installations, some now 35 years old.
A.3
This indication of the broad experience of the interviewees, in respect of function and length of
involvement with Certification and Verification, provides assurance that an acceptable range of
views and experience has been elicited from the interviews to enable some generalised
conclusions to be reached as to the outcomes from the Verification regulations and to allow
meaningful comparison with previous Certification regime.
.
30
25
0
5
10
15
20
N o - V e rific a tion only Y es - C e rtific atio n A N D V e rific a tion
Figure 2 Response Distribution for Q 1.6: Do you have experience of Certification as
well as Verification?
QUESTIONNAIRE SECTION 2: ASSESSMENT OF VERIFICATION
Question 2.1 asked: How would you apportion the relative significance of the following key
elements of Verification for assuring offshore safety? Figure 3 to Figure 7, below, show the
percentage allocations given by the respondents, by organisational role, to each of the four
criteria viz:
a) Moving responsibility from the CA to Duty Holders (Figure 3).
b) Requiring risk-based targeting with the concept of SCEs (Figure 4).
c) Requiring independence of the Verification body (Figure 5).
d) Removing the link between the Regulator and the Certification / Verification body
(Figure 6).
A.4
e) Other (Figure 7)
Examination of the data indicates that, in general, the perception of the respondents is that (a)
and (b) above are the most significant contributors to improved safety, followed by (c).
Response (d) was considered a very low contributor with several respondent assessing negative
percentages indicating that the factor reduced rather than increased offshore safety. Negative
values were also assigned to (a) by one Regulator and one TU Officer.
Items assigned to (e) included:
Recognition of actions required on failure of an SCE (25%)
The requirement for a Written Scheme of Examination (WSE) system (no % assigned)
The link between SCEs and Performance Standards (PSs) (no % assigned)
The contractual relationship between the DHs and the ICPs (-22%)
Potential loss of reputation (30%)
8
7
6
5
4
3
2
1
0
)
)
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent faciliity(s
Duty Holder - mobile facility(s
-
1
5

-
1
25
1
5
2
0
2
5
3
0
3
5
5
0
6
0

N
i
l

R
e
s
p
o
n
s
e

%
Figure 3 Response distribution for Criterion (a): Moving responsibility from the CA to
Duty Holders.
A.5
9
8
7
6
5
4
3
2
1
0
i
lder -
- (s )
W o rk fo rc e R epresent ng B ody
R egulator
IC P
D u ty H o per m anent fa c iliity (s )
D u ty H o ld e r m o b ile fa c ility
15 20 22 25 30 40 50 Nil
R e s p ons e
%
Figure 4 Response distribution for Criterion (b): Requiring risk-based targeting with
the concept of SCEs.
7
6
5
4
3
2
1
0
i
l
lder - lii (s)
l - il i )
W o rk fo rce R e p re sen t ng B ody
Re g u a to r
IC P
D u ty H o pe rm an ent faci ty
D u ty H o d e r m o b e fa c lity(s
0
1
0
1
5
2
0
2
2
2
5
3
0
3
5

N
i
l

R
e
s
p
o
n
s
e

%
Figure 5 Response distribution for Criterion (c): Requiring independence of the
Verification body.
A.6
10
9
8
7
6
5
4
3
2
1
0
i
l
lii (s)
l il i )
W orkforce Represent ng Body
Regu ator
ICP
Duty Holder - perm anent faci ty
Duty Ho der - mob e fac lity(s
-
4
0
-
3
5
-
2
5
-
2
2
-
2
0
-
1
5
-
1
00
1
0
2
0
2
5
3
0
4
0
N
i
l

R
e
s
p
o
n
s
e

%
Figure 6 Response distribution for Criterion (d): Removing the link between the
Regulator and the Certification / Verification body.
25
20
15
10
5
0
-22 15 25 30 Nil R esponse
i
l
l lii (s)
l il i )
W orkforce Represent ng B ody
Regu ator
ICP
D uty Ho der - perm anent faci ty
D uty Ho der - m ob e fac lity(s
%
Figure 7 Response distribution for Criteria (e): Other.
A.7
Figure 8, below, shows the response distribution to Q 2.2: How do you view the HSEs
involvement with Verification? Whereas approximately the same number of respondents
viewed the involvement as Not involved enough as viewed it as a satisfactory level of
involvement, it is noteworthy that a strong majority of Duty Holders viewed it as a satisfactory
level of involvement while the majority of HSE personnel and all the ICPs rated it Not
involved enough.
16
14
12
10
8
6
4
2
0
Not involved enough Satisfactory level of involvement Too involved
i
ilii (s)
l ity(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent fac ty
Duty Hoder - mobile facil
Figure 8 Response distribution to Q 2.2: How do you view the HSEs involvement
with Verification?
When asked for the rationale for their response to Q 2.2 the following were considered key
responses:
Too much inspecting hardware, too little analysing Verification findings.
(Regulator)
The HSE should make more effort to ensure more standardisation across operators.
There should be less exposure to the different opinions of Inspectors regarding
Verification. (Duty Holder)
The HSE needs to follow up on comments on the Written Schemes of Examination
(WSEs) and survey findings. HSE findings are not directly communicated to the
ICPs. There should be an open forum for the exchange of information. (ICP)
A.8
Clarify the HSEs needs to Duty Holders in the context of the regulations. Improve
alignment of Duty Holders to the regulations. (Regulator)
As a result of resource constraints and prioritisation, HSE doesn't do enough on the
working of Verification or abstract all the learnings from the Verification process.
(Regulator)
This is improving but there is an ongoing need to ensure the arrangements are
working; particularly, to ensure the significance of failure of an SCE is understood and
dealt with correctly and that ICPs' findings are appropriately reported. (Regulator)
Principal [HSE] contribution is to demonstrate the importance of verification. HSE
should not duplicate ICPs activities but should have better focus. (ICP)
Inspection is key activity. (Duty Holder)
[More] Inspection of the Verification Scheme [is needed]. HSE / ICPs should move
closer. More clarity of the role of the HSE is required. There is a need for more
'meaningful' schemes. (Regulator)
Not more [HSE] repetition of ICPs' work but more emphasis on ensuring Duty
Holder action on ICP recommendations. (ICP)
More direct contact between HSE and ICPs to alleviate contractual pressures on the
ICPs. (Regulator)
The HSE could receive and follow up more information from the ICPs. (Regulator)
HSE involvement in Verification has been minimal, other than the Regulations
themselves, I feel they have not made any significant contribution. The Duty Holders
have largely been left to sort out the process based on initial guidance from the HSE.
(Duty Holder)
HSE should use Verification to allow them to stand back from operations
Inspections but should more closely inspect the Verification arrangements. (Duty
Holder)
There should be more HSE Guidance to the ICPs as to what should be done. There
should be more consistency between Inspectors (Duty Holder)
HSE could do more to share good practice across the industry. HSE comments
sometimes tend towards 'prescription'. There could be more system management.
(Duty Holder)
The HSE has not fully understood the process and it is not working yet. More
Inspection of management systems is required. The HSE should seek more
documentary evidence of the ICPs' confirmation of Duty Holders' activities.
(Regulator)
A.9
[The HSE should] Provide a second level of Verification. (Duty Holder)
Too many 'warnings' / letters to Duty Holders, not enough 'enforcement' [by HSE].
(TU Officer)
[The HSE] Should provide more Guidance and more 'lateral learning' across the
industry. (Duty Holder)
The ICP being employed by the Duty Holder is a major problem. The ICPs have no
power. (Duty Holder)
Emphasis should be on audits / inspections of Duty Holders and ICPs rather than
more general offshore Inspection. (Duty Holder).
Figure 9, below, shows a keyword analysis of the responses to Q 2.3, by organisational role.
The dominance of the responses referring to communication, evidence and inspection reflect the
emphasis put on these topics by the Regulators and ICPs, supplemented by the high emphasis
on Inspection by the Duty Holders.
0
5
10
15
20
25
i
- ( s )
- ( s )
W o r k f o r c e R e pr e s e n t ng B o dy
R e gula t o r
I C P
D u t y H o ld e r p e r m a n e n t f a c iliit y
D u t y H o ld e r m o b ile f a c ilit y
C
o
m
m
u
n
i
c
a
t
i
o
n

C
o
n
s
i
s
t
e
n
c
y

E
v
i
d
e
n
c
e

F
o
c
u
s

G
o
o
d

p
r
a
c
t
i
c
e
I
m
p
o
r
t
a
n
c
e

I
n
s
p
e
c
t
i
o
n

I
n
v
o
l
v
e
m
e
n
t
O
K
P
r
o
c
e
s
s

R
e
g
u
l
a
t
i
o
n
s

R
e
s
o
u
r
c
e
s

R
e
s
p
o
n
s
i
b
i
l
i
t
y

Figure 9 Results of keyword analysis of the responses to Q 2.3: What is HSEs
principal contribution; what more / less should be done; with what benefits / cost (by
organisational role)
Question 2.4 asked: What value do you think Duty Holders (i.e. their senior management)
place on Verification please indicate the one answer nearest to your observations /
experience? Of the 24 responses only 3 respondents thought senior management placed No
value on Verification; a further 13 considered their senior management thought Verification
provided Some value but not good value for money; 3 responded that Verification was
perceived as Valuable and cost effective for ensuring safety; while only 5 respondents stated
that it was perceived as an Highly valued part of safety management.
A.10
Q 2.5 asked What health and safety benefits do you see of Verification over Certification?
The keyword analysis of the responses to this question, by organisational role, is shown in,
Figure 10 below. It indicates that the key safety benefits are seen as:
Transfer of ownership from the CA to the Duty Holder.
Flexibility to tailor the system to the installation.
The risk based approach to identify the SCEs, and
The focus on ensuring that the SCEs are in an appropriate condition and have the
required availability, reliability and functionality as required by their performance
standards.
This prioritisation is generally consistent with the responses given to Q 2.1. but these items are
primarily outcomes of the Safety Case Regulations, whereas Verification is an independent
check on these Duty Holders activities. Some respondents (particularly among the Regulators)
indicated that, although the process was good, its full effectiveness had not been achieved
because of poor understanding and implementation. However, there was also a feeling that the
system was now starting to bed down.
There was noticeably strong alignment between the Regulators, ICPs and Duty Holders as to the
importance of Ownership and the Focus on SCEs
9
8
7
6 W o r k f o r c e R e p r e s e n tin g B o d y
R e g u l a t o r
5
I C P
4
D u t y H o ld e r - p e r m a n e n t f a c i li it y ( s )
3 D u t y H o ld e r - m o b il e f a c ili t y ( s )
2
1
0
C
l
a
r
i
t
y
C
o
m
p
l
i
a
n
c
e

E
d
u
c
a
t
i
o
n

F
l
e
x
i
b
i
l
i
t
y
F
o
c
u
s

o
n

S
C
E
s

O
w
n
e
r
s
h
i
p
P
o
w
e
r

Q
u
a
l
i
t
y

v

Q
u
a
n
t
i
t
y

R
i
s
k

b
a
s
e
d

S
a
m
p
l
i
n
g

W
e
a
k

i
m
p
l
e
m
e
n
t
a
t
i
o
n

Figure 10 Results of keyword analysis of the responses to Q 2.5 What health and
safety benefits do you see of Verification over Certification (by organisational role).
Question 2.6 asked: If the legal requirement for Verification were now removed, which one of
the following scenarios would you envisage? The responses to this question are shown in
Table 1 below. The vast majority of responses indicate that Duty Holders would continue with
verification but might reduce the scope of the activity and might well undertake it using in-
house personnel.
A.11
RESPONSE
NUMBER OF RES
RESPONSE
PONDENTS SELECTING
DHs would voluntarily seek a similar level of 5
independent verification.
DHs would voluntarily seek independent 7
verification but with more limited scope.
DHs would implement more internal 11
verification but not employ independent
verifiers.
DHs would seek an approach more akin to 0
Certification.
DHs would drop verification activity entirely. 2
Table 1 Responses to Question 2.6: If the legal requirement for Verification were now
removed, which one of the following scenarios would you envisage?
Figure 11, below, shows the response distribution for Q 2.7: If, on a scale of 1 to 5, current
practices with Verification deliver a safety level of 3 for SCEsif Verification requirements
were now removed (as in Question 2.6) where would you expect safety of those elements
currently classed as safety-critical to rate in 3 years time given the scenario you envisage?
Whilst a significant majority of respondents thought there would be little change to the safety
level of SCEs, some respondents thought there would be a reduction in safety and, of these,
some anticipated the reduction would be significant. Reasons for future deterioration
included:
The system is struggling now and would only get worse. (TU Officer)
Things are not presently very good so, perhaps, can not get much worse!
(Regulator)
Lack of compulsion. No policing. (Regulator)
The rationales for those expecting little change included:
I feel there is a general feeling that the Verification approach is "good practice".
have evidence that the same general principle is being used in other geographical
locations. (Duty Holder)
Company philosophy and practice of categorising equipment in the same way
(Safety-Critical Elements +Operationally Critical Elements +others). (Duty Holder)
Because of Class. (Duty Holder Floaters)
A.12
I
Only two respondents thought there might be some improvement, the reasons were:
[Because of] Continuous improvement. (Duty Holder)
Because of the HSEs Key Programme on Installation Integrity, otherwise it would
be unchanged. (Regulator)
One Regulator noted, in respect of change, that This would vary with the Duty Holder.
14
12
10
8
6
4
2
0
1 Significantly worse 2 3 Comparable to today 4
i
il (s)
ility(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent fac iity
Duty Holder - mobile fac
Figure 11 Response distribution for Q 2.7: If, on a scale of 1 to 5, current practices
with Verification deliver a safety level of 3 for SCEsif Verification requirements were
now removed (as in Question 2.6) where would you expect the safety of those
elements currently classed as safety-critical to rate in 3 years time given the scenario
you envisage?
Figure 12, below, shows the response distribution for Q 2.8: If on a scale of 1 to 5, current
practices deliver a safety level of 3 for SCEslooking back to around 1996 and the
Certification regime, where would you rate safety for those elements now classed as safety-
critical? Here, again, the majority of respondents thought there had been no significant
change in the level of safety for SCEs under the present regime compared to that provided under
the previous certification regime although a significant number of responses, from the
A.13
Regulators and Duty Holders, indicated that the safety of SCEs was somewhat better under the
new arrangements.
Comments for the responses included, for the previous Certification regime providing higher
safety:
Certification was less constrained by the WSE - effort could be directed to perceived
needs. There was more examination and less review of documentation. (ICP)
Reasons given for the Verification regime providing higher safety included:
The propensity for 'cat and mouse' has gone. Risk management is more focused.
Work is undertaken in excess of Verification e.g. extra PFEER reviews, taking a 'full
system' approach. This has emanated from Verification. (Duty Holder)
There are now much better PM [Planned Maintenance] systems. (ICP)
I feel there is a better understanding of the performance requirements of key systems
and, therefore, an improvement in the overall level of safety. (Duty Holder)
There were no structured processes. (Regulator)
There were no SCEs, no risk based approach. There was a much higher onus on the
CAs. (Regulator)
Improved understanding, by education, that SCEs are important. Making appropriate
measurements to ensure knowledge of SCEs' condition. (Regulator)
One Regulator noted that safety was now higher than prior to Verification Because of the
HSEs Key Programme on Hydrocarbon Releases.
Reasons for little change were ascribed, inter alia, to:
More focus and better systems has been offset by less sanction to drive
improvement. (ICP)
Less reliance on statistics and direct sanction by the CAs has been balanced by the
focus on SCEs. (Regulator)
Because Safety Cases were [already] in place. (Duty Holder)
Change in real safety is probably small as CA Surveyors were then very experienced
- this is not so true now! (Duty Holder)
More effort is needed to ensure Performance Standards are suitable. (Regulator)
SCE's are maintained to the performance standard, these in general were [already] in
place as most equipment was manufactured well before the introduction of
Verification. (Duty Holder)
A.14
Improvements to the system have been offset by reduced spending on maintenance.
(TU Officer)
14
12
10
8
6
4
2
0
1 Significantly 2 3 Com parable to 4 5 Significantly
worse today better
i
l
l lii (s)
l il i )
W orkforce Represent ng Body
Regu ator
ICP
Duty Ho der - perm anent faci ty
D uty Ho der - m ob e fac lity(s
Figure 12 Response distribution for Q 2.8: If on a scale of 1 to 5, current practices
deliver a safety level of 3 for SCEslooking back to around 1996 and the Certification
regime, where would you rate safety for those elements now classed as safety-
critical?
Question 2.9 asked Why do you think there is a general preference to appoint ICPs who were
formally CAs?
A keyword analysis of responses was undertaken and the results are presented in Figure 13,
below. The analysis shows the number of responses for the keyword by organisation. The
dominant reason was the perceived competence and experience of the CAs, strongly supported
by comfort and convenience.
A.15
2 0
1 8
1 6
1 4
1 2
1 0
8
6
4
2
0
W i n g
l
l d e r - i l i i ( s )
l d e r - l e i l i ( s )
o r k f o r c e R e p r e s e n t B o d y
R e g u a t o r
I C P
D u t y H o p e r m a n e n t f a c t y
D u t y H o m o b i f a c t y
A
c
c
e
p
t
a
b
i
l
it
y

t
o

H
S
E

A
v
a
i
la
b
il
it
y
C
o
m
f
o
r
t

C
o
m
p
e
t
e
n
c
e

\

e
x
p
e
r
i
e
n
c
e
C
o
n
v
e
n
i
e
n
c
e

E
f
f
ic
i
e
n
c
y

w
it
h

C
la
s
s

R
e
s
o
u
r
c
e
s
S
t
a
b
il
i
t
y

Figure 13 Results of keyword analysis of the responses to Q 2.9: Why do you think
there is a general preference for DHs to appoint external ICPs? (by organisational
role)
Figure 14, below shows the response distribution to Q 2.10: If you are a Duty Holder, have you
considered appointing an ICP who was not previously an established CA? The responses
indicated that 30% (4) of the respondents had considered the appointment of an ICP who was
not previously a CA. Reasons given for not considering organisations other than the CAs were:
CA availability
No alternatives
HSE influence
CA competence
CAs were able to deliver the required levels of service and had staff who were
considered to be competent in their area.
Lack of Duty Holder resources
Reasons given for considering non CAs were their specific skills and expertise, not necessarily
available from the CAs, needed for specialised equipment and items such as helidecks.
A.16
8
7
6
5
D u t y H o l d e r - p e r m a n e n t f a c i l i i t y ( s )
4
D u t y H o l d e r - m o b i l e f a c i l i t y ( s )
3
2
1
0
N o Y e s
Figure 14 Response distribution to Q 2.10: If you are a Duty Holder, have you
considered appointing an ICP who was not previously an established CA?
Figure 15, below, shows the response distribution for Q 2.11: If you are a Duty Holder and a
non CA ICP was considered, did you proceed with such an appointment. Only two Duty
Holders, out of the 4 Duty Holders respondents who have considered the appointment of an ICP
who was not previously a CA, have proceeded with such an appointment and did so because
their proposed, non CA, ICP had specialised skills. Of the organisations not proceeding with
the appointment of a non CA ICP, one gave their reason as contractual and the other that it
was too onerous to assure independence.
3
2
D u ty H older - pe rm an en t fa ciliity(s)
D u ty H old e r - m o bile fa cility(s)
1
0
N o P le a s e s ele c t..... Y e s
Figure 15 Response Distribution for Q 2.11: If you are a Duty Holder and a non CA
ICP was considered, did you proceed with such an appointment?
A.17
Figure 16, below, shows the keyword analysis of the responses to Q2.12: Why do you think
there was a general preference for DHs to appoint external ICPs? By far the most quoted
reason was that there was demonstrable independence by appointing an external ICP.
Knowing that appointing an ICP, who was previously a CA, was most likely to be acceptable to
the HSE was also a significant consideration.
18
16
14
12
10
8
6
4
2
0
ICP
il )
l il lity(s)
Regulator
Duty Holder - permanent fac iity(s
Duty Hoder - mob e faci
Demonstrable HSE influence Inadequate in-house outsourced costs Risk of 2nd Party
independence resources route failure
Figure 16 Response distribution to Q 2.12: Why do you think there was a general
preference for DHs to appoint external ICPs?
Figure 17, below, shows the response distribution for Q 2.13: If you are a Duty Holder, has 2
nd
Party Verification (i.e. independent in-house Verification) been considered? Only 3 Duty
Holders, out of the 10 responding, have considered 2
nd
Party Verification; reasons given for not
considering 2
nd
Party Verification included:
Insufficient resources
Comfort using a 3
rd
Party
Poor industry experience with 2
nd
Party approach.
Difficulty demonstrating independence
A.18
8
7
6
5
D u ty H older - pe rm an ent faciliity(s)
4
D u ty H old e r - m o bile fa cility (s)
3
2
1
0
No Y e s
Figure 17 Response distribution for Q 2.13: If you are a Duty Holder, has 2
nd
Party
Verification (i.e. independent in-house Verification) been considered?
Figure 18, below, shows that of the 3 Duty Holders who considered 2
nd
Party Verification only
2 proceeded with this approach. Reasons given for not progressing 2
nd
Party Verification
included lack of resources, difficulties demonstrating independence (particularly from
Operations) and having more comfort with a 3
rd
Party approach.
3
2
D u ty H older - pe rm an en t fa ciliity(s)
D u ty H old e r - m o bile f a cility(s)
1
0
No Y e s
Figure 18 Response distribution for Q 2.14: If you are a Duty Holder and 2
nd
party
Verification was considered, was this implemented?
A.19
Figure 19, below, shows the response distribution to Q 2.15: If you are a Duty Holder, do you
have more than one Verification body covering the complete range of work? Five, of the 11
Duty Holders questioned used two ICPs and two Duty Holders used three ICPs. The rationale
for use of more than one ICP was generally to support specialist areas such as structures,
helidecks, drilling equipment and wells. One Duty Holder used a separate ICP for PFEER
related SCEs and one Duty Holder reported that using two ICPs was historical and resulted from
company mergers where the original ICPs were retained for continuity reasons.
6
5
4
D u t y H o ld e r - p e r m a n e n t f a c i li it y(s )
3
D u t y H o ld e r - m o b i le f a c i li t y(s )
2
1
0
2 3
Figure 19 Response distribution of positive responses to Q 2.15: If you are a Duty
Holder, do you have more than one Verification body covering the complete range of
work? (showing the number of bodies used).
Figure 20 to Figure 23, below, show the response distributions received to questions addressing
respondents views on the legal and commercial relationships between the HSE, the Duty
Holders and the ICPs viz:
Q 2.16: Are you satisfied with the legal relationship between the DHs and the HSE in
the context of Verification?
Q 2.17: Are you satisfied with the legal relationship between the DH and ICPs?
Q 2.18: Is the Duty Holder / ICP commercial relationship conducive to rigorous
independent scrutiny?
Q 2.19 Are you satisfied with the legal relationship between the ICPs and HSE?
The responses to Q2.16 indicate a strong general satisfaction with legal relationship between the
Duty Holders and the HSE. However, it should be noted that, whereas all the Regulators and
most Duty Holders responding were satisfied, this was true of only one ICP with the other ICPs
being either unsure or dissatisfied with the arrangement as also was one TU Officer. Sources of
dissatisfaction stated included:
A.20
Less satisfied than under the CA regime. (ICP)
The duties are very spread / diluted. There is a lack of sanction. (ICP)
[ICPs] powers may be insufficient. (Regulator)
Too remote. Only requirement is for a scheme. The ICPs have been driven down the
value chain with loss of respect. (ICP)
The HSE are not enforcing on this issue. (TU Officer)
No direct contact with the HSE in respect of Verification. Relationship should be
better. (Duty Holder)
There was also a high level of satisfaction regarding the legal relationship between the Duty
Holders and the ICPs although, in this instance a considerable number of Regulators expressed
dissatisfaction with the relationship. Reasons for dissatisfaction included:
The ICPs have no power to ensure their comments are 'closed out'. (ICP)
The relationship is too comfortable. More ICP independence is needed. (TU
Officer)
The contractor relationship puts the ICPs in a weak position. (Regulator)
There is a question of the understanding between the ICPs and the Duty Holders.
The legal relationship could be better. (Regulator)
Duty Holders are not capable, or positioned, to ensure the ICPs are sufficiently
rigorous and critical. (Regulator)
The Duty Holders want to squeeze costs and reduce the ICPs' scope of work. (ICP)
Because of the 'charging' regime, the ICPs are under pressure. (TU Officer)
In contrast to the responses to Q 2.16 and Q 2.17, the responses to Q2.18 indicate strong
dissatisfaction with the commercial relationship between the Duty Holders and the ICPs; all of
the Regulator and all but one of the ICP responses expressed dissatisfaction. However, all but
one of the Duty Holders expressed satisfaction with this relationship. Comments noted
included:
It ensures a level of pragmatism; otherwise the ICP could 'dictate' their scope. The
commercial relationship moderates this. (Duty Holder)
As professional bodies [ICPs] economics is not the [their] priority. (Duty Holder)
Changing ICP is more frequent [than previously] with the loss of the ICP's corporate
knowledge. (Regulator)
A.21
The ICPs suffer from commercial pressure but it is difficult to find an acceptable
alternative. (Regulator)
It doesn't adversely affect independence. (ICP)
[independence] Ensured by independent audit. (Duty Holder)
How can you be independent when the Duty Holder pays? (Regulator)
Because we respect the judgement of the organisation and its employees. (Duty
Holder)
I feel that in my experience the commercial issues have no impact on the service
provided by the ICP. (Duty Holder)
But it depends on how it is managed - there is a risk to independence. (Duty Holder)
Yes - but it needs to be carefully managed. (Duty Holder)
Too heavily influenced by liabilities. The use of 'Dutch Auctions' is highly
questionable. There is a 'power play' between the commercial and technical
perspectives. (ICP)
It hasn't been an issue. The surveyors act independently. (Duty Holder)
It is cost driven, Dutch Auctions etc. (Regulator)
Duty Holders need to take value from Verification. (Duty Holder)
Internal audit showed no concerns. (Duty Holder)
With respect to Q 2.19, here again there was general satisfaction with the legal relationship
between the HSE and the ICPs; however, dissatisfaction was expressed by some Regulators and
some ICPs. Examples of comments received are given below:
There are no specific ICP duties. (Regulator)
There is no direct legal relationship but there could be more communication and
liaison. (ICP)
There should be a clearer legal duty. Presently only the HSWA Section III.
Compare with LOLER where there is a duty to copy reports to the enforcing
authority. (Regulator)
There should be a duty on the ICPs to report to the HSE (possibly significant failures
of SCEs and an Annual Report). (Regulator)
A more direct relationship would be beneficial. (ICP)
A.22
It is not the legal relationship that causes the problems but lack of communication
and interfacing. The legal standing of the ICPs is questionable. (Duty Holder)
It might be better if the HSE nominated the ICPs. (ICP)
Some HSE personnel sometimes seem to have a bad attitude towards certain ICPs.
(Duty Holder)
There should be duties on the ICPs to report findings to the HSE. (Regulator)
Little interaction. Contact has to be through the Duty Holder. (Regulator)
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
i
ICP
i (s)
i (s)
Workforce Represent ng Body
Regulator
Duty Holder - permanent fac liity
Duty Holder - moble facility
No, dis-satisfied Unsure Yes, satisfied
Figure 20 Response distribution for Q 2.16: Are you satisfied with the legal
relationship between the DHs and the HSE in the context of Verification?
A.23
16
15
14
13
12
11
10
Workforce Representing Body
9
Regulator
8 ICP
Duty Holder - permanent faciliity(s)
7
Duty Holder - mobile facility(s)
6
5
4
3
2
1
0
No, dis-satisfied Not in a position to say Unsure Yes, satisfied
Figure 21 Response distribution for Q 2.17: Are you satisfied with the legal
relationship between the DH and ICPs?
17
16
15
14
13
12
11
10
Workforce Representing Body
9
Regulator
ICP
8
Duty Holder - permanent faciliity(s)
Duty Holder - mobile facility(s)
7
6
5
4
3
2
1
0
No Unsure Yes
Figure 22 Response distribution for Q 2.18: Is the DH / ICP commercial relationship
conducive to rigorous independent scrutiny?
A.24
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
i
l lii (s)
l (s)
Workforce Representng Body
Regulator
ICP
Duty Hoder - permanent faci ty
Duty Hoder - mobile facility
No, dis-satisfied Not in a position to say Unsure Yes, satisfied
Figure 23 Response distribution for Q 2.19: Are you satisfied with the legal
relationship between the ICPs and HSE?
A.25
Table 2, below, summarises the number of responses given by category (Inadequate,
Acceptable and Thorough) to Q 2.20: Overall within the industry, how well is the concept
of SCEs and the importance of their suitability / condition in preventing / controlling major
accidents understood by personnel within the following groups? Although the respondents
were opined that there was generally an acceptable level of understanding for each group
(presumed to be relative to their needs) it is concerning that such strong perceptions of
inadequate understanding were reported for the Operational Workforce, Teams involved in
Modification Work, and Inspection / Maintenance Teams with this last being considered of
particular import.
GROUP INADEQUATE ACCEPTABLE THOROUGH
Corporate Management 7 14 4
Offshore Installation
Managers
0 14 11
Operational Workforce 10 13 1
Teams involved in
Modification Work
11 12 2
Inspection / Maintenance
Teams
11 12 3
Table 2 Distribution of Responses to Q 2.20: Overall within the industry, how well is
the concept of SCEs and the importance of their suitability / condition in preventing /
controlling major accidents understood by personnel within the following groups?
Q 2.21 asked the respondents to: Please identify examples of risk control benefits / safety
improvements brought about by Verification during operations Respondents were also asked
whether the item would have been identified under Certification and the potential consequences
had it not been identified. The results of a keyword analysis to this question are shown in
Figure 24 below.
The consensus was that the main benefits of Verification were the risk based approach allowing
a rigorous process for identification of hazards, SIL level and SCEs. This, in turn, led to
improved planned maintenance systems with focus of resources on the maintenance and testing
of the SCEs.
As to whether specific items would have been identified under Certification the answers were
generally either No or Possibly in approximately equal proportions. The consequences of
their not being identified included:
Low reliability
Big fire
A.26
i
Catastrophic
Loss of life or property
Severe
High
Significant
More incidents, and
Critical This respondent also stated that it is important to note that many other
aspects have changed as well as the change from Certification to Verification.
6
5
4
3
2
1
0
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent faciliity(s)
Duty Holder - mobile facility(s)
t
y e

g

s

E
s

g
m
s

g i
c
i
p
w
n

p
t

n
c
t

n

S
s

t
s

t
s
n
t

e
s

n
L
s

e

c
y
g
s

e
a g n n
e
s
s
n
h
a
s n
i
n
i
n o o
i
o
n
c
n
c
a
l
l
e
e e
b
l
u
t
S
I
n
n
t t
i
t t
i
C
g
P
i
i
l
o o i
i
l
e
s r
s
c
m
p
c
m
e
c
n
c
m w
d
p
o
c e e
e e e e
d
i
n
a a
e
e
k
n
s
t
n
S m
n o a
r t e
l
c
i
y
n e
n
o
f
c
t
i n
l
a o
s
t r
a
l
e s
t
o
t
p
i
n
e
n
i
m
s
s
e
s
a
e
i
n
t
g r
e
p f
i
d
p
R
c e e d
i
p t
n
s
w n
t
n
a
e
e
f
f
s
o
e
R
p
r
a
a
v
e
d
n
g o
a
i
p n c
n
e
&
B h
n e
l
M o
r
r
e
o
i
o c
a
i
a o
i
s
m
e o
o
v
n
c
m
m
a T t s e r
i
u
a
d
P
s
c a
k
A n
t
N
o
t
i
n
c P k
k
m
n
c
e
a
v
d
e o
c
n
h
o
r
p
r
r
c a
c a
t
T o
e
r
C i
c o
d
o
F o
I
i
r
m n
t
f a
c
i f
p
r s e
s
s
u
I
m
i
y
m
p i E f i
r
p t n s i
i
k
f
n
F n
s
o
v e r
m
d
R m r
y I
m
r
i
e
e
e g
r E
f
f
m o
t
R
d
e
c
t
n
t
a u
n
a l
i
s p
n
c
I
d
e
V
b e
p T
i
e
d
e
p
e
e
w t
e e
a
i
D
I
m u
i
e
.
o
q o
v
o
v a
s
o
d
m g
r
I
e
i
i
r
i
r r
P
r l
o
r
f
e
r
F p
r
p
x
E n u
R
e
d
M m
o
m
e
m
o
c
s
s
i
d
E
o
v
E
I I
v
F
o
p
r
u r
e
s
c p
m
o
v
i m
d
d
I
p
r I
m e
I
o
v
p
r
m
I
Figure 24 Keyword analysis of the responses to Q 2.21: Please identify examples of
risk control benefits / safety improvements brought about by Verification during
operations?
Q 2.22 asked: What proportion of ICP recommendations from the Verification process are
closed out in a timeframe you consider to be Satisfactory, Slow but low risk areas (i.e.
tolerable), or Slow high risk issues (i.e. unacceptable)? Figure 25 to Figure 27, below,
show the percentages assigned to the three options categorised by the respondents
organisational role.
A.27
0
05
5
1
9
1
0
1
5
2
0
2
5
2
5
3
0
3
0
4
0
3
5
4
5
5
0
4
0
5
0
6
0
6
0
7
0
6
5
8
0
7
5
9
0

N
i
l

R
e
s
p
o
m
n
s
e

8
0
9
0
1
0
0
N
i
l

R
e
s
p
o
n
s
e

i Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity(s)
Duty Holder - mobile facility(s)
Figure 25 Response distribution to Q 2.22 showing the percentage assigned to the
option Satisfactory.
i
ICP
ilii (s)
i (s)
Workforce Representng Body
Regulator
Duty Holder - permanent fac ty
Duty Holder - moble facility
6
5
4
3
2
1
0
10
9
8
7
6
5
4
3
2
1
0
Figure 26 Response distribution to Q 2.22 showing the percentage assigned to the
option Slow but low risk (i.e. tolerable)
A.28
B
u
r
e
a
u
c
r
a
c
y

C
e
r
t
i
f
i
c
a
t
i
o
n

t
y
p
e

t
h
i
n
k
i
n
g

C
o
m
m
u
n
i
c
a
t
i
o
n
01
C
o
m
p
l
e
x
i
t
y
5
D
H

a
t
t
i
t
u
d
e
D
H

c
o
m
m
o
n

f
a
i
l
u
r
e

1
0
m
o
d
e
I
n
d
i
v
i
d
u
a
l
s

I
n
t
e
r
p
r
e
t
a
t
i
o
n

o
f

D
C
R

2
0
3
0
M
a
n
a
g
e
m
e
n
t
s
y
s
t
e
m
s

N
o

I
C
P

e
n
f
o
r
c
e
m
e
n
t

4
5
P
o
o
r

u
n
d
e
r
s
t
a
n
d
i
n
g

P
r
i
o
r
i
t
i
s
a
t
i
o
n

1
0
0
N
i
l

R
e
s
p
o
n
s
e

S
e
r
v
i
c
e

p
r
o
v
i
d
e
r

a
t
t
i
t
u
d
e

R
e
s
o
u
r
c
e
s

ilii (s)
ili (s)
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent fac ty
Duty Holder - mobile fac ty
Figure 27 Response distribution to Q 2.22 showing the percentage assigned to the
option Slow high risk issues (Unacceptable)
The distribution of responses to Q 2.23: What obstacles do you see (if any) in the effective
close out of issues raised through the Verification process? by keyword analysis (by
organisational role) is shown in Figure 28, below.
The responses to this question identified the principal obstacle to the close out of issues as
inadequate resources, poor prioritisation; Duty Holder attitude; poor understanding (of the
potential risk from the findings); and poor management systems were also considered
significant factors.
1 4
1 2
1 0
8
6
4
2
0
W in g
l
l d e r - il i i ( s )
l - i le il i ty ( s )
o r k f o r c e R e p r e s e n t B o d y
R e g u a t o r
I C P
D u t y H o p e r m a n e n t f a c t y
D u ty H o d e r m o b f a c
9
8
7
6
5
4
3
2
1
0
Figure 28 Results of keyword analysis of the responses to question Q 2.23 (by
organisational role).
A.29
Question 2.24 asked: Please list precursors / indicators of major accident potential which have
changed since 1998 and indicate the extent to which Verification has contributed to the change.
Respondents had difficulties in identifying specific precursors and found assigning the
contribution to change resulting from Verification generally impossible because of the
confounding factors. Changes identified included:
Reduced hydrocarbon releases 10 counts
None known / No change 8 counts
Reduced ignitions 2 counts
Safer control systems 2 counts
Reduced fires and explosions; emphasis on risk based approach; better ExE equipment;
improved Performance Standards (PSs); and improved maintenance were each mentioned by
single respondents.
There were three changes mentioned specific to drilling operations viz:
Reduced dropped objects
Reduced well kicks, and
Reduced structural failures
Only two respondents gave a quantitative assessment of the Quantitative change in risk.
The first assigned a reduction of 50% in the risk from hydrocarbon releases and attributed 5% of
this 50% reduction to the contribution from Verification. The second assigned a 10% reduction
in the risk from ignitions but considered Verification had made no contribution to this reduction.
Other estimates of the contribution of Verification to risk reduction were:
Reduced hydrocarbon releases 7 counts: 3 @ 0% (2 Regulator, 1 ICP); 1 @ 5%
(Regulator); 1 @ 10% (ICP), 1 @ 20% (DH), & 1 @ 50% (DH)
Reduced ignition 1 count @10% (ICP)
Reduced fire and explosion 1 count @ 0% (Regulator)
Reduced fatalities 1 @ 0% (Regulator)
Safety-critical systems 1 @ 100% (DH)
Improved ExE equipment 1 @ 0% (Regulator)
Reduced dropped objects 1@ 5% (Regulator)
A.30
Reduced structural failures 1 @ 0 % (ICP)
Other factors influencing reduction in risk since Verification (which can be seen as confounding
factors) were:
Regulator influence / KP1 8 counts
Focus on inspection, testing and maintenance 4 counts
The goal-setting regime; demanning; Step Change; and derrick inspections were each cited
once.
Q 2.25 asked: What do you consider are the relative contributions of Verification /
Examination schemes to major accident prevention offshore? (i.e. SCR Verification scheme,
PFEER Examination scheme and DCR Wells Examination scheme). Again, respondents found
it difficult to make such assignments. It was noted that a considerable part of the PFEER
regulations are not specifically concerned with the prevention of Major Accidents whereas
other parts of SCR (other than the Verification requirement) do make significant contributions.
The histograms indicate that the respondents attributed the changes to risk since 1998 at
approximately 40% due to SCR / Verification; 40% due to PFEER; and 20% due to DCR.
8
7
6
5
4
3
2
1
0
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
5
2
0
3
0
3
3
3
4
4
0
5
0
6
0
7
0
9
0
N
i
l

R
e
s
p
o
n
s
e

Figure 29 Respondents estimates (%) of the change to risk attributable to SCR /
Verification.
A.31
10
9
8
7
6
5
4
3
2
1
0
0

5
5
1
0
1
0
2
0
2
0
2
5
2
5
3
0
3
0
3
3
3
3

4
0
3
5
4
0
5
0

9
0
N
i
l

R
e
s
p
o
n
s
e

6
0
N
i
l

R
e
s
p
o
n
s
e

il )
l il i (s)
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent fac iity(s
Duty Hoder - mob e fac lity
Figure 30 Respondents estimates (%) of the change to risk attributable to PFEER.
i
ilii )
il (s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent fac ty(s
Duty Holder - mob e facility
7
6
5
4
3
2
1
0
Figure 31 Respondents estimates (%) of the change to the DCR Wells Examination
Scheme.
A.32
QUESTIONNAIRE SECTION 3: IMPACT OF THE WIDER POST-CULLEN REGIME
Figure 31 to Figure 34 show the degree of the respondents agreement with the following
statements taken from the AUPEC report that evaluated the post-Cullen regime in 1998 and thus
pre-dating Verification:
A key benefit was the risk assessment process, because of the increased focus and
awareness of higher risk categories, prioritisation of mitigation measures,
identification of cost savings and allocation of funds to those areas most in need.
There was generally agreement and strong agreement with this statement; however the two
TU Officers expressed disagreement and strong disagreement as a result of their perception
of a general reduction in maintenance activities.
Building on structured decision making processes, led to the targeting of risk
reduction efforts.
Many respondents felt neutral about this statement, while others either expressed either
agreement or disagreement in almost equal proportions. While most Duty Holders either
expressed neutrality or agreement, disagreement was expressed by some Regulators and the TU
Officers.
An early benefit of DCR (e.g. in relation to well ops) was from identification of
SCEs and development of performance standards.
Here, although there was generally either agreement or strong agreement with the statement,
the TU Officers expressed neutrality and disagreement. An ICP and one Regulator also
expressed disagreement.
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
i
l ilii )
(s)
Workforce Represent ng Body
Regulator
ICP
Duty Hoder - permanent fac ty(s
Duty Holder - mobile facility
StrongDisagree Disagree Neutral Agree StrongAgree
Figure 32 Response Distribution for Q 3.1: Degree of agreement with AUPEC
statement that a key benefit was the risk assessment process, because of the
A.33
increased focus and awareness of higher risk categories, prioritisation of mitigation
measures, identification of cost savings and allocation of funds to those areas most in
need.
10
9
8
7
6
5
4
3
2
1
0
i
l
(s)
(s)
Workforce Represent ng Body
Reguator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
StrongDisagree Disagree Neutral Agree StrongAgree
Figure 33 Response Distribution for Q 3.2: Degree of agreement with AUPEC
statement that Building on structured decision making processes, led to the targeting
of risk reduction efforts.
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
i
(s)
(s)
Workforce Represent ng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
Disagree Neutral Agree Strong Agree
Figure 34 Response Distribution for Q 3.3: Degree of agreement with AUPEC
statement that an early benefit of DCR (e.g. in relation to well ops?) was from
identification of SCEs and development of performance standards.
A.34
QUESTIONNAIRE SECTION 4: SPECIFIC QUESTIONS
Section 4 comprised a number of statements and respondents were asked to assess the extent of
their agreement or disagreement with the statements using a 5 point Likert scale. The following
information can be drawn from analysis of the response distributions, for Question 4.1 through
to Question 4.18 (by organisational role) that are shown in Figure 35 to Figure 52.
Question 4.1: The benefits of Verification on risk and safety are consistent with those
experienced from the original SCR.
Most respondents were either neutral or in agreement with this statement with all the
Duty Holders responses falling in these categories; however several Regulators
expressed disagreement.
10
9
8
7
6
5
4
3
2
1
0
i
Regul
(s)
(s)
Workforce Representng Body
ator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 35 Response distribution for Q 4.1: The benefits of Verification on risk and
safety are consistent with those experienced from the original SCR
Question 4.2: Beyond the initial implementation of Verification, inspection programmes have
been the primary means of assuring the SCEs meet their performance standards.
There was a significant spread of responses to this question but the majority of Duty
Holders and ICPs expressed various levels of agreement. Responses from Regulators
ranged from Strongly Agree through to Strongly Disagree with six of the ten
Regulator responses indicating neutrality or disagreement.
A.35
10
9
8
7
Workforce Representing Body
6
Regulator
ICP
Duty Holder - permanent faciliity(s)
5
4
Duty Holder - mobile facility(s)
3
2
1
0
Figure 36 Response distribution for Q 4.2: Beyond the initial implementation of
Verification, inspection programmes have been the primary means of assuring the
SCEs meet their performance standards.
Question 4.3: The scope and breadth of the CAs approval / inspection process under
Certification diluted the attention paid to elements now classed as SCEs.
There was a broad spread of responses to this question from the Duty Holders with
several, including two replying in respect of floating installations, expressing
disagreement. Although the majority of the Regulators agreed with the statement, two
strongly disagreed as did one ICP.
ICP
ilii )
il ity(s)
Workforce Representing Body
Regulator
Duty Holder - permanent fac ty(s
Duty Holder - mob e facil
Figure 37 Response distribution for Q 4.3: The scope and breadth of the CAs
approval / inspection process under Certification diluted the attention paid to elements
now classed as SCEs.
A.36
10
9
8
7
6
5
4
3
2
1
0
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

D
i
s
a
g
r
e
e
N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l
R
e
s
p
o
n
s
e

Question 4.4: Certification provided a safety check in relation to major accident precursors
which may not be identified by the Verification process.
There was general disagreement with this statement with five respondents expressing
Strong Disagreement; however two ICPs agreed with the statement.
12
10
8
6
4
2
0
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 38 Response distribution for Q 4.4: Certification provided a safety check in
relation to major accident precursors which may not be identified by the Verification
process.
Question 4.5: The formal identification of SCEs and the Verification process have achieved the
intended regulatory purpose of allowing safety management effort / resources to be
appropriately focused on those items with the greatest contribution to safety of the installation.
There was generally agreement or strong agreement with this statement particularly
amongst the Regulators and the Duty Holders but disagreement is evident in the
responses of some Regulators, ICPs and a TU Officer.
A.37
14
12
10
8
6
4
2
0
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent faciliity(s)
Duty Holder - mobile facility(s)
e

e

e
e

e
e e e
e
p
o
n
s
g
r
a
g
r
A
g
r
A
g
r
s
a
D
i
s
g
e
s
D
i
n
o

R g
t
r
o
n
N
i
l
S
S
t
r
Figure 39 Response Distribution for Q 4.5: The formal identification of SCEs and the
Verification process have achieved the intended regulatory purpose of allowing safety
management effort / resources to be appropriately focused on those items with the
greatest contribution to safety of the installation.
Q 4.6: For fixed installations, the schedule of SCEs originally identified with the introduction
of Verification has not changed significantly over the subsequent 6-8 years.
There was a strong indication of agreement with this statement by most organisational
roles but with some disagreement from two Duty Holders and two Regulators
12
10
8
6
4
2
0
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent faciliity(s)
Duty Holder - mobile facility(s)
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l

A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 40 Response Distribution for Q 4.6: For fixed installations, the schedule of
SCEs originally identified with the introduction of Verification has not changed
significantly over the subsequent 6-8 years.
A.38
Q 4.7: Verification has been beneficial in highlighting the importance of particular safety
elements to those involved in their design, operation, maintenance, verification, modification
and future decommissioning.
Here, again, there was strong agreement with the statement but with an element of
disagreement from all organisational roles.
12
10
8
6
4
2
0
f
l
l ilii )
il ity(s)
Work orce Representing Body
Reguator
ICP
Duty Hoder - permanent fac ty(s
Duty Holder - mob e facil
Figure 41 Response distribution for Q 4.7: Verification has been beneficial in
highlighting the importance of particular safety elements to those involved in their
design, operation, maintenance, verification, modification and future decommissioning.
Q 4.8: Freedom for the DH in appointing ICPs has improved the expertise DHs draw on.
The great majority of respondents expressed either Neutrality or Disagreement
with this statement no doubt reflecting the continuing use of the CAs as ICPs by most
Duty Holders in most instances.
12
10
8
6
4
2
0
l
ICP
ilii (s)
l i (s)
Workforce Representing Body
Reguator
Duty Holder - permanent fac ty
Duty Hoder - moble facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e
D
i
s
a
g
r
e
e
D
i
s
a
g
r
e
e
N
e
u
t
r
a
l

N
e
u
t
r
a
l
A
g
r
e
e

A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

N
i
l

R
e
s
p
o
n
s
e

R
e
s
p
o
n
s
e

Figure 42 Response distribution for Q 4.8: Freedom for the DH in appointing ICPs
has improved the expertise DHs draw on.
A.39
Q 4.9: The free market in procuring ICP services has increased the value for money / cost
effectiveness of the services.
There was significant Neutrality expressed about this statement by both Regulators
and Duty Holders that probably reflects the lack of meaningful cost data on both the
previous and present regime. Other factors may include the difficulties in assessing
the benefits of the systems (where it is virtually impossible to know if the fortunate
dearth of Major Accidents is a result of Verification or attributable to other causes and
also whether the number of Major Accidents would have been the same had the
previous Certification continued). This is discussed further in Section 11.
10
9
8
7
6
5
4
3
2
1
0
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 43 Response distribution: for Q 4.9: The free market in procuring ICP services
has increased the value for money / cost effectiveness of the services.
Q 4.10: The effectiveness of Verification in relation to major accident prevention could be
improved.
There was nearly universal Strong agreement or Agreement with this statement
with disagreement being stated by only two Duty Holders. The rationale behind these
responses is addressed in the responses to Questions 6.1 and 6.2, which are discussed
in Section 5 of the report.
A.40
0
2
4
6
8
10
12
14
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
Figure 44 Response Distribution for Q 4.10: The effectiveness of Verification in
relation to major accident prevention could be improved
Q 4.11: By placing responsibility on DHs away from CAs, Verification has improved
standards of installation safety.
Here there is a marked contrast in the responses of most Duty Holders, who expressed
agreement and most Regulators and ICPs who either express neutrality or
disagreement. A preference for the Certification regime by the previous CAs is
perhaps not unexpected but the disagreement by the Regulators is harder to rationalise.
12
10
8
6
4
2
0
i
ilii )
ility(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent fac ty(s
Duty Holder - mobile fac
D
i
s
a
g
r
e
e
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l

N
e
u
t
r
a
l
A
g
r
e
e

A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

N
i
l
R
e
s
p
o
n
s
e

Figure 45 Response Distribution for Q 4.11: By placing responsibility on DHs away
from CAs, Verification has improved standards of installation safety.
A.41
Q 4.12: ICPs have been over cautious in the definition of SCEs and controls because safety-
criticality is open to interpretation.
There was a significant expression of agreement from some Duty Holders, Regulators
and a TU Officer. The question is now recognised to be ambiguous as the words over
cautious could be taken to mean not defining an element as an SCE when its safety
criticality is marginal or the direct opposite of this approach.
9
8
7
6
5
4
3
2
1
0
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
c
e

Figure 46 Response Distribution for Q 4.12: ICPs have been over cautious in the
definition of SCEs and controls because safety-criticality is open to interpretation.
Q 4.13: Under Verification, inadequate attention is now placed on assessing the suitability and
condition of elements of installations which are not safety-critical.
The responses to this statement were overall weighted to Disagreement but the
responses of the Regulators, ICPs and a TU Officer were weighted to Agreement.
A.42
0
2
4
6
8
10
12
i
Regul
(s)
(s)
Workforce Representng Body
ator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
p
o
n
s
e

Figure 47 Response Distribution for Q 4.13: Under Verification, inadequate attention
is now placed on assessing the suitability and condition of elements of installations
which are not safety-critical.
Q 4.14: The lack of prescription under Verification has enabled cost-saving reductions in the
scope and \ or frequency of physical inspections to be implemented by the industry which may
be open to debate from a rigorous risk-based perspective.
There was a high degree of disagreement with this statement by the Duty Holders and
some Regulators but a significant number of Regulators and the ICPs tended to
indicate Agreement.
A.43
l
i
il i (s)
Workforce Representing Body
Reguator
ICP
Duty Holder - permanent fac liity(s)
Duty Holder - mob e facil ty
Figure 48 Response Distribution for Q 4.14: The lack of prescription under
Verification has enabled cost-saving reductions in the scope and \ or frequency of
physical inspections to be implemented by the industry which may be open to debate
from a rigorous risk-based perspective.
4.15: Over the period Verification has been in force, cost savings have been achieved in the
assurance of SCE integrity without compromising safety.
There was generally disagreement with this statement but also a high Neutral
response which perhaps reflects the lack of cost data as noted in the response to Q 4.9
above.
i (s)
l ility(s)
Workforce Representing Body
Regulator
ICP
Duty Holder - permanent fac liity
Duty Hoder - mobile fac
Figure 49 Response Distribution for Q 4.15: Over the period Verification has been in
force, cost savings have been achieved in the assurance of SCE integrity without
compromising safety.
A.44
10
9
8
7
6
5
4
3
2
1
0
10
9
8
7
6
5
4
3
2
1
0
S
t
r
o
n
g
D
i
s
a
g
r
e
e

S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e
N
e
u
t
r
a
l
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Q 4.16: Through life experience from Verification is being fed back effectively to
improvements for new design.
There was some agreement with this statement. One respondent noted that
considerable safety improvements could be made by early consideration of SCEs
during the design process so as to ensure their easy accessibility for inspection, testing
and maintenance.
9
8
7
6
5
4
3
2
1
0
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 50 Response distribution for Q 4.16: Through life experience from Verification
is being fed back effectively to improvements for new design.
Q 4.17: The safety benefits of Verification for major accident prevention outweigh the costs of
ongoing Verification activity.
With one Duty Holder exception, the majority of respondents expressed either
Neutrality or Agreement / Strong Agreement with this statement. Although, as
mentioned above, there is apparently little to no data available as to the costs of
Verification, there is probably an intuitive perspective that ongoing Verification costs
form only a very small component of the overall operating budgets and that the
additional safety assurance that Verification provides is, although mandatory, never-
the-less valuable.
A.45
0
2
4
6
8
10
12
14
i
(s)
(s)
Workforce Representng Body
Regulator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
S
t
r
o
n
g
D
i
s
a
g
r
e
e
N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e
N
i
l

R
e
s
p
o
n
s
e

Figure 51 Response distribution for Q 4.17: The safety benefits of Verification for
major accident prevention outweigh the costs of ongoing Verification activity.
Q 4.18: The costs of establishing and implementing the Verification regime were small
compared with the benefits achieved over Certification.
Most respondents were either Neutral (probably through lack of knowledge of cost
data) or in Agreement with this statement.
A.46
10
9
8
7
6
5
4
3
2
1
0
i
ICP
(s)
l i ility(s)
Workforce Represent ng Body
Regulator
Duty Holder - permanent faciliity
Duty Hoder - moble fac
S
t
r
o
n
g
D
i
s
a
g
r
e
e

D
i
s
a
g
r
e
e

N
e
u
t
r
a
l
A
g
r
e
e

S
t
r
o
n
g
A
g
r
e
e

N
i
l

R
e
s
p
o
n
s
e

Figure 52 Response distribution for Q 4.18: The costs of establishing and
implementing the Verification regime were small compared with the benefits achieved
over Certification.
QUESTIONNAIRE SECTION 5: COSTS OF VERIFICATION.
Although some cost data was provided by the respondents it was very limited and restricted in
nature. For instance, no Duty Holder provided cost data related to Fixed Installations. Two
Duty Holders did provide information relating to the costs of Certification and Verification for
mobile offshore drilling units (MODUs) but this cannot be taken as representative of the cost to
Duty Holders at large because of the confounding factor of the general requirement for
Classification for MODUs. The lack of cost data is thought to be attributable to, inter alia, the
following potential causes:
Cost data relating to Certification and Verification may not have been accounted as
specific line items but included in an overall category such as Operating Expense
making its subsequent abstraction difficult and perhaps, generally, impossible.
The extended time period between Certification and the present study. Historic cost
data for the Certification era may have been destroyed or be unavailable as result of
company acquisitions and mergers.
A.47
Many personnel associated with Certification will have left the industry and corporate
knowledge of Certification and Verification transition costs thereby lost.
Where cost data are available the respondents may not have the resources to abstract it
for the purposes of this study.
The electronic questionnaire was distributed to prospective respondents via their Trade
Associations and may not necessarily have been directed to the most appropriate
personnel in each Duty Holders organisation.
Because of the paucity of cost data it was not possible to draw any strong conclusions regarding
the relative cost of Certification and Verification but some subjective comments on the subject
are offered for consideration in Section 9, below.
QUESTIONNAIRE SECTION 6: GENERAL COMMENTS
The results of a keyword analysis of the responses to Q 6.1: Please add any additional
comments you may have about your experiences that you consider are relevant to this
evaluation of the impact of the Verification regulations? are depicted in Figure 53, below. The
analysis shows that areas of significant concern included the implementation of the regulations,
the ICP / Duty Holder power balance (particularly amongst the Regulators) utilisation of
information from the Verification systems and communication (particularly amongst the ICPs).
These aspects are discussed further in Section 5 of the report.
9
8
7
6
5
4
3
2
1
0
l lii (s)
ility(s)
Workforce Representing Body
Regulator
ICP
Duty Hoder - permanent faci ty
Duty Holder - mobile fac
A
l
l
o
w
s

r
i
s
k

b
a
s
e
d

a
p
p
r
a
i
s
a
l

o
f

P
S

d
e
f
a
u
l
t
s

C
h
a
n
g
e

I
C
P

\

D
H

p
o
w
e
r

b
a
l
a
n
c
e

C
o
m
m
u
n
i
c
a
t
i
o
n

I
C
P

m
a
k
e
s

v
a
l
i
d

c
o
n
t
r
i
b
u
t
i
o
n

I
C
P
s

s
o
m
e
t
i
m
e
s

c
o
m
m
e
n
t

o
n

t
h
e
i
r

o
w
n

s
c
h
e
m
e
s

N
o

r
e
c
o
g
n
i
t
i
o
n

o
f

p
r
o
b
l
e
m
s

P
o
o
r

T
U

l
i
n
k
a
g
e

w
i
t
h

d
r
i
l
l
i
n
g

D
H
s

R
e
g
s

u
n
n
e
c
c
e
s
s
a
r
i
l
y

c
o
m
p
l
e
x

U
n
t
i
m
e
l
y

i
m
p
l
e
m
e
n
t
a
t
i
o
n

o
f

r
e
c
o
m
m
e
n
d
a
t
i
o
n
s

M
o
r
e

u
s
e

o
f

V
e
r
i
f
i
c
a
t
i
o
n

s
y
s
t
e
m

d
a
t
a

S
l
o
w

\

p
o
o
r

u
n
d
e
r
s
t
a
n
d
i
n
g

&

i
m
p
l
e
m
e
n
t
a
i
o
n

b
y

H
S
E

&

D
H
s

I
m
p
r
o
v
e
d

w
o
r
k
f
o
r
c
e

u
n
d
e
r
s
t
a
n
d
i
n
g

P
o
o
r

I
C
P

c
o
n
t
i
n
u
i
t
y

\

c
o
m
p
e
t
e
n
c
e

P
o
o
r

H
S
E

c
o
n
s
i
s
t
e
n
c
y

\

e
n
f
o
r
c
e
m
e
n
t

\

g
u
i
d
a
n
c
e

Figure 53 Keyword analysis of the responses, by organisational role, to Q 6.1: Please
add any additional comments you may have about your experiences that you consider
are relevant to this evaluation of the impact of the Verification regulations?
A.48
The responses to Question 6.2: What steps would you recommend to improve the effectiveness
of Verification and what benefits would these bring? are extensively discussed in Section
5.2.5, below.
QUESTIONNAIRE SECTION 7: COMMENTS ON QUESTIONNAIRE
Figure 54 shows the response distribution for Q 7.1: Were the questions clear and
straightforward to answer?
A significant number of respondents answered No to this question. With hind-sight it would
have been preferable to have asked this as two questions as, it is believed from the piloting and
interviews, the questions were generally clear but not always, particularly in the case of
questions requiring historic cost and other such information, straight forward to answer.
Indeed, in some cases providing an answer was subsequently found not to be possible.
A fundamental difficulty was the interpretation of words by the various respondents. For
instance, some took Verification to mean the entire requirements for Verification as given in
the regulations whereas other s considered it to be only the activities of the ICPs. In future, the
use of a definitions list to accompany the questionnaire might alleviate this issue.
Other areas of difficulty arose when the respondent might agree with one part of a statement but
not another. Examples are:
Q 3.1 A key benefit was the risk assessment process, because of the increased focus
and awareness of higher risk categories, prioritisation of mitigation measures,
identification of cost savings and allocation of funds to those areas most in need.
Here several respondents agreed with the bulk of the statement but not the clause
relating to cost savings.
Q 4.7 Verification has been beneficial in highlighting the importance of particular
safety elements to those involved in their design, operation, maintenance, verification,
modification and future decommissioning. Here there were instances where the
respondent generally agreed with the statement but not in the case of design.
This issue is illustrated by comments received in respect of Question 4.7: For people in design,
verification and modification Agree, but for people in operation and maintenance -
Disagree. and for decommissioning not known. and Question 2.20: Understanding is
'acceptable' for inspection teams but only 'minimal' for maintenance teams. There were other
similar instances.
There were instances where respondents found a question ambiguous e.g.:
Question 4.2: Are 'inspection programmes' intended to be those of the Duty Holder,
ICP, HSE or all of these?
Many respondents had difficulty giving generic answers as there was perceived to be an high
variation of factors across the industry:
A.49
Question 2.4: There is a difference for floating and fixed installations and it also
varies with the Duty Holder's culture.
Questions 2.4 & 2.6: The Duty Holder cultures vary too much to allow this question
to be answered.
Question 2.6: This has been answered in respect of Mobiles where Class
requirements would continue. For fixed platforms, the suspected action is that Duty
Holders would drop Verification entirely.
Question 2.6: Answered for the 'Majors' probably different for the smaller players.
Question 2.20: The answer to this varies enormously across the industry. The rapid
rotation of senior managers is an issue in ensuring their understanding of the process,
particularly the change from prescription to goal-setting.
Question 2.20: The answers to the first three questions vary so much with the Duty
Holder it makes it impossible to answer.
Question 3.3: This answer very much depends on the Duty Holder.
Question 4.7: It is hard to give a generic answer. It varies too much with Duty
Holder to answer meaningfully.
Other comments included:
Question 2.1: SCEs are purely consequence, not risk, based.
Question 3.2: The answer is it should have done but did not do so. It was used to
justify the status quo.
Question 3.3: This should have happened but has not necessarily done so.
Question 4.2: Surely it is maintenance and not inspection that does this?
Question 4.12: The statement 'open to interpretation' is not true as Safety-Critical
Elements are defined in the Regulations.
Question 4.14: Although Verification may have led to unjustifiable cost savings etc.
the cause may not be 'lack of prescription'.
A.50
16
14
12
10
8
6
4
2
0
Yes No Nil Response
Regul
(s)
(s)
Workforce Representing Body
ator
ICP
Duty Holder - permanent faciliity
Duty Holder - mobile facility
Figure 54 Response Distribution for Q 7.1: Were the questions clear and
straightforward to answer?
A.51
APPENDIX B
TENDER SPECIFICATION
B.1
Annex B HSE/2939
Tender specification for an impact evaluation of current
legislative requirements for the verification of elements
critical to the safety of offshore installations (PR0/139)
BACKGROUND
The UK Health and Safety Commission (HSC) and the Health and Safety
Executive (HSE) are responsible for the regulation of almost all the risks to
health and safety arising from work activity in Britain. Our mission is to protect
peoples health and safety by ensuring risks in the changing workplace are
properly controlled including offshore gas and oil industry installations. Local
authorities are responsible to HSC for enforcement in offices, shops and other
parts of the services sector.
The current requirements for verification of safety-critical elements on offshore
installations replaced in 1996 earlier requirements under the Offshore
Installations (Construction arid Survey) Regulations 1974. The Construction and
Survey Regulations required, among other things, that all offshore installations
in UK waters had to have a Certificate of Fitness issued by a Certifying
Authority approved for the purpose by the Secretary of State. Lord Cullens
report on the Piper Alpha disaster recommended that HSE should review the
continuing need for the system of certification.
This review led in due course to the current requirement under the Offshore
Safety Case Regulations 1992 to establish a system of independent and
competent scrutiny of safety critical elements throughout the life cycle of an
installation, to obtain assurance that satisfactory standards will be achieved and
maintained. The verification requirements apply to all installations, both fixed
and mobile, on the UK Continental Shelf.
The aims of this change from certification to verification were to improve
standards by placing the duty to verify the integrity of safety-critical elements on
the installation duty holder, i.e. without the appointment by HSE of a Certifying
Authority as part of the arrangements. This was accompanied by more freedom
for the duty holder to make verification arrangements appropriate to each
installation. It was also intended to encourage more effective targeting of
verification resources on the areas of greatest risk potential, through the
introduction of the concept of safety criticality.
The impact evaluation will consider whether the change from installation
certification to verification achieved its objectives and at proportionate cost.
B.2
Annex B HSE/2939
WHY THE RESEARCH IS REQUIRED
The Offshore Installations (Safety Case) Regulations 1992 require all offshore
installations in UK waters to be subject to schemes to verify the suitability of
elements identified as critical to the safety of the installation. These
requirements were introduced in J une 1996 and came fully into effect in J une
1998. They were therefore excluded from an evaluation of the 1992
Regulations (and related offshore safety law) carried out in 1998/9. The new
arrangements have now had sufficient time to have an effect and it is now
feasible to evaluate their impact on offshore safety. It is important to
determine whether verification schemes have succeeded in maintaining or
improving standards of safety on the installations they cover. This project is
necessary to evaluate the impact of the regulations and provide
recommendation to improve the effectiveness of the regulations. HSE
requires independent advice on the impact of the Regulations. HSE wishes to
use an external contractor to ensure an objective and impartial evaluation that
will be accepted as valid by the offshore industry. The impact evaluation will
also consider whether the verification requirements are overly burdensome
and the reasons why the legislation could be considered to be
ineffective/effective. The results of the research will be used to inform a
review of the legislation and if necessary to revise it.
SCOPE AND OBJECTIVES OF RESEARCH
The contractor will be required to:
i) evaluate the verification requirements both in terms of achievement
(benefits) and costs. Annex 2 gives further details of questions that
will need to be addressed in the evaluation report.
ii) make recommendations to improve the effectiveness of the
requirement for verification.
METHOD
The proposal should describe and justify the method(s) to be used and it
should include a detailed programme of work with a timetable, which identifies
milestones for review. It is envisaged that the contractor should follow the
rough guide below: -
1. Identify the evidence needed and target groups. (Month 1)
2, Agree methods for data gathering and analysis. (Month 1)
3. Collect the evidence. (Month 2-4)
4. Evaluate and analyse the evidence for impact. (Month 4-5)
5. Produce a draft interim report on completion of Stage 3. (End of Month 6)
B.3
Annex B HSE/2939
6. Produce a final report, with recommendations. (Month 9-12)
The proposal should include a description and justification of the populations
to be studied, the power of the study, the statistical analysis required, the
sampling frame to be used, the experimental techniques to be employed and
the strategies to be used to maximise response rates wherever appropriate.
The industry is aware of this project and the trade associations are supportive
of it. The successful bidder will be put into contact with the industry via the
trade associations.
A risk assessment for the project should be undertaken. This should identify
the- main risks, which may affect the timescales and costs of the project or
the adequacy of the final report. It should identify what mitigating actions can
be taken, an estimate of their effectiveness and the impact on time, cost and
performance should the risk occur.
RESULTS
The results should be presented in the form of a full report (in MS Word) with
an executive summary drawing out the relevant recommendations I
conclusions for the data in the light of the original aims and objectives of the
project. All supporting data should be annexed.
Interim reports will be required quarterly. The interim report should give a
report of the progress of the project measured against the detailed
programme of work. It should include any mitigating actions to be taken to
avoid any delays in the work against the agreed milestones.
The contractor will also provide a short progress report to the Project Officer
(P0) on a monthly basis by email. This short progress report should detail
what the contractor has done to meet the milestones agreed in the schedule
of work and alert the P0 to all work schedule / programme issues as soon as
they arise. The Contractor, Technical Client and Project Officer will meet at
key stages of the -project at HSEs Rose Court Headquarters to monitor
progress/delivery, and when necessary if any difficulties arise, otherwise
meetings will be held quarterly.
The final evaluation report will be published as an HSE Contract Research
Report. Annex I contains outline contents of the final evaluation report and
should be regarded as guidelines.
B.4
Annex B HSE/2939
TIMESCALE
It is anticipated that this research may take up to 9-12 months to complete.
The final report will be required by the end of December 2004 at the latest.
B.5
Annex B HSE/2939
Annex 1
Outline of the evaluation report
Title page: title of evaluation. Title of programme/project. Author, date of
submission, commissioning Directorate.
Table of contents: main and subheadings. Index of tables of figures and
graphs.
Executive summary: an overview of the report in no more than five pages.
A discussion of the strengths and weakness of the chosen evaluation design
Description of the policy/programme to be evaluated: description of its
activities, objectives, and success indicators
Purpose of the evaluation: introduction, background, aims of the research
and main evaluation questions. Description of other similar studies that have
been done.
In formation sources and background assumptions: a description of
the sources used to collect data, including parties consulted. This section
should also clearly indicate: the baseline year for the evaluation, the period
over which the policy/programme is to be evaluated and costs and benefits
are calculated, and the price base used.
Research method: design of research and collection of data.
Counterfactual: a quantified description of what would have happened if
intervention had not taken place and of the necessary assumptions.
Data analysis: description and commentary of data collected, before,
during and following the intervention
Comparison of the outturn with the counterfactual: a quantified
assessment of the outturn and a comparison with the target outturn;
comparison with the counterfactual; discussion of any other intervening
variables, stakeholders participation, and identification of HSEs contribution
Costs to HSE: a detailed item by item list of the policy/project/programme
costs to HSE with an illustration of how totals are derived. Staff costs should
be shown in terms of hours per band spent. Note that costs are to be derived
on a full cost basis in line with HM Treasury Fees and Charges Guidance
and in line with Resource Accounting and Budgeting. Advice on how to
calculate staff costs will be given by HSEs Finance Unit 3 (Financial
Accounting and Advice FAA).
Costs to the Exchequer: an item by item list (e.g. costs to Local
Authorities)
Sectors affected and compliance costs: a brief description of the main
sectors affected. Numbers of firms, employees etc. will be given,
distinguishing between large, medium and small firms. An item by item list of
costs to business, with an illustration of how totals are derived. The starting
point for the calculation of compliance costs is the current level of compliance.
B.6
Annex B HSE/2939
Compliance costs might have been calculated in the course of a Regulatory
Impact Assessment (RIA); in this case, the evaluation serves as an ex-post
check that the assumptions made and the figures estimated in the RIA were
correct.
Other costs: other cost items (e.g. environmental costs)
Total costs to society: this will state the total costs of the
project/programme from all sources. If there are distributional and transfer
issues the implications to the different parties should be clarified here. If the
costs are expected to continue after the evaluation date, a forecast of future
costs should be included.
Health and safety benefits: this section examines the health and safety
benefits arising from the policy/programme, item by item, with an illustration of
how totals are derived.
Other benefits: this section will consider, item by item, any other benefits
(e.g., environmental benefits, cost savings for businesses, savings in
enforcement costs etc.)
Total benefits: as with costs, if the impact is expected to continue to
progress after the evaluation date, a forecast of future benefits should be
included. The forecasted period must be the same for costs as for benefits.
Weighing the balance: summary table of the information on quantified and
unquantified costs and benefits. The quantified totals should be presented
along with major qualitative considerations. Value for money (VFM)
considerations.
Conclusions on process and impact: summarises why the outturn
differed from the target (if it did); discusses what worked, and HSEs
contribution to the achievement of the outcomes.
Uncertainties: this section should discuss where the greatest uncertainties
in data and assumptions lie, and their likely impact on the results. In particular
whether conclusions and recommendations would be altered if data or
assumptions change.
Lessons and recommendations: discusses what the results imply for
future project management or policy decisions; makes clear recommendations
to the responsible Directorate (e.g. continuation, modification or replacement
of a programme). Identifies any problems encountered with the study itself
and how these problems may be avoided in future studies.
Annexes: tender specification, additional tables/questionnaires, references
and sources, glossary of terms
B.7
Annex B HSE/2939
Annex 2
Questions to be addressed in the evaluation report
The move from certification to verification was intended to improve standards
of installation safety by removing responsibility from the Certifying Authority
onto duty holders. Has this worked? If not, why not?
Has the new concept of identifying safety critical elements, led to -
improvement in risk control?
What factors determine the choice of in house or external Independent
Competent Persons to carry out the verification arrangements?
Why is there an apparent preference for the use of an external Independent
Competent Person?
How realistic were the original cost and benefit estimates in the Compliance
Cost Assessment and have they been achieved?
Is the customer! contractor relationship between the duty holder and the
Independent Competent Person conducive to rigorous independent scrutiny e.g.
do commercial pressures and competition for work compromised the scope,
depth and competence of scrutiny.
How do senior management in industry see the value of verification or does it
add value?
Are the current arrangements effective in identifying deficiencies in duty
holders arrangements for maintaining integrity of safety critical elements?
Could / should the arrangements be more effective and if so how?
Is the relationship between HSE and the Independent Competent Persons
appropriate?
Does the requirement for competent persons to be independent add value to
the process? How?
Are there any other intervening variables apart from HSEs contribution? For
example the state of the economic cycle and the structure of employment;
working and management practices (e.g. the use of subcontract labour);
production techniques - and the extent to which hazardous practices or pieces of
equipment have been engineered out; the state of scientific knowledge.
What is the contribution of HSE to the achievement of the outcomes (i.e.
excluding other stakeholders and intervening variables)?
Were any problems encountered with the study itself? How many of these
problems can be avoided in future studies?
B.8
APPENDIX C
BLANK QUESTIONNAIRE
C.1
QUESTIONNAIRE FOR AN EVALUATION OF THE VERIFICATION REQUIREMENTS
OF THE OFFSHORE INSTALLATIONS (SAFETY CASE) REGULATIONS
This questionnaire is to provide information for an HSE impact evaluation of the SCR requirements for
the Verification of elements critical to the safety of offshore installations. The specific objectives of the
research are to:
Look back and evaluate the achievements and costs of Verification compared with the previous
Certification regime.
Look forward and make recommendations for improving the effectiveness of Verification.
To achieve these objectives, it is necessary to isolate the effects specific to moving from Certification to
Verification in 1996, as opposed to other influences such as the move from a prescriptive to a goal
setting regime when SCR was introduced in 1992.
Responses will be obtained from a range of organisations e.g Duty Holders, ICPs, HSE Inspectors,
workforce representatives etc..
Except where the context is obviously wider, please respond in terms of your own organisation. The
information that you supply will be held confidentially and any openly available report will not
identify individual contributors.
Questions have been structured to address the following:
1. The perspective from which you are responding
2. An assessment of elements of Verification
3. Consideration of the impact of the wider Post-Cullen regime
4. Costs associated with Verification and wider expenditure on integrity assurance
5. Your comments and suggestions for improving the Verification regime.
Where the meaning of a question is unclear, an interpretation has been assumed, or where you wish to
comment on a question, please enter this in question 7.1 & 7.2.
Information is requested in the format considered likely to be most accessible. If information in the units
requested (e.g. cost per installation per annum) is not available, please provide any similar information
regardless of format (e.g. total days expended per annum). Do not update historic information to allow
for inflation.
Additional comments on the Verification requirements of SCR can be entered in Section 6.
Where the word Verification is used it should be taken to mean all the requirements of Regulation 15A
of SCR and not just the activities undertaken by the ICPs.
The questionnaire is designed for electronic completion. Answers are recorded in different formats
requiring a check/cross, comment, selection from a drop down pick list, or number and are generally
self-explanatory. The tab key should be used to move between questions that require answers in
text/numeric od Please select form, whilst the mouse is most suited when a check/cross is required.
Microsoft Word does not enable the spelling and grammar facility in this format, though text can be
pasted from other sources.
Some questions may not be applicable to your company: in such circumstances please select N/A
not applicable. When no reasonable answer can be provided, please select N/K not known or dont
know.
When completed, save the file giving it your organisations own name (e.g. BOMEL.doc). The
questionnaire should then be returned via email, as an attachment, to johnsmith@bomelconsult.com.
Hand completed copies can also be accepted, please return these to:J ohn K Smith, BOMEL Limited,
Ledger House, Forest Green Road, Fifield, Maidenhead SL6 2NR.
In the event of queries, please contact J ohn on 01628 777707.
Thank you for your assistance with this important safety study.
C.2
Section 1: Respondent Profile
1.1 Your name:
The information in Qs
1.1 to 1.4 will remain
confidential to
1.2 Telephone No:
BOMEL and is for
clarification purposes 1.3 Email
only
1.4 Company
1.5 Please confirm your organisations role Please select.......
If Other, please specify
1.6 Do you have experience of Certification as well as Please select......
Verification?
1.7 If answer to 1.6 is Yes, is your experience of
Certification and Verification with this
organisation?
1.8 What is your job title
1.9 What are your present roles and responsibilities
with respect to verification?
1.10 What is the profile of installations you are dealing
with and to which your responses relate?
Installation types Nos (approx)
Fixed structures 0
J ack-ups 0
Semisubs 0
FPSO 0
1.11 Please add any comments you feel relevant to
explain your experience of Certification /
Verification (e.g. previous employment, roles, age
of installations etc)
C.3
Section 2: Assessment of Verification
Verification Elements
2.1 How would you apportion the relative significance of the following key elements of Verification for
assuring offshore safety:
Moving responsibility from the CA to Duty Holders
%
Requiring risk-based targeting with the concept of SCEs
%
Requiring independence of the verification body (ICP)
%
Removing the link between the regulator and the certification/verification body
%
Other
%
If Other, please specify: Total 100%
2.2 How do you view HSEs involvement with Verification? Please select.....
2.3 Please explain your answer to 2.2 What is HSEs principal contribution; what more / less should be
done; with which parties; with what benefit / cost?
The Role of Verification
2.4 What value do you think Duty Holders (i.e. their senior management) place on Verification please
indicate the one answer nearest to your observations / experience
No value only undertaken to satisfy legal requirement
Some value but not good value for money for ensuring safety
Valuable and cost-effective for ensuring safety
Highly valued part of safety management
2.5 What health and safety benefits do you see of Verification over Certification?
2.6 If the legal requirement for Verification were now removed, which one of the following scenarios would
you envisage:
DHs would voluntarily seek a similar level of independent verification
DHs would voluntarily seek independent verification but with more limited scope
DHs would implement more internal verification but not employ independent verifiers
DHs would seek an approach more akin to certification
DHs would drop verification activity entirely
Other
If Other, please specify
2.7 If, on a scale of 1 to 5, current practices with Verification deliver a
safety level of 3 for SCEsif Verification requirements were now
removed (as in Question 2.6) where would you expect the safety of
those elements currently classed as safety critical to rate in three
years time given the scenario you envisage.
Please select for 3 years' time
What factors influence your rating?
2.8 If, on a scale of 1 to 5, current practices with Verification deliver a
safety level of 3 for SCEslooking back to around 1996 and the
Certification regime, where would you rate safety for those elements
now classed as safety critical.
Please select for Certification
What factors influence your rating?
C.4
The Parties under Verification
Appointment of ICPs
2.9 Why do you think there is a general preference to appoint ICPs who were formerly CAs?
2.10 If you are a Duty Holder, have you considered appointing an ICP who was
not previously an established CA?
Please select.....
Why / why not?
2.11 If you are a Duty Holder and a non CA ICP was considered, did you proceed
with such an appointment?
Please select.....
Why / Why not?
2.12 Why do you think there is a general preference for DHs to appoint external ICPs?
2.13 If you are a Duty Holder, has 2
nd
party Verification (i.e. independent in-
house verification) been considered?
Please select.....
Why / why not?
2.14 If you are a Duty Holder and 2
nd
party Verification was considered, was this
implemented?
Please select.....
Why / Why not?
2.15 If you are a Duty Holder, do you have one Verification body covering the
complete range of work?
Please select.....
If No (i.e. more than one), please indicate the number of Verification bodies you appoint
and the split across different disciplines / functions / assets etc
No. of bodies Description of split across disciplines / functions / assets etc
Relationships between the parties
2.16 Are you satisfied with the legal relationship between DHs and the HSE in the
context of Verification?
Please select.....
If not, please explain
2.17 Are you satisfied with the legal relationship between the DH and ICP(s)? Please select.....
If not, please explain
2.18 Is the DH/ICP commercial relationship conducive to rigorous independent
scrutiny?
Please select.....
Please explain why you hold this view
2.19 Are you satisfied with the legal relationship between ICPs and HSE? Please select.....
If not, please explain
C.5
Impact of Verification
2.20 Overall within the industry, how well is the concept of SCEs and
the importance of their suitability / condition in preventing /
controlling major accidents understood by personnel within the
Understanding is:
following groups: Inadequate Acceptable Thorough
Corporate management
Offshore Installation Managers
Operational workforce
Teams involved in modification work
Inspection / maintenance teams
2.21 Please identify examples of risk control benefits / safety improvements brought about by
Verification during operations.
Would this have
Example of Verification benefits been identified under
certification?
Potential consequences if not
identified
1 Please select....
2 Please select....
3
Please select....
2.22 What proportion of ICP recommendations from the Verification process are closed out in a
timeframe you consider to be:
Satisfactory %
Slow but low risk issues (i.e. Tolerable) %
Slow high risk issues (i.e. Unacceptable) %
Total 100% (check)
2.23 What obstacles do you see (if any) in the effective close out of issues raised through the
Verification process?
2.24 Please list precursors / Indicators of major accident potential which have changed since 1998
and indicate the extent to which Verification has contributed to the change:
Indicator Nature of change Quantitative Extent of this Other key factors
realised change in risk change due to influencing this change
(eg per QRA) Verification (%)
1 %
2 %
3 %
2.25 What do you consider are the relative contributions of Verification / examination schemes to
major accident prevention offshore
SCR Verification scheme %
PFEER examination scheme %
DCR wells examination scheme %
Total 100% (check)
C.6
Section 3: Impact of the wider Post-Cullen Regime
In 1998 AUPEC undertook an evaluation of the post-Cullen offshore safety
legislative regime. As Verification only came into force fully in J une 1998,
this aspect was excluded. However a number of key conclusions from the
AUPEC report are presented below. Please confirm the degree to which
you agree with the 1998 assessment of the regime.
Please check one box only in each row
Strongl y
disagree
<---- Neutral ---->
Strongl y
agree
3.1 A key benefit of SCR was the risk assessment process, because of
the increased focus and awareness of higher risk categories,
prioritisation of mitigation measures, identification of cost savings and
allocation of funds to those areas most in need
3.2 Building on structured decision making processes, led to targeting of
risk reduction efforts
3.3 Early benefit of DCR (e.g. in relation to well ops) was from
identification of SCEs and development of performance standards
C.7
To what extent would you agree with the following statements?
comments are welcome and can be made under Section 6.
Strongl y
disagree
<---- Neutral ---->
Strongl y
agree
4.1 The benefits of Verification on risk and safety are consistent with those
experienced from the original SCR
4.2 Beyond the initial implementation of Verification, inspection
programmes have been the primary means of assuring the SCEs meet
their performance standards
4.3 The scope and breadth of the CAs approval / inspection under
certification diluted the attention paid to elements now classed as SCEs
4.4 Certification provided a safety check in relation to major accident
precursors which may not be identified by the Verification process
4.5 The formal identification of SCEs and the Verification process have
achieved the intended regulatory purpose of allowing safety
management effort/resource to be appropriately focused on those items
with the greatest contribution to safety of the installation
4.6 For fixed installations, the schedule of SCEs originally identified with
the introduction of Verification has not changed significantly over the
subsequent 6-8 years
4.7 Verification has been beneficial in highlighting the importance of
particular safety elements to those involved in their design, operation,
maintenance, verification, modification and future decommissioning
4.8 Freedom for the DH in appointing ICPs has improved the expertise DHs
draw on
4.9 The free market in procuring ICP services has increased the value for
money/cost-effectiveness of the services
4.10 The effectiveness of Verification in relation to major accident prevention
could be improved
4.11 By placing responsibility on DHs away from CAs, Verification has
improved standards of installation safety.
4.12 ICPs have been over-cautious in the definition of SCEs and controls
because safety-criticality is open to interpretation
4.13 Under Verification, inadequate attention is now placed on assessing the
suitability and condition of elements of installations which are not
deemed safety critical
4.14 The lack of prescription under Verification has enabled cost-saving
reductions in the scope and/or frequency of physical inspection to be
implemented by the industry which may be open to debate from a
rigorous risk-based perspective.
4.15 Over the period Verification has been in force, cost savings have been
achieved in the assurance of SCE integrity without compromising
safety.
4.16 Through life experience from Verification is being fed back effectively to
improvements for new design
4.17 The safety benefits of Verification for major accident prevention
outweigh the costs of ongoing Verification activity
4.18 The costs of establishing and implementing the Verification regime
were small compared with the benefits achieved over certification
C.8
Section 5: Costs of Verification
This work is
concerned solely with the additional
Verification provisions (excluding wells).
ive



Significant changes had already been implemented by industry in the post-Cullen regime with SCR,
PFEER, MAR when the DCR introduced Verification in 1996 (fully effective 1998).
costs and benefits associated with the introduction of the
It is recognised that this cost / benefit information may be difficult to provide. However, it is very
important for testing the original assumptions about the costs and benefits of Verification and to g
realistic insight to the burden and safety benefits of the regulation. Please do your best to provide this
information, indicating the potential degree of uncertainty in your estimates if necessary.
Costs are considered below in relation to the following elements:
Baseline familiarisation / preparations for Verification
Initial implementation of Verification appointing ICP(s), developing schemes, identifying SCEs etc.
Ongoing annual expenditure on Verification activity.
Baseline Familiarisation / Preparations for Verification
5.1 What was the relative split of effort / cost across the following activities in preparing for Verification,
as relevant to your organisation?
Familiarisation with regs (professionals) %
Informing the offshore workforce (all grades) %
Amending / writing internal procedures %
Preparation to offer verification services %
Participation in industry committees on Verification provisions of DCR %
Participation in industry briefings / conferences on Verification provisions of %
DCR
Other, please specify %
Total 100%
(check)
5.2 Total Verification preparation costs related to 5.1 above
Please indicate year to which costings relate prices (year)
5.3 Total Verification preparation effort Mandays
5.4 Uncertainty in cost / effort estimate for Verification preparation ? % %
Initial Implementation of Verification (including Initial Entry to UK Waters for Mobiles)
5.5 What was the relative split of effort / cost across the following activities when Verification was
initially implemented, as appropriate to your organisation?
Effort to procure / sell ICP services and assess / demonstrate %
independence and competence
Work beyond SCR / Class to define SCEs, performance standards and
scheme for Verification
- internal effort %
- external consultancy services %
Initial Verification round
- internal effort %
- ICP services %
Liaison with HSE %
Other, please specify %
Total 100% (check)
C.9
5.6 Total Verification implementation costs related to 5.5
Please indicate year to which costings relate

prices (year)
5.7 Total Verification implementation effort Mandays
5.8 Uncertainty in cost / effort estimate for Verification implementation ?
%
%
Annual Expenditure on Ongoing Verification
5.9 What is the relative split of annual effort / cost across the following activities associated with
ongoing Verification excluding IRM contractor costs, as appropriate to your organisation:
DH Management of verification %
DH Review and maintenance of scheme %
DH Review / amendment of performance standards %
ICP services (excluding survey) %
ICP services (offshore survey) %
Other, please specify .. %
Total 100% (check)
5.10 Total ongoing annual Verification costs related to 5.9 /annum
Give costings in 2004 prices 2004 prices (year)
5.11 Typical annual IRM contractor costs relating to examination of SCEs /annum
Give costings in 2004 prices 2004 prices (year)
5.12 Total ongoing annual Verification effort Mandays
5.13 Uncertainty in cost/effort estimate for ongoing annual Verification ?% %
5.14 In comparison with the ongoing annual resource devoted to Verification (per 5.9 to 5.13 above),
what was the relative level of resource devoted under Certification?
Duty holder work under Certification cf Verification (all activities) %
CA services compared with current ICP cost / resources (excluding survey) %
CA services compared with current ICP cost / resources (offshore survey %
only)
Offshore inspection/maintenance required under Certification cf that deemed %
necessary under Verification (IRM contractor costs)
Other, please specify.. %
Distribution of Effort
5.15 Please confirm the approximate number of installations Installation types Nos (approx)
over which the above costs are spread compare your
answer to 1.10 if necessary.
Fixed structures
J ack-ups
0
0
Semisubs 0
FPSO 0
5.16 If you are an Operator of mobile units, what is the split between work undertaken to satisfy Class
and additional work necessary to meet the requirements of Verification?
From Class activity %
From additional Verification specific activity %
Total 100% (check)
C.10
Overall Integrity Assurance Costs
5.17 What cost savings or increases have been experienced in the overall Please select.....
processes and activities for assuring integrity in the period since
Verification replaced Certification? Change=
Present cost saving / increase as /installation/annum in 2004 prices
Notes/clarification:
5.18 In your judgement, what is the relative contribution (+or -) of each of the following to these net cost
savings (or increases) in the assurance of SCE integrity?
Note: Some factors may have had counteracting influences but the total should equal 100%
(e.g. +20%, +40%, +50% and -10%)
Reduced offshore IRM expenditure from risk-based targeting / prioritisation
%
Lesser ICP scope/effort compared with CA
%
CRINE driven measures
%
De-manning / rationalisation
%
ICP competitive market
%
Accumulated knowledge / experience of installation(s)
%
Contractorisation / outsourcing
%
New hardware technology
%
New scientific knowledge
%
IT/Communications technology
%
Industry collaboration / standardisation
%
In-house maturation of systems
%
Other, please specify..
%
Total 100%
(check)
C.11
Section 6: General Comments
6.1 Please add any additional comments you may have about your experiences that you consider are
relevant to this evaluation of the impact of the Verification regulations
6.2 What steps would you recommend to improve the effectiveness of Verification and what benefits
would these bring?
Steps to improve verification Benefits
1
2
3
Section 7: Comments on Questionnaire
7.1 Were the questions clear and straightforward to answer?
Yes No
7.2 If no, please advise any problems you encountered or assumptions you made in giving your
responses noting the question reference in each case.
Thank you, again, for your valuable assistance with this important safety study.
C.12
APPENDIX D
DEFINITIONS, ABREVIATIONS AND ACRONYMS
D.1
DEFINITIONS:
Verification
The SCR were amended by the 1996 Offshore Installations and Wells (Design and Construction, etc.)
Regulations (DCR) with the introduction of requirements for the Safety-Critical Elements (SCEs) of
an offshore installation to be verified as suitable by an independent and Competent Person (ICP).
These regulatory requirements are referred to throughout the report as Verification.
Safety-Critical Elements
Safety-Critical Elements (SCEs) means such parts of an installation and such of its plant (including
computer programmes), or any part thereof:
The failure of which could cause or contribute substantially to a major accident;
A purpose of which is to prevent, or limit the effect of, a major accident.
Major Accidents
Major Accidents are defined as:
A fire, explosion or the release of a dangerous substance involving death or serious personal injury
to persons on the installation or engaged in an activity on or in connection with it;
Any event involving major damage to the structure of the installation or plant affixed thereto or any
loss in the stability of the installation;
The collision of a helicopter with the installation;
The failure of life support systems for diving operations in connection with the installation, the
detachment of a diving bell used for such operations or the trapping of a diver in a diving bell or other
subsea chamber used for such operations;
Any other event arising from a work activity involving death or serious personal injury to five or
more persons on the installation or engaged in an activity in connection with it,
Certification
Prior to Verification, the Offshore Installations (Construction and Survey) Regulations required that
Certification Authorities (CAs) appointed by the Regulator should inspect and award Certificates of
Fitness for installations. This regime, finally revoked in 1998, is referred to throughout the
questionnaire as Certification.
D.2
ABBREVIATIONS AND ACRONYMS:
CA Certifying Authority
DCR The Offshore Installations and Wells (Design and Construction etc.) Regulations 1996
1996/913)
DH Duty Holder
FPSO Floating Production, Storage and Offloading Vessel
HSE Health and Safety Executive
ICP Independent and Competent Person
KPI Key Performance Indicator
MODU Mobile Offshore Drilling Unit
PFEER The Offshore Installations (Prevention of Fire and Explosion, and Emergency Response)
Regulations 1995 (SI 1995/743)
PM Planned Maintenance
PS Performance Standard
SCE Safety-Critical Element
SCR The Offshore Installations (Safety Case) Regulations 1992 (SI 1992/2885)
SIL Safety Integrity Level
SMS Safety Management System
WSE Written Scheme of Examination
Published by the Health and Safety Executive
11/05
RR397
A
n

e
v
a
l
u
a
t
i
o
n

o
f

c
u
r
r
e
n
t

l
e
g
i
s
l
a
t
i
v
e

r
e
q
u
i
r
e
m
e
n
t
s

f
o
r

v
e
r
i
f
i
c
a
t
i
o
n

o
f

e
l
e
m
e
n
t
s

c
r
i
t
i
c
a
l

t
o

t
h
e

s
a
f
e
t
y

o
f

o
f
f
s
h
o
r
e

i
n
s
t
a
l
l
a
t
i
o
n
s

H
S
E

B
O
O
K
S