Diagnosis of your food safety

management system
performance : output
Pieternel Luning & Willem Marcelis (WUR),
Liesbeth Jacxsens (UGent)

Food Quality and Safety

 Topics

Introduction
Food Safety Management System (FSMS)
diagnostic instrument
Food Safety Performance Indicators (SPI)
diagnose
Example of case company X

Food Quality and Safety

 Introduction
Dynamic environment
Socio-demographic changes, technological developments,
emerging pathogens, global sourcing, etc

Stakeholders
Government, retailers, branch organisations, sector boards, etc

Food safety assurance requirements

PRP, CODEX, GMP,HACCP, BRC, ISO22000, SQF, etc

Company specific Food Safety Management System

Safe products
Food Quality and Safety
This slide illustrates that companies have to operate in a dynamic environment, and
that stakeholders (that are government, retailers, branch organisations, etc) put
assurance demands on the companys FSMS.

They require that the set-up of the companys FSMS is according to e.g. PRP, HACCP
guidelines and or BRC standard, and or ISO22000 standard, etc.

So the stakeholders affect the companys FSMS set up by demanding the

implementation of certain QA guidelines/standards.

Keep in mind however, that each company has a UNIQUE FSMS which is a
translation of all the QA requirements into their own specific system

Food Quality and Safety

 Food Safety Management System
Food Safety Management System
Assurance
Food safety assurance activities on
to control safety management system and to provide Product
evidence and confidence to stakeholders about Safety
meeting safety requirements
Stakeholder
System requirements feedback requirements

Food safety control activities

aim at keeping product properties, production Product
processes, and human processes between certain
acceptable tolerances safety

Food Quality and Safety

This slide is to show the companies that there are actually to outputs of their
FSMS.
One output is the realisation of safe products, which is the result of all actual
control activities as executed by equipment and people during daily operation
under the given contextual situation.

The other output is assurance of product safety, which means being able to give
the assurance that product as made under the companys conditions are safe.

Control activities contribute to the first output whereas assurance (indirectly affects
the first output) but results in assurance.

So it does not mean that if a company is not/scarcely elaborating assurance

activities that products are not safe.but it induces a risk!!!

Food Quality and Safety

 Topics
Introduction
Food Safety Management System (FSMS)
diagnostic instrument
Food Safety Performance Indicators (SPI)
diagnose
Example of case company X

Food Quality and Safety

 FSMS diagnostic instrument :
general picture
Food safety assurance
Product Assurance
characteristics Setting system requirements
Validation on product
Verification safety

Process Documentation and record keeping

characteristics
System requirements feedback

Food safety control

Organisational Preventive measures design
characteristics
Intervention processes design
Product
Environmental safety
Monitoring system design
characteristics

Operation control strategies

Food Quality and Safety

 FSMS Diagnostic Instrument :
control activities
CORE CONTROL ACTIVITIES
Preventive measures design
Sophistication hygienic design equipment & facilities
Specificity of sanitation program
Extent personal hygiene requirements
Raw material control, etc.

Intervention processes design

Adequacy intervention equipment
Specificity maintenance program
.etc

Monitoring system design

Appropriateness CCP analysis
Adequacy analytical equipment
..etc

Operation control strategies

Appropriateness and compliance to procedures
Actual performance cooling, intervention, measuring equipment ,..etc

Food Quality and Safety

 FSMS Diagnostic Instrument :
assurance activities

CORE ASSURANCE ACTIVITIES

Defining system set-up
Sophistication translating external requirements
Extent of systematic use of feedback information

Validation Verification
Sophistication validation
Extent of verification of people
preventive measures
related performance
etc
Extent of verification equipment
related performance

Documentation and record-keeping

Food safety control system

Food Quality and Safety

This slide shows the core assurance activities. The primary objectives of assurance
are to control the system, check its effectiveness, check whether it works in practice
as designed, and adapt the system when changes are necessary. All in order to
provide evidence and confidence to stakeholders that your system is able to comply
with the safety and assurance requirements.

An important assurance activity is defining the specific system set up for the company
(based on stakeholder requirements and internal feedback information about the
FSMS performance).

Validation is aimed ate checking effectiveness in advance, whereas verification is a

check afterwards (when to check if the system is functioning properly in practice).

Documentation and record keeping support above activities and play a major role in
providing evidence and confidence to stakeholders.

Food Quality and Safety

 Example of grid
Indicator: Specificity cleaning and disinfection program

Mechanism Level 1 Level 2 Level 3

Complete, tailored Incomplete program Complete programme Complete programs,

sanitation program equal for all equal for all equipment tailored different
with appropriate Equipment & facilities. and facilities. equipment/facilities
cleaning agents, Common cleaning Cleaning agents Cleaning agents
supported with agents not specific selected based on specifically modified
appropriate user for production system. advices of suppliers; and tested on
instructions better Cleaning instructions specific agents for effectiveness for
prevents derived from typical product specific food
contamination information on label applications. production system.
positive contribution or company Idem for instructions Instructions based
to food safety experience on test results.

Food Quality and Safety

 Criteria to differentiate FSMS
(=FSC + FSA) levels
Control activities Assurance activities

Level 3 high i.e. scientifically Level 3 high i.e. systematic,

underpinned accurate, pro-active, scientific evidence,
complete, stable, predictable, independent, experimental
tailored trials, well-documented

Level 2 medium, i.e. Best Level 2 medium, i.e. regular,

practice knowledge, sometimes reactive, expert
variable, not fully predictable, knowledge/opinion, internal
generic information independent, no actual testing,
restricted documentation

Level 1 low i.e. lack of Level 1 low i.e. ad hoc,

scientific evidence, use
company experience, history,
variable, unknown,
unpredictable, common
materials, equipment
reactive, history/ company
knowledge, dependent, no
testing, no documentation
Food Quality and Safety
 CONTEXTUAL FACTORS FOOD SAFETY MANAGEMENT SYSTEM OUTCOME

Product characteristics
Assuranc
Process characteristics
Food safety assurance e on
system product
safety
Organisational characteristics
Lack of technical workforce
Variability in workforce composition
Insufficiency operators competence
Lack of commitment
Deficiency of employee involvement
Absence of formalisation
Insufficiency information support Food safety control Product
system safety
systems

Environmental characteristics

Food Quality and Safety

 Criteria to differentiate context situations

Product/process Organisational Environmental

Situation 1 : not Situation 1: low
critical, not Situation 1: high ability, dependency, not
vulnerable advanced, specific vulnerable

Situation 2 : Situation 2:
potentially critical, constrained ability, Situation 2: restricted
potentially restricted advanced, dependency,
vulnerable potentially vulnerable
specific

Situation 3: low ability, Situation 3: highly

Situation 3 : highly not advanced not dependent, highly
critical, highly vulnerable
vulnerable specific

Food Quality and Safety

 FSMS diagnostic instrument :
interpretation of results
Addresses:
Core control activities
Core assurance activities
The contextual factors of company
Assesses
at which level control & assurance activities are
executed (0,1,2,3, simple sophisticated)
in which risky contextual situation the FSMS has to
operate (1,2,3 not demanding very demanding and
riskfull)
Aggregates
Insights in relationships between context, activities
and food safety performance
Food Quality and Safety
 FSMS diagnostic instrument :
interpretation of results

Output of the assessment gives indication of:

Insights in relationships between context, activities
and food safety performance
At which level the specific control and assurance
activities of food safety control and assurance are
now
How to go what is needed for the higher level in
control and assurance activities
Middle or long term improvements can be identified
NOT:
Judgement tool no audit is performed
Food Quality and Safety
 Interpretation results of contextual factors
The contextual situations are represented in spiderweb diagrams
A more coloured spiderweb diagram is associated with a more
demanding and risky contextual situation
A more risky contextual situation is expected to result more easily in
food safety problems, which will put higher demands on the FSMS.

To obtain an overall picture of the contextual situation, overall scores

have been assigned For this purpose the mean scores for a certain
set of characteristics are calculated by dividing the sum of situation
numbers by the number of situation assessments for this specific set of
characteristics (see the legend of a spiderweb diagram).

The overall score has been assigned by interpreting the mean score of
all contextual characteristics as:
Overall score 1: if mean score of characteristics is 1 - 1.2
Overall score 1-2: if mean score of characteristics is 1.3-1.7
Overall score 2: if mean score of characteristics is 1.8 - 2.2
Overall score 2-3: if mean score of characteristics is 2.3 - 2.7
Overall score 3: if mean score of characteristics is 2.8 - 3.0
Food Quality and Safety
 Interpretation results of Food Safety Control and
Assurance Activities
The levels of the core safety control and assurance activities are
represented in spiderweb diagrams
The spiderweb diagrams represent the detailed levels of the core safety
control activities and the core safety assurance activities
A more coloured spiderweb diagram is associated with a higher/more
sophisticated level of control activities and assurance activities.

To obtain an overall picture of the FSMS activities, overall scores have

been assigned For this purpose the mean scores of levels for a
certain set of activities were calculated by dividing the sum of activity
levels by the number of activities of this specific set.

The overall score has been assigned by interpreting the mean score:
Overall score 1: if mean score of activities is 0 - 1.2
Overall score 1-2: if mean of score of activities is 1.3-1.7
Overall score 2: if mean score of activities is 1.8 - 2.2
Overall score 2-3: if mean score of activities is 2.3 - 2.7
Overall score 3: if mean score of activities is 2.8 - 3.0
Food Quality and Safety
 Assumption behind diagnosis
High risks products and/or processes (= higher overall
score in contextual factors) put higher demands on
FSMS to achieve safety requirements

Less supporting organisational conditions and/or higher

chain dependency (= higher overall score in contextual
factors) result in higher impact on FSMS to achieve
safety requirements
Which FSMS levels is necessary?

It depends !!
Food Quality and Safety
 Assumption of output

Context FSMS FS
3 3 3

2 2 2

1 1 1

3=most dangerous 3=highest level 3=best performance

Food Quality and Safety

 Topics
Introduction
Food Safety Management System (FSMS)
diagnostic instrument
Food Safety Performance Indicators (SPI)
diagnose
Example of case company X

Food Quality and Safety

 Food safety performance indicators
The microbiological food safety output of a companies
Food Safety Management System can be measured via
microbiological analysis of raw materials, intermediated
products and final food products
But the food safety output can as well be measured via
Food Safety Performance Indicators (SPI)

Indicators are measurable, objective, quantitative

measures of key system elements performance
They indicate the extent upto which a certain Food
Safety Management System meets the needs and
expectations towards microbiological food safety
They can be applied for self-evaluation or for
benchmarking
They are applied in this work to measure the food safety
output Food Quality and Safety
 Food safety performance indicators
For each indicator, a food safety level is attributed :

Level 0 : not conducted, not present, not appropriate

Level 1 : low performance of food safety problems
regarding food safety can be expected and are not
under control by the current food safety management
system
Level 2 : medium performance of food safety problems
regarding food safety can be expected and are under
control by the current food safety management system
Level 3 : high performance of food safety no problems
regarding food safety can be expected

Food Quality and Safety

 Food safety performance indicators

To obtain an overall picture of the food safety

performance, overall scores have been assigned.
For this purpose the mean scores of levels for a certain
set of food safety performance indicators need to be
calculated by dividing the sum of performance levels by
the number of indicators of this specific set.
The overall score has been assigned by interpreting the
mean score:
Overall score 1: if mean score of activities is 0 - 1.2
Overall score 1-2: if mean of score of activities is 1.3-1.7
Overall score 2: if mean score of activities is 1.8 - 2.2
Overall score 2-3: if mean score of activities is 2.3 - 2.7
Overall score 3: if mean score of activities is 2.8 - 3.0
Food Quality and Safety
 Assumption behind Food Safety
Performance Indicators diagnosis

More sophisticated FSMS would be better

able to realise products with lower
contamination levels and less deviation in
contamination loads
Different FSMS level/context
situations

may result in good FS !!

Food Quality and Safety
 Assumption of output

Context FSMS FS
3 3 3

2 2 2

1 1 1

3=most dangerous 3=highest level 3=best performance

Food Quality and Safety

 Topics
Introduction
Food Safety Management System (FSMS)
diagnostic instrument
Food Safety Performance Indicators (SPI)
diagnose
Example of case company X

Food Quality and Safety

 Assumption of output

Context FSMS FS
3 3 3

2 2 2

1 1 1

3=most dangerous 3=highest level 3=best performance

Food Quality and Safety

 Profiles contextual situation
company X
1-2 Product and process characteristics
2 Environmental characteristics

Safety contribution in chain

Risk level raw materials position
3 3
Rate of product / process 2 Risk level most critical 2
design changes 1 product groups
1
Strictness Power in supplier
0 0 relationship
of legislative
Level of production Safety contribution of requirements
process packaging concept
changes
Intervention steps Authority in customer
relationship

Organizational characteristics
2-3
Technical workforce
Supporting information
systems
3
2 Variability workforce
composition
Overall context score:
1

Formalisation
0
Operators' competence
2
Employee involvement Management commitment

Food Quality and Safety

 Profiles of core control activities
company X
1-2
Preventive measures design 2 Monitoring system design

Product specific measures CCP analysis

3 3
Corrective actions Standards and tolerances
2 2
Personal hygiene assessment
1 Hygienic design 1
requirements
0 0
Sampling design Analytical methods

Calibration program Measuring equipment

Sanitation program Critical cooling facilities

Intervention process design Operation of FSCS

0 2
Intervention equipment Appropriateness of
3 Actual analytical procedures
3 Compliance to
2 equipment performance
1 2 procedures
0 1
Actual performance of 0 Actual hygienic
Intervention methods Maintenance program measuring equipment performance of equipment
and facilities
Actual process
capability
Overall score control of intervention process
Actual cooling capacity

activities 1-2 Food Quality and Safety

 Profile of core assurance activities
company X
Food safety assurance system

Translating external
requirements
Documentation and record Use of feedback
keeping system 3 information
2
Verifying equipment and 1
methods related Validating
performance 0 preventive measures

Verifying people related Validating

intervention systems
performance
Validating monitoring system

Overall score System req.: 1-2

Validation: 1
assurance activities
Verification: 2-3
1-2
Documentation: 2
Food Quality and Safety
 Assigning overall scores to mean scores
of control and assurance activity levels
Overall score 1 if mean score of activities 0 - 1.2
Overall score 1-2 if mean of score of activities 1.3-
1.7
Overall score 2 if mean score of activities 1.8 - 2.2
Overall score 2-3 if mean score of activities 2.3 - 2.7
Overall score 3 if mean score of activities 2.8 - 3.0

To calculate the overall score for FSMS one should

take all original scores as basis!!!

Food Quality and Safety

 Assigning overall scores to mean scores
of contextual situations
Overall score 1 if mean score of activities 1 - 1.2
Overall score 1-2 if mean of score of activities 1.3-
1.7
Overall score 2 if mean score of activities 1.8 - 2.2
Overall score 2-3 if mean score of activities 2.3 - 2.7
Overall score 3 if mean score of activities 2.8 - 3.0

To calculate the overall score for context one should

take all original scores as basis!!!

Food Quality and Safety

 Food safety performance indicators
company x

Overall score 2 (because average of the score is 2)

Food Quality and Safety

 Assigning overall scores to mean scores
of food safety performance indicators

Overall score 1 if mean score of activities 0 - 1.2

Overall score 1-2 if mean of score of activities 1.3-1.7
Overall score 2 if mean score of activities 1.8 - 2.2
Overall score 2-3 if mean score of activities 2.3 - 2.7
Overall score 3 if mean score of activities 2.8 - 3.0

To calculate the overall score for FSMS one should take

all original scores as basis!!!

Food Quality and Safety

 Overall results company X

Context FSMS FS
3 3 3

2 2 2
1-2

1 1 1

3=most dangerous 3=highest level 3=best performance

Food Quality and Safety

 Conclusion : overall results company X
Assumption for this company : In context 2 we
expect FSMS level 2 and FS level 3
so, FS level lower than expected probably due to
FSMS level being lower than 2
With FS level 2 there is a potential risk on
microbiological food safety
When FS level perceived as unacceptable by
the company, measures could be discussed in
FSMS activities (towards level 2/3) and or in the
context (towards level 1)
Possible measures/interventions can be
discussed from a detailed analysis of the spider-
webs
Food Quality and Safety
 Detailed discussion FSMS interventions
for company X
Score 0: Product specific measures; intervention strategies
Score 1: Extent of personal hygiene requirements; corrective actions;
appropriateness of procedures; use of feed back information and
validation
Score 2: Sophistication hygienic design, specificity sanitation program,
compliance to procedures, sampling design and tolerances design
(important for control raw materials)
Resulting in discussions about:
-preventing raw material contamination by product specific measures
and or raw material control (including sampling design, tolerances
design of monitoring system)
-possible techological intervention strategies (because now score 0)
-personal hygiene, hygienic design, hygiene control, and cleaning and
disinfection program
-use of information for feedback and corrective actions
-adequacy of procedures and instructions and compliance to procedures
-validation of preventive measures and monitoring system
Food Quality and Safety
 Detailed discussion context interventions
for company X
Score 3: Risk level of raw materials and products; lack of
technological staff; lack of information systems; lack of
formalisation/procedures; lack of management commitment;
strictness of legislative requirements

This results in discussions about:

-reducing the risk level of raw materials (i.e.. Reducing initial load
and variation in initial load) in collaboration with suppliers
-redesigning their information system (to support decision making)
-the quality management system with its procedures (introduce
some extent of formalisation)
-people in quality staff positions (technological staff, people
responsible for FSMS activities), enhance competences/skills (e.g.
by training)
-management commitment

Food Quality and Safety

 Thanks for your attention
and
for contributing to our research

2 2 3

Food Quality and Safety