Discovering Knowledge

SEN-455
Knowledge Based
Management System

© Bahria University 2018 Department of Computer Science | Bahria University 1

 Discovering Knowledge

SYSTEM TESTING AND DEPLOYMENT

© Bahria University 2018 Department of Computer Science | Bahria University 2

 Discovering Knowledge

Chapter Objectives
• Quality Assurance
• KMS Testing
• Issues in KMS Testing
• User Acceptance Testing Approach
• Test Team/Plan
• KMS Deployment
• Factors affecting User Training
• Post-Implementation Review
© Bahria University 2018 Department of Computer Science | Bahria University 3
 Discovering Knowledge

System Testing & Deployment after Knowledge Codification

Intelligence
Capture Tools
gathering Shells, tables,
Programs, books,
articles, experts tools, frames
maps, rules

KNOWLEDGE
CAPTURE KNOWLEDGE
(Creation) CODIFICATION

TESTING AND
DATABASES DEPLOYMENT

Explicit Knowledge
KNOWLEDGE
SHARING

KNOWLEDGE
CODIFICATION KNOWLEDGE
BASE

© Bahria University 2018 Department of Computer Science | Bahria University 4

 Discovering Knowledge

Quality Assurance
• KM system should meet expectations of members
represented in the KM team.
• Performance depend highly on quality of
explicit/tacit knowledge stored in KB.
• For expert, quality relates to reasoning process that
produces reliable and accurate solutions.
• For user, quality relates to system’s ease of use and
efficiency.
• For knowledge developer, quality relates to how
valid the knowledge sources are and how well they
are codified into KB.
© Bahria University 2018 Department of Computer Science | Bahria University 5
 Discovering Knowledge

KMS Testing
• To control quality, performance
and efficiency of KMBS, it is
evaluated
• Types of Testing
1) Logical Testing, make sure that the
system produces correct results.
2) User Acceptance Testing, follows
logical testing and check system’s
behavior in a realistic environment.

© Bahria University 2018 Department of Computer Science | Bahria University 6

 Discovering Knowledge

Issues in KMS Testing

• Subjective nature of tacit
knowledge.
• Lack of reliable specifications to
guide testing
• How to decide whether knowledge
is complete
• Human errors
• Lack of time
• User interface not-friendly
© Bahria University 2018 Department of Computer Science | Bahria University 7
 Discovering Knowledge

(1) Logical Testing Approaches

• Verify Knowledge base formation
– Structure of knowledge as it relates
to circular or redundant errors
– Consistency, correctness, and
completeness of knowledge base
rules
• Verify Knowledge base
functionality
– Deals with confidence and
reliability of the knowledge base.
© Bahria University 2018 Department of Computer Science | Bahria University 8
 Discovering Knowledge

Circular Errors
• Rule 1:
– IF John and Rob are co-owners of company A
THEN John and Rob own the same company
• Rule 2:
– IF John and Rob own the same company
THEN John and Rob are co-owners

© Bahria University 2018 Department of Computer Science | Bahria University 9

 Discovering Knowledge

Redundancy Errors
• Rule 1:
– IF (Salary > 50,000 OR HOME > 100,000)
THEN Loan = ‘yes’
• Rule 2:
– IF (Salary > 60,000)
THEN Loan = ‘yes’

© Bahria University 2018 Department of Computer Science | Bahria University 10

 Discovering Knowledge

Logical Errors/Attributes
• Circular errors - contradictory in meaning
or logic
• Redundancy errors - offer different
approaches to the same problem
• Unusable knowledge - comes up no
matter conditions succeed or fail
• Asumption errors - if one rule is true, one
knows the second rule is always true
• Inconsistent knowledge - the same inputs
yield different results

© Bahria University 2018 Department of Computer Science | Bahria University 11

 Discovering Knowledge

Logical Errors/Attributes
• Completeness – System/module is not
completed
 Confidence – Deals with confidence of

the knowledge base.

• Correctness – How much correct output
 Reliability – Deals with reliability of the

knowledge base.

© Bahria University 2018 Department of Computer Science | Bahria University 12

 Discovering Knowledge

2) User Acceptance Testing Approach

• Select a person or a team for testing

• Decide on user acceptance test criteria
• Develop a set of test cases unique to the
system
• Maintain a log on various versions of the
tests and test results
• Field-test the system

© Bahria University 2018 Department of Computer Science | Bahria University 13

 Discovering Knowledge

User Acceptance Testing Approach :

Test Team/Plan
• Independent of the design or
codification of the system
• Understand the knowledge
problem and computer technology
• Well versed in the organization’s
goals
• Know how the system being tested
will fit into the designated area of
operation

© Bahria University 2018 Department of Computer Science | Bahria University 14

 Discovering Knowledge

Content of Test Plan

• Items of KM system that require
testing
• Schedule of user acceptance
testing
• Description of test methods that
are approved by user and
company
• Guidelines for documenting test
results
© Bahria University 2018 Department of Computer Science | Bahria University 15
 Discovering Knowledge

User Acceptance Criteria

• Accuracy and correctness of outcome
for test items
• Suitability of the solutions
• Adaptability to novel input
• Ease of use
• Robustness
• Technical/operational test

© Bahria University 2018 Department of Computer Science | Bahria University 16

 Discovering Knowledge

User Acceptance Test Techniques

• Developing a set of test
cases
• Subsystem Validation
• Maintaining a log on
different versions of the
tests/test results
• Field testing the system

© Bahria University 2018 Department of Computer Science | Bahria University 17

 Discovering Knowledge

Managing Test Phase

The following tasks are included
• Deciding what, when, how, and where to
evaluate the knowledge base.
• Deciding who will be doing the logical
and user acceptance testing.
• Deciding about a set of evaluation
criteria.

© Bahria University 2018 Department of Computer Science | Bahria University 18

 Discovering Knowledge

Managing Test Phase

• Deciding about what should be recorded
during the test.
The following statistics are to be recorded:
– Those rules that always fire and succeed.
– Those rules that always fire and fail.
– Those rules that never fire.
– Those test cases that have failed.

© Bahria University 2018 Department of Computer Science | Bahria University 19

 Discovering Knowledge

Managing Test Phase

• Reviewing training cases (provided by the
knowledge developer, the expert or the
user).
• Testing all the rules. Two types of errors:
– Type-I Error: A rule that fails to fire when it
is supposed to fire.
– Type-II Error: A rule that fires when it is not
supposed to fire.
© Bahria University 2018 Department of Computer Science | Bahria University 20
 Discovering Knowledge

KMS Deployment
• Transfer of KM System from
knowledge developer to the
organization’s operating unit.

• Transfer of KM’s skills from

knowledge developer to the
organization’s operators.

© Bahria University 2018 Department of Computer Science | Bahria University 21

 Discovering Knowledge

System Deployment
• Factors affecting System Deployment
– Technical
– Organizational
– Procedural
– Behavioral
– Political
– Economical
© Bahria University 2018 Department of Computer Science | Bahria University 22
 Discovering Knowledge

Issues in System Deployment

• Selection of KB Problem
• Ease of Understanding the KM System
• Knowledge Transfer
• Integration Alternatives
• Maintenance
• Organizational factors
• More factors
• Champion's Role
© Bahria University 2018 Department of Computer Science | Bahria University 23
 Discovering Knowledge

Selection of KB Problem
The KM system can be assured to be successful if:
• The user(s) have prior experience with systems
applications.
• The user is actively involved in defining /identifying the
specific systems functions.
• The user is actively involved in user acceptance testing and
the final system evaluation.
• It is possible to implement the system in the working
environment without interrupting the ongoing activities.

© Bahria University 2018 Department of Computer Science | Bahria University 24

 Discovering Knowledge

Ease of Understanding the KM

System
• Reliable documentation (especially during
user training) plays a key role during
deployment.
• Documentation including examples,
illustrations, and graphics may reduce
training time.

© Bahria University 2018 Department of Computer Science | Bahria University 25

 Discovering Knowledge

Knowledge Transfer
Two Approaches used for transferring KM system
technology in implementation:
– The system is actually transferred from the
knowledge developer directly to the working unit
in the organization.
– Installing the system on the resident hardware.
• One way is abrupt i.e. one time transfer resulting
in a permanent installation and the other way is
a gradual transfer over a given time period

© Bahria University 2018 Department of Computer Science | Bahria University 26

 Discovering Knowledge

Knowledge Transfer
• Implementation can also be approached as a
stand alone installation or as a fully
integrated application that can interface with
other applications /databases.
• KM systems should be designed on platforms
which are compatible with other KM systems
in the organization.

© Bahria University 2018 Department of Computer Science | Bahria University 27

 Discovering Knowledge

Integration Alternatives
• Technical Integration: Occurs through the
organization's LAN environment, the resident
mainframe, or existing IS infrastructure.
• Knowledge Sharing Integration: Often requires the
upgradation of the LAN, the mainframe, or lines.
• Decision Making Flow Integration: Suggests that the
way the KM system assesses a problem situation
should match the user's way of thinking.
• Workflow Reengineering: Considered when
implementation of the new KM system can propose
changes in the workplace.
© Bahria University 2018 Department of Computer Science | Bahria University 28
 Discovering Knowledge

Maintenance
• Maintenance implies the way of making the
required corrections which can continue to meet
user's expectations.

Organizational factors
 Strong leadership;
 User participation in the process.
 Organizational politics.
 Organizational climate.
 User readiness (willing)
© Bahria University 2018 Department of Computer Science | Bahria University 29
 Discovering Knowledge

More factors
• Return on investment (ROI) factor.
• Quality information.

Champion's Role
 Champion is the person who, because of his/her position, influence,
power, or control, is capable to acquire and secure organizational
support for the new system (from inception to deployment).
 He/she needs to be at the executive level to act as a member of the
project's board of directors.
 He/she needs to be aware of the fact that politics, budgetary
problems or conflict of interest can stand in the way of deployment.

© Bahria University 2018 Department of Computer Science | Bahria University 30

 Discovering Knowledge

Factors affecting User Training

• Primary
– The user’s knowledge of
knowledge-based system
– Complexity of KM system and user
interface’s ease of use
• Secondary
– Trainer’s technical experience/
communication skills
– Environment of training venue

© Bahria University 2018 Department of Computer Science | Bahria University 31

 Discovering Knowledge

Pre User Training Session

• Define how the KM system agrees with
organizational mission
• Demonstrate how the system can
support organizational goals
• Allocate adequate resources for the
training project
• Advocate positive effects of the system
• Explain cost-benefit analysis of the KM
system

© Bahria University 2018 Department of Computer Science | Bahria University 32

 Discovering Knowledge

Overcome Resistances to Change

• User attitude survey

• Communication sessions

• Role negotiation

© Bahria University 2018 Department of Computer Science | Bahria University 33

 Discovering Knowledge

Post-Implementation Review
• How KM System has changed
accuracy/timeliness of decision
making
• How KM System has affected
attitude of end users
• Whether the system has caused
constructive organizational changes?
• Whether the system has benefited
the operation cost?

© Bahria University 2018 Department of Computer Science | Bahria University 34

 Discovering Knowledge

Post-Implementation Review:
Security Issues
• The new system should
provide password / protocol
protection.
• Security procedures should be
consistently observed.
• Access should be restricted
regarding update of the KB.

© Bahria University 2018 Department of Computer Science | Bahria University 35