A Bayesian Based Machine Learning Application to Task Analysis
that have arguably little face validity. Such as ap-
proach also helps balance out the effect of many
biasing factors such as individual differences
among the knowledge agents, unusual work ac-
tivities followed by a particular knowledge agent,
or sampling problems related to collecting data
from only a few or particular groups of customers
with specific questions.
2. The next step is to document the agents’ trouble-
shooting process and define the subtask categories.
The assigned subtasks can then be manually as-
signed into either the training set or testing set.
Development of subtask definitions will normally
require inputs from human experts. In addition to
the use of transcribed verbal protocols, the analyst
might consider using synchronized forms of time
stamped video/audio/data streams.
3. In our particular study, the fuzzy Bayes model
was developed (Lin, 2006) during the text min-
ing process to describe the relation between the
verbal protocol data and assigned subtasks. For
fuzzy Bayes method, the expression below is used
to classify subtasks into categories:
P( E j | S i ) P( S i )
P ( S i | E ) = MAX j were repeated ten times to allow cross-validation
P( E j )
where P(Si|E) is the posterior probability of sub-
task Si is true given the evidence E (words used
by the agent and the customer) is present,
P(Ej|Si) is the conditional probability of obtaining
the evidence Ej given that the subtask Si is true,
P(Si) is the prior probability of the subtask being
true prior to obtaining the evidence Ej, and
“MAX” is used to assign the maximum value of
calculated P(Ej|Si)*P(Si)/P(Ej).
When agent performs subtask Ai, words used by
the agent and the customer are expressed by word
vectors
WAi = (WAi1, WAi2, …, WAiq) and WCi = (WCi1,
WCi2, …, WCiq) respectively,
where WAi1, WAi2, …, WAiq are the q words in
the ith agent’s dialog/narrative;
WCi1, WCi2, …, WCiq are the q words in the ith
customer’s dialog/narrative.
Ai is considered potentially relevant to WAi, WCi-1,
and WCi for i greater than 1.

The posterior probability of subtask Ai is calculated
as follows:
P(Ai|WAi,WCi,WCi-1)=MAX[P(WAi|Ai)*P(Ai)/
P(WAi), P(WCi|Ai)*P(Ai)/P(WCi),
P(WCi-1|Ai)*P(Ai)/P(WCi-1)]
=MAX [MAXj [P(WAij|Ai)*P(Ai)/P(WAij)], MAXj
[P(WCij|Ai)*P(Ai)/P(WCij)],
MAXj [P(WC(i-1)j|Ai)*P(Ai)/P(WC(i-1)j)]] for
j=1,2,….q
To develop the model, keywords were first parsed
from the training set to form a knowledge base.
The Bayesian based machine learning tool then
learned from the knowledge base. This involved
determines combinations of words appearing in
the narratives that could be candidates for subtask
category predictors. These words were then used
to predict subtask categories, which was the output
of the fuzzy Bayes model.
4. The fuzzy Bayes model was tested on the test
set and the model performance was evaluated in
terms of hit rate, false alarm rate, and sensitivity
value. The model training and testing processes
of the accuracy of the predicted results. The testing
results showed that the average hit rate (56.55%)
was significantly greater than the average false
alarm rate (0.64%), and a sensitivity value of 2.65
greater than zero.
FUTURE TRENDS
The testing results reported above suggest that the fuzzy
Bayesian based model is able to learn and accurately
predict subtask categories from the telephone conversa-
tion between the customers and the knowledge agents.
These results are encouraging given the complexity
of the tasks addressed. That is, the problem domain
included 24 different agents, 55 printer models, 75
companies, 110 customers, and over 70 technical issues.
Future studies are needed to further evaluate model
performance that includes topics such as alternative
groupings of subtasks and words, as well as use of
word sequences. Other research opportunities include
further development and exploration of a variety of
Bayesian models, as well as comparison of model