Journal 4

Development and evaluation of a multimedia
e-learning resource for electrolyte and acid-base

disorders
Mogamat Razeen Davids, Usuf M. E. Chikte and Mitchell L. Halperin
Advan in Physiol Edu 35:295-306, 2011. doi:10.1152/advan.00127.2010
You might find this additional info useful...
This article cites 39 articles, 3 of which can be accessed free at:
/content/35/3/295.full.html#ref-list-1
This article has been cited by 5 other HighWire hosted articles
Teaching general practitioners and doctors-in-training to discuss advance care planning:
evaluation of a brief multimodality education programme
Karen Detering, William Silvester, Charlie Corke, Sharyn Milnes, Rachael Fullam, Virginia
Lewis and Jodie Renton
BMJ Support Palliat Care, September , 2014; 4 (3): 313-321.
[Abstract] [Full Text] [PDF]
Teaching general practitioners and doctors-in-training to discuss advance care planning:

evaluation of a brief multimodality education programme
Karen Detering, William Silvester, Charlie Corke, Sharyn Milnes, Rachael Fullam, Virginia
Lewis and Jodie Renton
BMJ Support Palliat Care, May 20, 2014; .
Interactive computer-assisted instruction in acid-base physiology for mobile computer
platforms
Kenneth J. Longmuir
Advan in Physiol Edu, March , 2014; 38 (1): 34-41.
An efficient approach to improve the usability of e-learning resources: the role of heuristic
evaluation
Advan in Physiol Edu, September , 2013; 37 (3): 242-248.
Updated information and services including high resolution figures, can be found at:
/content/35/3/295.full.html
Additional material and information about Advances in Physiology Education can be found at:
http://www.the-aps.org/publications/advan
This information is current as of March 18, 2015.
Advances in Physiology Education is dedicated to the improvement of teaching and learning physiology, both in specialized
courses and in the broader context of general biology education. It is published four times a year in March, June, September and
December by the American Physiological Society, 9650 Rockville Pike, Bethesda MD 20814-3991. Copyright 2011 by the
American Physiological Society. ISSN: 1043-4046, ESSN: 1522-1229. Visit our website at http://www.the-aps.org/.
Downloaded from on March 18, 2015
Effect of improving the usability of an e-learning resource: a randomized trial

Advan in Physiol Edu, June , 2014; 38 (2): 155-160.
Adv Physiol Educ 35: 295306, 2011;

doi:10.1152/advan.00127.2010.
How We Teach
Development and evaluation of a multimedia e-learning resource for

electrolyte and acid-base disorders
Mogamat Razeen Davids,1 Usuf M. E. Chikte,2 and Mitchell L. Halperin3
1
Division of Nephrology and Department of Medicine, Stellenbosch University and Tygerberg Hospital, Cape Town;
Department of Interdisciplinary Health Sciences, Stellenbosch University, Cape Town, South Africa; and 3Division
of Nephrology, St Michaels Hospital and University of Toronto, Toronto, Ontario, Canada
Received 2 December 2010; Accepted 9 May 2011
clinical problem solving; Flash; wireframing; prototypes; software

development; System Usability Scale; hyponatremia; usability
ELECTROLYTE AND ACID-BASE DISORDERS
are clinical problems that

are common and may be life threatening. This area is highly
integrative and quantitative, and it is one that students and
clinicians find particularly difficult to master (9).
Medical experts solve most clinical problems using pattern
recognition, drawing on a large domain-specific database of
schemata or illness scripts (15, 22, 41). When an unusual or
complex situation is encountered, however, the expert can
draw on extensive relevant basic science knowledge and apply
it to the problem (38). This is often required in disciplines such
as anesthesiology, nephrology, and intensive care medicine,
where much of the clinical reasoning involves the application
Address for reprint requests and other correspondence: M. R. Davids,

Division of Nephrology and Dept. of Medicine, Stellenbosch Univ. and
Tygerberg Hospital, PO Box 19063, Tygerberg 7505, Cape Town, South
Africa (e-mail: mrd@sun.ac.za).
of physiology (37). Electrolyte and acid-base disorders are

typical examples common to these disciplines where an understanding of physiology is central to correct diagnosis and
treatment.
As teachers, our challenge is to help students and clinicians
develop an expertise in clinical problem solving that can be
effectively applied when they encounter related, but different,
problems. This transfer of expertise is difficult to achieve (10,
14, 39). It can be facilitated by active learning and deliberate
practice with carefully selected examples (12, 13, 40, 42).
This helps to develop a fund of domain-specific knowledge and
facilitates the abstraction of underlying concepts and the transfer of clinical reasoning ability from one problem to another.
Increasingly, medical educators are engaging learners using
animations to illustrate dynamic processes and simulations to
provide the opportunity to interact with clinical problems.
Well-known examples include Chopras operating room simulator and Harvey, the cardiology patient simulator (7, 21,
44). A receptive atmosphere now exists for the increased use of
simulations following large studies describing preventable injuries to patients as a result of medical error (4, 20, 27, 28).
Some authors view the use of simulations as an ethical
imperative and argue that harm, or exposure to the possibility
of harm, to patients in the course of training or resulting from
trainees lack of experience can only be justified once approaches that do not put patients at risk have been maximized
(62). Additional advantages of simulations include the ability
to provide exposure to uncommon conditions and a variety of
clinical presentations. Errors can be allowed, and even encouraged, as they provide valuable learning opportunities. Trainees
can then have their first encounters with real patients having
already attained higher levels of confidence and proficiency.
Developing good simulations and other e-learning materials
can be resource intensive, and it is therefore important to
ensure that the time and money invested is justified by the
educational impact. The usability of computer interfaces has a
major impact on the effectiveness of e-learning but is an aspect
that has not been sufficiently emphasized in the medical education literature (26, 33, 45). The concept derives from the field
of human-computer interaction and describes how easy a
technology interface is to use. The terms fitness for purpose
and quality of use (2) are also used to describe usability. The
International Organization for Standardization, in ISO 9241-11
(25), defines usability as the extent to which a product can be
used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of
use.
1043-4046/11 Copyright 2011 The American Physiological Society
295
Davids MR, Chikte UME, Halperin ML. Development and

evaluation of a multimedia e-learning resource for electrolyte and
acid-base disorders. Adv Physiol Educ 35: 295306, 2011;
doi:10.1152/advan.00127.2010.This article reports on the development and evaluation of a Web-based application that provides
instruction and hands-on practice in managing electrolyte and acidbase disorders. Our teaching approach, which focuses on concepts
rather than details, encourages quantitative analysis and a logical
problem-solving approach. Identifying any dangers to the patient is a
vital first step. Concepts such as an appropriate response to a given
perturbation and the need for electroneutrality in body fluids are used
repeatedly. Our Electrolyte Workshop was developed using Flash and
followed an iterative design process. Two case-based tutorials were
built in this first phase, with one tutorial including an interactive
treatment simulation. Users select from a menu of therapies and see
the impact of their choices on the patient. Appropriate text messages
are displayed, and changes in body compartment sizes, brain size, and
plasma sodium concentrations are illustrated via Flash animation.
Challenges encountered included a shortage of skilled Flash developers, budgetary constraints, and challenges in communication between
the authors and the developers. The application was evaluated via user
testing by residents and specialists in internal medicine. Satisfaction
was measured with a questionnaire based on the System Usability
Scale. The mean System Usability Scale score was 78.4 13.8,
indicating a good level of usability. Participants rated the content as
being scientifically sound; they liked the teaching approach and felt
that concepts were conveyed clearly. They indicated that the application held their interest, that it increased their understanding of hyponatremia, and that they would recommend this learning resource to
others.
How We Teach
296
AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS
resources required, and highlight the lessons learned. Recommendations are made for managing the development of similar
projects. We then report on our evaluation of user satisfaction
involving a group of specialists and postgraduate trainees in
internal medicine.
METHODS AND RESULTS
Ethical approval for the project was granted by the Health

Research Ethics Committee of Stellenbosch University (approval no. N08/05/158).
The Underlying Teaching Approach
Our teaching approach is based on an understanding that
learning in this area is most effective when it is built around the
relevant basic sciences (9, 37, 38, 59, 60). Building sound
mental models based on physiological principles is likely to aid
knowledge retention and retrieval, especially when novel or
complex problems are encountered and when pattern recognition might not be effective (38). We emphasize an understanding of integrative whole body physiology, deductive reasoning,
a focus on concepts rather than details, and quantitative analysis. We emphasize principles of control and point out to our
students that if one recognizes the function of a metabolic
process, it is often possible to deduce the nature of its likely
controls (46). For example, if one views buffering as a way to
protect enzymes, receptors, and transporters from an acid load,
then the importance of directing protons to the bicarbonate
buffer system becomes clear. Students are then better able to
deduce that this process can be optimized by effective removal
of the resultant CO2 through hyperventilation and the provision
of an adequate circulation.
We use clinical and laboratory data from real cases in our
teaching. Each case starts with the identification of a key
abnormality, usually the most abnormal electrolyte or acidbase parameter. This is usually self-evident and often the
reason for the consultation. When there is more than one major
abnormality present, any one can be selected as the starting
point and the analysis followed through to the end. Thereafter,
the next abnormality is dealt with in a similar way.
The first step is always to identify and address immediate
threats for the patient and to anticipate dangers that may
develop later. These often arise as unintended consequences of
therapy. The primary imperative is always to save the patient!
We continue with the diagnostic workup once urgent therapeutic issues have been addressed. Patient data are analyzed
and interpreted focusing on the actual versus expected (appropriate) response to the particular perturbation. We repeatedly
emphasize the concept of an appropriate response and the
ability to recognize it in a given situation. This requires an
understanding of the relevant physiology. Students are asked
what an appropriate response would be and then what clinical
or laboratory data are required to identify whether their patient
is responding appropriately, for example, How should your
patients kidneys respond to hypernatremia? and What data
would tell you that antidiuretic hormone is acting on the
kidneys?
A problem-solving process that is simple and logical is
encouraged. Basic principles such as mass balance and the
need for electroneutrality in body fluid compartments are used
Advances in Physiology Education VOL
35 SEPTEMBER 2011
High usability of e-learning materials is an essential element

in ensuring maximum educational impact (3, 5), especially
when dealing with complex subject matter (49, 50). Poor
design can impair learning, as the user has to struggle with
challenging content as well as with the technology interface.
Reducing such an extraneous cognitive load can lead to large
gains in learning efficiency (49); thus, optimizing the usability
of learning resources seems essential. A recent review of
internet-based medical education (58) reported that learners are
more likely to be engaged when the technology is easy to use.
The routine evaluation of usability is well established in the
software development industry. An iterative design approach is
followed and involves the creation of prototypes, testing them,
and making improvements based on the test results (32). This
cycle is repeated until performance and usability goals are met,
and only then will the application be shipped to the marketplace. This approach is seldom used in the development of
e-learning resources, especially in the area of medical education (45).
The two main types of evaluation techniques are empirical
user testing, which involves typical end users using the application, and usability inspection, which involves experts evaluating the application against established metrics or design
principles (5). Evaluations are conducted in a wide range of
settings, from sophisticated usability laboratories (34) through
to informal settings using paper prototypes and think-aloud
protocols (47). Many tools are available to assist with the
collection, analysis, and reporting of usability data (24). These
range from self-administered questionnaires (6) to specialized
software for recording and analyzing user testing sessions.
Selecting which measures of usability to use is difficult. Some
are subjective and others objective, all have their own cost and
time requirements, and all examine a particular aspect of
usability. Among the objective measures are parameters such
as successful task completion, completion times, and error
rates, whereas subjective measures include aspects such as
satisfaction, perceived workload, fun, aesthetics, and flow (23).
Nielsen and colleagues (35, 36, 54) have popularized simpler
and less expensive methods, pointing out that any usability
testing is better than no testing at all, and demonstrating that
four to five users are sufficient for each cycle of testing.
We have developed a Web-based multimedia resource to
help students and clinicians acquire expertise in the diagnosis
and treatment of electrolyte and acid-base disorders. Colleagues in disciplines such as internal medicine, pediatrics,
nephrology, endocrinology, emergency medicine, and intensive care medicine are included in our target user population.
Our Electrolyte Workshop provides instruction and allows
for hands-on practice in treating electrolyte disorders via a
highly interactive simulation. This e-learning application can
be freely accessed at http://www.learnphysiology.org/sim1/.
An evaluation of the Electrolyte Workshop was conducted with
a group of residents and specialists in internal medicine to
determine its level of usability. User satisfaction was measured
via a questionnaire based on the widely used and validated
System Usability Scale (SUS) (6). The SUS is a low-cost
option that is easily administered and scored and is therefore an
especially useful tool in resource-constrained environments.
In this article, we briefly describe our general teaching
approach and report on the development of our Electrolyte
Workshop. We discuss the challenges encountered, outline the
How We Teach
Our teaching approach provided the background for the

development of the multimedia, Web-based learning resource
that is discussed below. Here, we describe our e-learning
application, the development process involved, and the challenges encountered and suggest some recommendations for
managing similar projects.
Development of the Electrolyte Workshop
Description of the application. Flash from Adobe Systems
(www.adobe.com) was used for the development of our Electrolyte Workshop, a case-based multimedia application illustrating and simulating the pathophysiology, diagnosis, and
treatment of a variety of electrolyte and acid-base disorders.
Flash can provide an engaging user experience by producing
interactive content that can include pictures, sound, and video.
It is well suited to creating rich content for the Web because its
files are relatively small.
The Electrolyte Workshop (Fig. 1) consists of two main
sections, each currently containing one case-based tutorial on
hyponatremia. Eventually we plan to cover all the common
electrolyte and acid-base disorders, with several examples of
each. In the WalkThru section, the case consists of a series of
14 slides that presents the clinical problem of acute hyponatremia related to ingesting the drug Ecstasy. Through words
and pictures, we demonstrate how an expert would interpret the
patient data and embark on treatment. Flash animation is used
to illustrate and emphasize important changes in body compartment sizes, brain size, blood pressure, and plasma sodium
concentrations (PNa). The pace at which information is presented is controlled by the user as s/he navigates from one slide
Fig. 1. The Electrolyte Workshop. In the WalkThru section of the application, the user navigates through case scenarios to learn how an expert would analyze
the data and embark on treatment. Animation is used to illustrate changes in body compartment sizes, brain size, blood pressure, and plasma sodium
concentrations. In the interactive HandsOn section, a treatment console allows users to practice managing the patient. The glossary provides explanations for
terms that may be unfamiliar. Hyperlinks in the text of the case scenarios link to the appropriate glossary entries.
35 SEPTEMBER 2011
repeatedly. Other frequently used concepts are those of driving forces and permeability, which are the elements that
determine whether water or solutes will move across cell
membranes. Through this line of inquiry and educational
approach the student arrives at a functional diagnosis, e.g.,
fast sodium absorption disease in a patient with hypokalemia
and hypertension. This is followed by making a structural or
anatomic diagnosis, e.g., overactive epithelial sodium channel
disease, and is sometimes followed by assigning a specific
diagnostic label, e.g., Liddles syndrome. We emphasize the
systematic analytic process and not the arrival at the correct
diagnostic label. To discourage students from taking shortcuts
and jumping to possible final diagnoses too quickly (i.e.,
guessing!), we often ask them to interpret a set of clinical and
laboratory data, specifying that a diagnostic label is not
required.
A quantitative analytic approach is always promoted. For
example, In this 60-kg female with a plasma sodium concentration of 130 mmol/l where weve estimated the extracellular
fluid volume to be contracted by 10%, what would be the
magnitude of her sodium deficit? followed by What volume
of 0.9% saline would be required to correct a sodium deficit of
230 mmol? Since exact answers are seldom required at the
bedside, students are urged to round off numbers and develop
their skills at estimating rather than resorting to using a
calculator. Another example comes from a case of cholera (61)
where severe extracellular fluid contraction completely masked
metabolic acidosis. An important teaching point was that one
has to consider the content, and not just the concentration, of
plasma bicarbonate when assessing acid-base status.
297
How We Teach
298
To provide support, especially for novice users, there is a

glossary (Fig. 3), which can be accessed from the main navigation menu or via text hyperlinks in the cases. Terms that
might be unfamiliar or require further explanation are underlined to indicate a hyperlink to the glossary.
The development process. We contracted a team of Flash
developers with extensive experience in Web design and animation but with no background in the biomedical sciences.
After initial discussions, we constructed wireframes or storyboards to communicate our ideas for the two case scenarios.
Microsoft PowerPoint was used for this purpose.
The iterative development process that followed is shown in
Figs. 2 and 4. From each set of PowerPoint slides, the developers created static screenshots (.jpeg files) to reflect the
different screens or slides in the case. We reviewed these and
compiled a list of changes, which were then implemented by
the developers. Two iterations were required during this phase.
The application was designed to fit into an area of 800 600
pixels to accommodate users with smaller computer screens. A
grayscale version was produced initially, and color was added
once all changes had been agreed upon.
An animated, interactive version was then built using Flash.
The ActionScript programming language, which is similar to
JavaScript, provided interactivity and controlled the simulation
(56). The algorithms and formulae embedded in the ActionScript code were provided by the authors. It calculated the
effects of user-selected therapies and then controlled the display of appropriate text messages and changes to graphic
elements. For example, in response to the administration of a
particular volume of hypotonic fluid, the brain of the patient
would swell by a precisely calculated amount, body fluid
compartment volumes would increase, and PNa would decrease accordingly. A text message would then be displayed
based on the resultant PNa, for example, PNa has fallen even
further! This means that brain cells are swelling. Would you
like to try something else?
After three cycles of development and reviewing and revising the application, the fully animated, interactive version was
completed.
Challenges. Several challenges were encountered during the
development process (Table 2). The shortage of skilled Flash
developers has been mentioned. A limited budget was another
constraint (see below for a discussion of the costs involved).
With respect to the actual development of the simulation, the
main challenge was the difficulty in communicating to the
developers exactly what was required. Their feedback was that
our PowerPoint wireframes were not detailed or accurate
enough. As this was a novel project for all concerned, it was
difficult to fix all the specifications for the work at the start.
Different expectations of the number of iterations that would
be allowed became apparent, as did differences in the list of
features that were included in the original costing. These issues
were amicably resolved through discussion.
Fig. 2. A and B: the treatment simulation illustrates the iterative development process. This simulation of a patient with chronic hyponatremia offers the user
a selection of treatments and dosages and displays important patient parameters, including brain size, fluid compartment volumes, and plasma sodium
concentration. Additional feedback is provided by way of text messages. A: a wireframe was constructed by the authors using PowerPoint, with each slide
representing a screen of the application. dDAVP, desmopressin; ICF, intracellular fluid; ECF, extracellular fluid; SBP, systolic blood pressure. B: the final, live
version of the treatment simulation, which was built using Flash. The images, values, and text messages are dynamic, changing in response to user input.
35 SEPTEMBER 2011
to the next. One of the interesting aspects highlighted in this

case is that of hidden dangers related to stomach contents.
Water ingested shortly before admission, and present in the
lumen of the gastrointestinal tract, still has to be absorbed and
may further aggravate the severe hyponatremia.
The HandsOn section is also self-paced but more interactive.
The case is that of chronic hyponatremia due to Addisons
disease. It starts off with five lead-in slides that describe the
clinical problem and highlight the key issues such as the
adrenal hormone deficiencies and the contacted extracellular
fluid volume. Users are then provided with a treatment console (Fig. 2, A and B), a highly interactive simulation where
they are able to select from a menu of therapies (and dosages)
and apply their treatment. The main issue here is the danger of
too rapid correction of the chronically low PNa, which may
result in serious neurological damage. The available therapies
include a selection of intravenous fluids, from water through to
3% saline. It also includes drugs (a vasopressin analog and
cortisol in this case) and sodium or potassium salts. More than
one treatment can be administered; this happens sequentially
and not simultaneously, so that feedback can be given after
each step via on-screen text and animations. The animations
illustrate the effects of treatment choices by showing changes
in body fluid compartment volumes, brain size, blood pressure,
and PNa. The text messages indicate the success of the interventions applied, for example, Your patient developed osmotic demyelination and died from serious neurological damage. This was caused by a too-rapid rise in PNa! Try again?
After successful completion of the treatment simulation, the
case concludes with a final slide of take home messages.
The two cases were chosen as they provide striking examples of acute and chronic hyponatremia, respectively. Additional examples will be added later so that so that students
encounter the same important principles in a variety of contexts. In both sections we use the teaching approach described
above and try to foster a better grasp of the underlying
physiology principles (Table 1). The quantitative aspect is
stressed consistently to reinforce the importance of accuracy in
the assessment of the disorders and in prescribing treatment.
The HandsOn simulation provides practice to help users develop a better feel for treating these conditions and, in particular, the ability to use intravenous fluids correctly and confidently.
An informal, conversational style was deliberately chosen
for the presentation of the cases. Users are addressed directly
using words such as you and your. Mayer and colleagues
(29) have shown that this personalization effect can improve
the engagement of users and promote active learning. As
undergraduate and postgraduate health science students are the
main target audience, we created a trendy young character
called Suzie (Fig. 1, bottom left) as the patient in the case
scenarios, also with the intention of increasing students engagement through the compelling need to save her from harm!
How We Teach
299
35 SEPTEMBER 2011
How We Teach
300
Table 1. Examples of physiology concepts taught in the cases

Concept
Comment
The PNa is the ratio of Na to water in the ECF compartment.

Effective osmolality (reflected by PNa) determines the size of the ICF
compartment.
Acute hyponatremia may develop when there is intake of EFW and vasopressin
prevents its renal excretion.
A smaller muscle mass results in a greater degree of hyponatremia if a given
volume of EFW is retained.
Water in the stomach can be absorbed after admission and further lower PNa.
Examine the mass balances for both water and Na to predict the change in PNa.
Na is the major ECF osmole, and Na content therefore determines ECF volume.
Na excretion should be minimal when ECF and effective arterial volume are low.
Water excretion may impaired when there is a low effective arterial volume.
The major danger with chronic hyponatremia is osmotic demyelination from a toorapid rise in PNa.
Hyponatremia may be due to a loss of Na and/or a net gain of

water.
Cells are swollen with hyponatremia and shrunken with
hypernatremia; brain cells are the most important in this
regard.
Both a source of EFW and vasopressin are needed to develop
and sustain acute hyponatremia.
Brain cell swelling is more likely in individuals who are
smaller or cachectic if EFW is retained.
Take a good history of fluids ingested and monitor PNa and
neurological status carefully.
This simple but very useful tool is also called a tonicity
balance.
The ECF volume should be contracted if hyponatremia is due
to a Na deficit.
Any Na excretion is excessive with a low effective arterial
volume.
This is due to the reduced glomerular filtration rate with low
filtrate delivery to the distal nephron and from the
nonosmotic release of vasopressin.
The risk is greater in patients with hypokalemia or malnutrition.
For future projects, the developers have requested that detailed and accurate screen-by-screen wireframes be provided
up front. This should include all content and all algorithms and
formulae with relevant text messages needed for user feedback
in the interactive parts of the application. It should be clearly
stated which parameters need to be tracked by the application.
Finally, it must be specified at the start which elements need to
be editable by the client. This would establish the full extent of
the development required. Not only would this be essential for

accurate cost assessment, but it provides the basis for a written
agreement from which the project can be managed and eventually signed off.
Resources required. There is a shortage of skilled Flash
developers in South Africa, and more than a year was spent
finding a suitable team who were willing and able to execute
the project within the available budget. We focused on finding
Fig. 3. The glossary. The glossary provides help with terms that may be unfamiliar or need further explanation. It can be accessed via hyperlinks within the text
of the cases or from the main navigation tab at the top of the screen.
35 SEPTEMBER 2011
PNa, plasma Na concentration; ECF, extracellular fluid; ICF, intracellular fluid; EFW, electrolyte-free water.
How We Teach
Fig. 4. The development process. From a PowerPoint wireframe, which was

supplied by the authors, the developers constructed static screenshots of the
application using .jpeg files. These were revised in several iterations until
agreement was reached on the look and functionality of the interface. The
coding and animation were then added, and, after several more cycles of
testing and debugging, the live application was finalized.
Evaluation of the Electrolyte Workshop

Participants and testing procedures. User testing was conducted at two South African academic departments of medicine
with 10 residents and 6 specialists in internal medicine, nephrology, and endocrinology. This group is typical of our
target user population because their disciplines involve the

management of electrolyte and acid-base disorders. Although
five users have often been reported to be sufficient to test
usability (35), we engaged a larger number of participants to
improve the usability error detection rate (16, 53) and to allow
us to consider the influence of participants knowledge and
experience. The application was presented via two 15-in.
laptop computers, each equipped with a mouse. Participants
received written instructions that included information about
the purpose of the application and of their involvement, which
was to help us improve the application. Their tasks were to
view the home page, familiarize themselves with the different
sections of the application, and then work through the cases in
the WalkThru and HandsOn sections, trying different options
in the treatment simulation. They were also asked to look at the
glossary. No time limits were set, and participants worked at
their own pace.
Data collection. Participants each completed a two-page
questionnaire at the end of their session. The first part consisted
of the 10-item SUS developed by Brooke (6). SUS yields a
single number representing a composite measure of the overall
usability of the system being studied. The scores have a
potential range of 0 to 100, with a score of 70 or greater
regarded as acceptable (1). SUS is widely used by usability
professionals and is reliable, freely distributed, easy to administer, and easy to score (1, 53). It can be used to provide a point
estimate measure of usability and customer satisfaction, compare different tasks within the same interface, compare different versions of a system, and compare competing systems or
interface technologies (1).
We used SUS with minor adaptations (Table 3), replacing
all occurrences of the word system with application and
changing the word cumbersome in item 8 to cumbersome/clumsy. Other authors have recommended replacing
cumbersome with awkward to improve understanding,
especially when the survey involves non-native English
speakers, as was the case with some of our participants
(1, 17).
The next section of the questionnaire (Table 4) included 11
statements about various aspects of the application, such as the
Table 2. Challenges in the development process

Client Experiences
Developer Experiences
Communication
Very challenging communicating ideas about a complex

simulation to developers with no background in
biomedical science. The importance of the simulation
being both qualitatively and quantitatively accurate
was not initially appreciated by the developers.
Expectations that the application

would be editable
We expected the application to serve as a template,

with the addition of further cases being easier and
less costly. We expected to be able to edit and add to
the glossary as well as the text on the various slides.
We viewed this as an iterative process where the final
product was not clear in our minds at the start. We
expected to go through many iterations until the
application was perfect.
Difficulties in the client effectively communicating the logic,

knowledge, and understanding of all the possible
consequences: . . .exist in your or your students
minds. . ., needs to be worked out on paper. . . Only at
the end did it become apparent that we are needing to be
tracking and reporting on more than one thing, i.e., ECF,
PNa, etc.; this was not apparent only from the
[PowerPoint] we were given.
The client expectations of the end product and deliverables
versus the available budget was too high. . . .Expectation
that this workshop would be fully editable is not realistic
given the existing budget.
Additional features and content were added after the initial
discussions [and cost assessment], e.g., the glossary.
Project scope and iterations

expected before completion
Shown are the challenges as perceived by the authors (client) and the development team. Both parties were inexperienced with software development projects
of this nature. Authors comments or interpretation are in brackets.
35 SEPTEMBER 2011
developers in the greater Cape Town area so that face-to-face

meetings could be held as required.
The development team comprised of one programmer and
one designer. The work was completed over a period of 7 mo.
Development costs amounted to approximately $5,300
(R40,000; $1 R7.60), a greatly reduced rate offered because
of the academic, not for profit nature of the project and because
the interest of the developers was piqued by the project. The
usual rate for commercial Flash development in South Africa is
$60 85/h (R450 650/h), and a fixed-price estimate for a
project such as this would usually be around $12,500
(R95,000).
301
How We Teach
302
Table 3. Summary of the SUS results

Strongly
Disagree
Modified SUS Statements*
1.
2.
3.
4.
5.
6.
7.
I would like to use this application often if more cases are added.
I found the application complex.
I thought the application was easy to use.
I need the support of an expert to be able to use this application.
I found the various parts of the application well integrated.
I thought there was too much inconsistency in the application.
I would imagine that most of my colleagues would learn to use
this application very quickly.
8. I found the application cumbersome/clumsy to use.
9. I felt very confident using the application.
10. Ill need to learn a lot of things before I could use this
application.
25
10
62.5
37.5
9
1
7
56.25
6.25
43.75
Disagree
n
7
1
4
1
8
4
1
2
Neutral
43.75
6.25
25
6.25
50
25
6.25
12.5
Agree
Strongly agree
4
2
4
25
12.5
25
12
1
9
75
6.25
56.25
2
2
2
2
1
1
12.5
12.5
12.5
12.5
6.25
6.25
8
1
6
50
6.25
37.5
31.25
56.25
1
4
1
6.25
25
6.25
1
5
3
6.25
31.25
18.75
1
5
3
6.25
31.25
18.75
n no. of particpants, with n 16 participants in total. SUS, System Usability Survey. Statements were scored with a five-point Likert scale, where 1
strongly disagree, 3 neutral, and 5 strongly agree. *For clarity, all occurrences of the word system were replaced with application, and cumbersome
in item 8 was changed to cumbersome/clumsy.
On analysis of individual questionnaire items, senior clinicians expressed a greater degree of confidence in using the
application (P 0.037), but there were no other differences
between the two groups. User satisfaction with various aspects
of the Electrolyte Workshop is shown in Table 4. Participants
rated the content as being scientifically sound (15 of 16
participants agreed); they liked the clinical detective story
approach (14 of 16 participants), the emphasis on key concepts
(14 of 16 participants), and felt that these concepts were
conveyed clearly (14 of 16 participants). They indicated that
the application held their interest (14 of 16 participants), that it
increased their understanding of the topic (14 of 16 participants), and that they would recommend this learning resource
to others (15 of 16 participants).
A few participants felt that the glossary was not useful (5 of
16 participants) and that navigation was difficult (3 of 16
participants). The treatment simulation was experienced as
realistic by 9 of 16 participants (5 participants were neutral)
and increased 8 participants confidence for managing similar
Table 4. User satisfaction with the Electrolyte Workshop

Strongly
Disagree
Statements
Content scientifically sound

Glossary was not useful
Liked clinical detective story
Animations distracted, unhelpful
Key concepts put over clearly
Did not like the character (Suzie)
Liked emphasis on key principles
Increased my understanding
Navigation was difficult
Failed to hold my interest
Would recommend to others
Treatment simulation
Very realistic
Increased my confidence
Difficult to use
Too far removed from the real world
37.5
8
9
7
9
1
8
7
Disagree
n
50
3
1
6
18.75
6.25
37.5
56.25
12.5
43.75
56.25
2
5
5
12.5
31.25
31.25
6.25
50
43.75
2
1
2
2
12.5
6.25
12.5
12.5
Neutral
Agree
Strongly Agree
1
2
1
1
2
5
2
6.25
12.5
6.25
6.25
12.5
31.25
12.5
5
4
7
1
6
31.25
25
43.75
6.25
37.5
10
1
7
62.5
6.25
43.75
50
50
25
6.25
6.25
6.25
37.5
62.5
18.75
6.25
31.25
8
4
1
1
1
6
10
3
1
5
5
6
4
6
31.25
37.5
25
37.5
4
4
1
25
25
10
62.5
5
4
2
31.25
25
12.5
6.25
n no. of particpants, with n 16 participants in total. Statements were scored with a five-point Likert scale, where 1 strongly disagree, 3 neutral, and
5 strongly agree.
35 SEPTEMBER 2011
soundness of the content, the ease of navigation, and whether

participants would recommend the resource to others. Participants indicated their level of agreement on a five-point Likert
scale. This was followed by questions on the treatment simulation (Table 4) and a question on the length of each case
(Table 5). Participants were also asked about the suitability of
the application as a learning resource for groups ranging from
specialists to medical and nursing students (Table 6).
Participants were then asked to rate their own level of
computer literacy (Table 7). Finally, they were asked to comment on things they liked about the application, anything they
did not like or which could be improved, and for final suggestions or comments.
Results of the evaluation. The results of the SUS are shown
in Table 3. The mean score was 78.4 13.8 (range: 45100).
There were no differences between senior (specialists, n 6)
and more junior (residents, n 10) colleagues. Mean scores
were 82.1 10.5 and 76.3 15.6 (P 0.477) in these two
groups, respectively.
How We Teach
Table 5. Participant evaluation of the case length

Very
Long
n
Long
Case length
Just Right
Very
Short
Short
6.25
12
75
18.75
n no. of particpants, with n 16 participants in total.
DISCUSSION
A multimedia application was successfully developed to

provide instruction in the area of electrolyte and acid-base
disorders. It is a reusable learning object that is sharable and
that can easily be incorporated into learning management
systems. Two case scenarios were built in this first phase to
explore the feasibility and optimal design of the application.
While learning from authentic, complex, and ill-defined
problems is encouraged by constructivist approaches and facilitates the transfer of expertise, novices may be overwhelmed
and demotivated if problems are too difficult (52). The WalkThru section of the application was therefore designed to
facilitate learning using worked-out examples (43, 51). This
allows students to move from focusing on finding solutions to
appreciating the underlying principles or rules. Offering stepby-step guidance in this section promoted the development of
expertise by making the thinking of experts visible, in line
with the cognitive apprenticeship model (8). This model involves a focus on teaching the processes, the cognitive and
metacognitive skills, by which experts solve complex problems.
Deliberate practice in a specific domain is important in the

acquisition of expertise (11), and the HandsOn section was
designed around this principle. The treatment console is an
example of a deterministic simulation, where a given action by
the user in a particular situation always produces the same
result. This predictability helps novices build confidence in
their ability to apply treatment correctly and accurately. In the
future, we intend to cater for more experienced users by
including probabilistic simulations to model the uncertainty
and unpredictability that is always part of managing real
patients (19).
Although the initial development included only two case
scenarios, eventually multiple examples of each type of disorder will be presented, so that students encounter key concepts
and the same step-by-step physiology-based approach in a
variety of contexts. Sound physiological principles should
provide the framework around which their new knowledge and
schemata are built. This facilitates knowledge retrieval and
application and also improves the accuracy of the nonanalytic
(pattern recognition) components of the clinical reasoning
process (60). The provision of a range of cases will also cater
for users with different levels of expertise, with more advanced
users free to skip some of the simpler WalkThru cases and
tackle the more challenging HandsOn cases directly.
Regarding our choice of development platform, we decided
on Flash to create a resource that was visually appealing and
interactive, increasing the likelihood that students would be
engaged and motivated to use it. It also needed to be delivered
via the internet so as to be accessible to a wide audience.
Learning objects created with Flash can be used with a variety
of learning management systems. They can be accessed via any
Web browser using the free Adobe Flash Player plug-in,
avoiding the problems of cross-browser and cross-platform
incompatibility. They will run on almost all personal computers as well as an increasing number of mobile phones and other
devices (30). The delivery of rich content with relatively small
file sizes is possible because Flash uses vector graphics, which
are represented by mathematical formulae, rather than bitmap
graphics with their larger file sizes.
Mastering Flash involves a very steep learning curve, however, and thus it is not a realistic option for most educators.
Skilled developers are in short supply and are expensive. Since
starting the project, we have become aware of many tools that
allow nonexperts to build e-learning courses without special
programming skills. These range from simple PowerPoint-toFlash converters through to high-end applications with prebuilt
templates, interactions, quizzes, branching logic, and the like. The
eLearning Guild (www.elearningguild.com) has compiled a useful report on authoring and development tools (55) that could
serve as a starting point for readers interested in developing their
Table 6. Participant evaluation of the suitability of the application

Subspecialty
Registrars
Suitability
Registrars
Specialists
Medical Students
Renal and/or
Intensive Care
Unit Nurses
Nursing Students
15
93.75
16
100
14
87.4
13
81.25
50
18.75

35 SEPTEMBER 2011
cases (6 participants were neutral). It was considered difficult

to use by two participants and too far removed from the real
world to be useful by one participant (6 participants were
neutral).
Participants considered the application suitable for residents,
subspecialty trainees, and specialists. It was also considered
suitable for medical students but not for nursing students. Half
of the participants considered it suitable for renal and intensive
care unit nurses. Of the three participants who indicated that
their level of computer literacy was weak, none found the
application difficult to use.
Comments from the open-ended questions are shown in
Table 8. Participants liked the interactive nature of the application, the real-life cases, and the overall design. Most of the
negative comments related to the treatment simulation. These
included being limited to apply only one treatment at a time,
struggling with the slider control, inadequate guidance and
feedback from the application, and an inability to navigate
back to the lead-in slides once in the treatment simulation.
303
How We Teach
304
Table 7. Participant evaluation of their own computer literacy

Very Weak
n
Weak
%
My computer literacy
Adequate
Good
Very Good
18.75
50
18.75
12.5
the chances of a successful outcome: the use of wireframes and

prototyping, following an iterative development process, and
drawing up a formal written agreement.
Recommendations
Wireframes and prototypes. Wireframes and prototypes are
used to represent the structure and functionality of a website or
desktop application (57) and are constructed early on in the
development process, before any artwork or coding is undertaken. They provide a basis for communication between clients
and developers, helping to define the functionality of each page
(or screen) and the positioning of elements such as navigation
menus and search fields. It is important to reach agreement on
the user interface and functionality right at the beginning of the
project. Once these specifications have been defined, an accurate cost assessment can be done. These critical early steps help
to reduce the risks and costs of the software development
process. Simple wireframes can be created using paper prototyping (47) or easy-to-use software such as Microsoft PowerPoint, Balsamiq Mockups (www.balsamiq.com), MockupScreens (www.mockupscreens.com), or Pencil Project (www.
evolus.vn/pencil), an add-on for the Firefox browser. Higher-
Table 8. Comments from the open-ended questions

Postitive Comments
Negative Comments and Suggestions for Improvement
Immediate feedback. All the information immediately available.

Realistic clinical scenario.
A new way of teaching such a difficult topic.
Interactivity. Format kept my interest.
User friendly. Colourful, fun.
Step by step approach. Good explanation; topic well covered.
I enjoyed the opportunity to change the different treatments and
see how it affects the patient.
Very practical and allows a realistic experience with
electrolytes and experimentation, seeing how different
modalities affect the patient.
It simulated real case scenario very well.
Visual learning - very applicable.
Relevant character with real-life problem (especially WalkThru
case). Extremely well designed with really funky layout
and illustrations.
I think this is a major leap forward in the training of electrolyte
and acid-base disorders. Thank you.
Much improved way of teaching.
Potentially good program especially because hyponatraemia
treatment can be very tricky.
A good, practical refreshing model to learn and become more
comfortable with electrolyte problems.
Program easy to use and understand - can benefit medical
personnel with electrolyte and fluid management.
Thanks, I would definitely use this as a learning aid if there
were more cases.
This makes electrolyte physiology fun and knowledge thereof
useful.
Add SI units [normal values] to numbers used.

Would have liked to be able to go back to clinical details after reaching treatment
console i.e. navigation difficult.
Did not like treatment options and approach to them. [only 1 treatment at a time, etc.]
Treatment console - not enough options, difficult to understand (is fluid given per hr
or per 24 hr?), after cortisol administration it gives the option of more treatment which ones? do they mean fluid? No feedback afterwards from program. Correct
answer not given. What is purpose of the cortisol block? A bit stressful!
I would like to be able to use combinations of therapy. One cant go back except to
start from the beginning of the case once one is in the treatment console. One of
the response messages was not complete
Technical aspects - cannot click directly onto treatment scroll bar [and pick dose],
must scroll [drag slider]. Select radio button - did not know what the radio button
was.
I was not aware there was [only] one case scenario; I thought there were more
coming so I sat for a long time thinking more were coming. Suggest informing
user how many cases there are.
Glossary can be emphasized - I missed this until the end.
Authors comments or interpretation are in brackets.

35 SEPTEMBER 2011
own materials. In the future, we plan to use Flash developers more

selectively, thus reducing our dependency on expensive, scarce
skills. For a project like this one, for example, we could use
PowerPoint to develop many of the slides and the simpler animations ourselves. The more complex animations and the treatment
simulation would then be the only parts we would need to
outsource, and these components would then simply be inserted
on the appropriate slide of the case. The entire PowerPoint
presentation could then easily be converted to Flash using a
variety of applications such as Articulate (www.articulate.com),
iSpring (www.ispringfree.com), Adobe Presenter (www.adobe.
com/products/presenter), and others.
Those e-learning applications that have to be custom built by
independent contractors need to produce the desired end product within time and budgetary restrictions and ideally allow for
easy expansion and maintenance. Unfortunately, this is the
case with only a minority of software development projects.
Research by the Standish Group (48) has found that the most
common outcome is that projects are late, over budget, or have
less than the required features. Many projects are never completed or used, and only around one-third are delivered on time,
within budget, and with the required features. For small projects such as ours, we offer some recommendations to increase
How We Teach
User Satisfaction
Overall, our Electrolyte Workshop was positively received
by the participants in our initial evaluation, with the SUS score
of 78.4 indicating a good level of usability. The additional
questionnaire items confirmed the satisfaction of participants
with the case-based approach and overall design of the application. They considered it useful, thought that it improved their
understanding of the topic, and would recommend the resource
to others. It was considered to be a suitable learning resource
for residents and specialists, our target audience, and also for
medical students. Of some concern was the data on the treatment simulation in the HandsOn case. Only 9 of 16 participants
found it realistic, and only half felt that it increased their
confidence for managing similar problems. Difficulties with
the selection and application of treatments as well as inadequate guidance and feedback were highlighted as issues in this
interactive part of the application.
Conclusions
In our Electrolyte Workshop, we have the foundation of a
multimedia resource that has the potential to offer a rich,
immersive learning experience and assist students and colleagues to acquire expertise in the area of electrolyte and
acid-base disorders. User testing with the aid of a standardized
questionnaire indicated that we achieved a good level of
usability. Further evaluation should include objective measures
of usability and an assessment of gains in knowledge. The
development of e-learning materials of high quality requires a
multidisciplinary team that includes content experts, instructional designers, and developers. Implementing good project
management, with clarification of roles and expectations, is
important in ensuring a successful outcome. Finally, using an
iterative development approach with the routine testing of
usability is an essential aspect in realizing the full educational
potential of the electronic medium.
ACKNOWLEDGMENTS
The authors thank the team at PixelProject (www.pixelproject.com) for
generously providing Flash expertise and Martin Schreiber for a critical review
of the final drafts of the manuscript.
GRANTS
This work was supported by grants from the South African Universities
Health Sciences Information Technology Consortium and Stellenbosch Universitys Fund for Innovation and Research into Learning and Teaching.
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the author(s).
REFERENCES
1. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system
usability scale. Int J Hum Comp Interact 24: 574 594, 2008.
2. Bevan N. Measuring usability as quality of use. Software Qual J 4:
115130, 1995.
3. Boling E, Sousa G. Interface design issues in the future of business
training. Business Horizons 36: 54, 1993.
4. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers
AG, Newhouse JP, Weiler PC, Hiatt HH. Incidence of adverse events
and negligence in hospitalized patients. Results of the Harvard Medical
Practice Study I. N Engl J Med 324: 370 376, 1991.
5. Brinck T, Gergle D, Wood SD. Usability for the Web: Designing Web
Sites That Work. San Francisco, CA: Morgan Kaufmann, 2002.
6. Brooke J. SUS: a quick and dirty usability scale. In: Usability Evaluation in Industry, edited by Jordan PW, Thomas B, Weerdmeester BA,
McClelland IL. London: Taylor & Francis, 1996, p. 189 194.
7. Chopra V, Gesink BJ, de Jong J, Bovill JG, Spierdijk J, Brand R.
Does training on an anaesthesia simulator lead to improvement in performance? Br J Anaesth 73: 293297, 1994.
8. Collins A, Brown J, Newman S. Cognitive Apprenticeship: Teaching the
Craft of Reading, Writing, and Mathematics. Technical Report No. 403
(online). http://www.eric.ed.gov/PDFS/ED284181.pdf [25 May 2011].
9. Dawson-Saunders B, Feltovich PJ, Coulson RL, Steward DE. A survey
of medical school teachers to identify basic biomedical concepts medical
students should understand. Acad Med 65: 448 454, 1990.
10. Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving: an
Analysis of Clinical Reasoning. Cambridge, MA: Harvard Univ. Press,
1978.
11. Ericsson KA. Deliberate practice and acquisition of expert performance:
a general overview. Acad Emerg Med 15: 988 994, 2008.
12. Ericsson KA. Deliberate practice and the acquisition and maintenance of
expert performance in medicine and related domains. Acad Med 79:
S70 S81, 2004.
13. Ericsson KA, Nandagopal K, Roring RW. Toward a science of exceptional achievement: attaining superior performance through deliberate
practice. Ann NY Acad Sci 1172: 199 217, 2009.
14. Eva KW, Neville AJ, Norman GR. Exploring the etiology of content
specificity: factors influencing analogic transfer and problem solving.
Acad Med 73: S15, 1998.
15. Eva KW, Norman GR, Neville AJ, Wood TJ, Brooks LR. Expertnovice differences in memory: a reformulation. Teach Learn Med 14:
257263, 2002.
16. Faulkner L. Beyond the five-user assumption: benefits of increased
sample sizes in usability testing. Behav Res Methods Instrum Comp 35:
379, 2003.
17. Finstad K. The system usability scale and non-native English speakers. J
Usabil Studies 1: 185188, 2006.
18. Fishman S. Legal Guide to Web & Software Development. Berkeley, CA:
Nolo, 2007, p. 528.
35 SEPTEMBER 2011
fidelity tools allow for the creation of prototypes with a richer user
interface, more interactivity, and even basic conditional logic.
They are usually more expensive, with a steeper learning curve,
which may be daunting for nontechnical users. Software programs
in this category include Flash and Flex (www.adobe.com), Axure
(www.axure.com), and Irise (www.irise.com).
Iterative development process. An iterative development
process usually involves the following: 1) identifying the basic
requirements; 2) developing a first version of the application,
typically a basic prototype; 3) review(s) by the client and
preferably end users; and 4) revision by the developers based
on the feedback received. There may be several iterations of
steps 3 and 4 until there is agreement on the user interface and
functionality. The code is then written to transform the prototype into a dynamic, interactive application.
Written agreement. A written agreement concluded at the
beginning of the project helps with managing the project and
preventing disputes. Essential elements include software specifications, timelines and payments, ownership of the software,
warranties, and dispute resolution (18). The developer should
be required to fix software errors at no charge for a specified
period of time. It is advisable to break down a sizable project
into discrete parts and link payments to the completion of each
part. This also makes it easier to monitor progress and avoids
the danger of getting an unsatisfactory product at the end. The
intellectual property rights to the software usually reside with
the developer. However, many different options can be negotiated, ranging from sole ownership by the client to ownership
by the developer with the client merely having a license to use
the software. Finally, provision must be made for resolving
disputes, preferably through mediation or arbitration.
305
How We Teach
306

40. Patel VL, Glaser R, Arocha JF. Cognition and expertise: acquisition of
medical competence. Clin Invest Med 23: 256 260, 2000.
41. Patel VL, Groen GJ, Frederiksen CH. Differences between medical
students and doctors in memory for clinical cases. Med Educ 20: 39,
1986.
42. Reimann P, Schult TJ. Turning examples into cases: acquiring knowledge structures for analogical problem solving. Educ Psychol 31: 123
132, 1996.
43. Renkl A. The worked-out examples principle in multimedia learning. In:
The Cambridge Handbook of Multimedia Learning, edited by Mayer RE.
Cambridge: Cambridge Univ. Press, 2005.
44. Sajid AW, Ewy GA, Felner JM, Gessner I, Gordon MS, Mayer JW,
Shub C, Waugh RA. Cardiology patient simulator and computer-assisted
instruction technologies in bedside teaching. Med Educ 24: 512517,
1990.
45. Sandars J. The importance of usability testing to allow e-learning to reach
its potential for medical education. Educ Prim Care 21: 6 8, 2010.
46. Schreiber M, Halperin ML. The Paleolithic curriculum: figure it out
(with the help of experts). Adv Physiol Educ 20: 185194, 1998.
47. Snyder C. Paper Prototyping: the Fast and Easy Way to Design and
Refine User Interfaces. San Diego, CA: Morgan Kaufmann, 2003.
48. Standish Group. CHAOS Summary 2009: the 10 Laws of CHAOS (online).
http://www.statelibrary.state.pa.us/portal/server.pt/document/690719/
chaos_summary_2009_pdf. [01 November 2010].
49. Sweller J. Cognitive load theory, learning difficulty, and instructional
design. Learn Instruct 4: 295312, 1994.
50. Sweller J, Chandler P. Why some material is difficult to learn. Cogn
Instruct 12: 185233, 1994.
51. Sweller J, Cooper GA. The use of worked examples as a substitute for
problem solving in learning algebra. Cogn Instruct 2: 59 89, 1985.
52. Tobias S, Duffy TM. Constructivist Instruction: Success or Failure? New
York: Routledge, 2009.
53. Tullis TS, Stetson JN. A Comparison of Questionnaires for Assessing
Website Usability (online). http://home.comcast.net/tomtullis/publications/
UPA2004TullisStetson.pdf [25 May 2011].
54. Virzi R. Refining the test phase of usability evaluation: how many
subjects is enough? Hum Factors 34: 457 468, 1992.
55. Wexler S, Schlenker Bruce B, Clothier P, Miller D, Nguyen F.
Authoring and Development Tools. Santa Rosa, CA: The e-Learning
Guild, 2008; http://www.elearningguild.com/research/.
56. Wikipedia. ActionScript. http://en.wikipedia.org/wiki/ActionScript [25
May 2011].
57. Wikipedia. Website wireframe. http://en.wikipedia.org/wiki/Website_
wireframe [25 May 2011].
58. Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a
realist review of what works, for whom and in what circumstances. BMC
Med Educ 10: 12, 2010.
59. Woods NN, Brooks LR, Norman GR. The value of basic science in
clinical diagnosis: creating coherence among signs and symptoms. Med
Educ 39: 107112, 2005.
60. Woods NN, Neville AJ, Levinson AJ, Howey EH, Oczkowski WJ,
Norman GR. The value of basic science in clinical diagnosis. Acad Med
81: S124 S127, 2006.
61. Zalunardo N, Lemaire M, Davids MR, Halperin ML. Acidosis in a
patient with cholera: a need to redefine concepts. QJM 97: 681 696, 2004.
62. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med 78: 783788, 2003.
35 SEPTEMBER 2011
19. Friedman CP. Anatomy of the clinical simulation. Acad Med 70: 205
209, 1995.
20. Gawande AA, Thomas EJ, Zinner MJ, Brennan TA. The incidence and
nature of surgical adverse events in Colorado and Utah in 1992. Surgery
126: 66, 1999.
21. Gordon MS, Ewy GA, DeLeon AC Jr, Waugh RA, Felner JM, Forker
AD, Gessner IH, Mayer JW, Patterson D. Harvey, the cardiology
patient simulator: pilot studies on teaching effectiveness. Am J Cardiol 45:
791796, 1980.
22. Groen GJ, Patel VL. Medical problem-solving: some questionable assumptions. Med Educ 19: 95100, 1985.
23. Hornbk K. Current practice in measuring usability: challenges to
usability studies and research. Int J Hum Comp Studies 64: 79 102, 2006.
24. Howarth J, Smith-Jackson T, Hartson R. Supporting novice usability
practitioners with usability engineering tools. Int J Hum Comp Studies 67:
533549, 2009.
25. International Organization for Standardization. ISO 9241-11. Ergonomic
Requirements for Office Work with Visual Display Terminals (VDTs)Part
11: Guidance on Usability (online). http://www.it.uu.se/edu/course/
homepage/acsd/vt09/ISO9241part11.pdf [25 May 2011].
26. Jones MG, Farquhar JD. User interface design for web-based instruction. In: Web-Based Instruction, edited by Khan BH. Englewood Cliffs,
NJ: Educational Technology, 1997, p. 239 244.
27. Kohn LT, Corrigan J, Donaldson MS. To Err is Human: Building a
Safer Health System. Washington, DC: National Academy, 2000.
28. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes
BA, Hebert L, Newhouse JP, Weiler PC, Hiatt H. The nature of adverse
events in hospitalized patients. Results of the Harvard Medical Practice
Study II. N Engl J Med 324: 377384, 1991.
29. Mayer RE. Principles of multimedia learning based on social cues:
personalization, voice and image principles. In: The Cambridge Handbook
of Multimedia Learning, edited by Mayer RE. New York: Cambridge
Univ. Press, 2005.
30. Millward-Brown. Adobe Plug-In Technology Study, September 2010 (online). http://www.adobe.com/products/player_census/flashplayer/ [28 November 2010].
31. Nielsen J. Guerrilla HCI: using discount usability engineering to penetrate
the intimidation barrier. In: Cost-Justifying Usability, edited by Bias RG,
Mayhew DJ. Boston, MA: Academic, 1994.
32. Nielsen J. Iterative user-interface design. Computer 26: 41, 1993.
33. Nielsen J. Usability engineering. Boston, MA: Academic, 1993.
34. Nielsen J. Usability Laboratories: a 1994 Survey (online). http://www.
useit.com/papers/uselabs.html [25 May 2011].
35. Nielsen J, Landauer TK. A mathematical model of the finding of
usability problems. In: Proceedings of the INTERCHI 1993 Conference on
Human Factors in Computing Systems. Amsterdam: IOS, 1993, p. 206
213.
36. Nielsen J, Mack RL: Usability Inspection Methods. New York: Wiley,
1994, p. 413.
37. Norman G. The essential role of basic science in medical education: the
perspective from psychology. Clin Invest Med 23: 4751, 2000.
38. Norman GR, Trott AL, Brooks LR, Smith EKM. Cognitive differences
in clinical reasoning related to postgraduate training. Teach Learn Med 6:
114 120, 1994.
39. Patel VL, Cranton PA. Transfer of student learning in medical education.
J Med Educ 58: 126 135, 1983.

Journal 4

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Journal 4

Загружено:

Авторское право:

Доступные форматы

Development and evaluation of a multimedia

e-learning resource for electrolyte and acid-base

Teaching general practitioners and doctors-in-training to discuss advance care planning:

This information is current as of March 18, 2015.

Downloaded from on March 18, 2015

Effect of improving the usability of an e-learning resource: a randomized trial

Adv Physiol Educ 35: 295306, 2011;

Development and evaluation of a multimedia e-learning resource for

Received 2 December 2010; Accepted 9 May 2011

clinical problem solving; Flash; wireframing; prototypes; software

are clinical problems that

Address for reprint requests and other correspondence: M. R. Davids,

of physiology (37). Electrolyte and acid-base disorders are

1043-4046/11 Copyright 2011 The American Physiological Society

Downloaded from on March 18, 2015

Davids MR, Chikte UME, Halperin ML. Development and

AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS

Ethical approval for the project was granted by the Health

Advances in Physiology Education VOL

Downloaded from on March 18, 2015

High usability of e-learning materials is an essential element

Our teaching approach provided the background for the

Downloaded from on March 18, 2015

AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS

To provide support, especially for novice users, there is a

Downloaded from on March 18, 2015

to the next. One of the interesting aspects highlighted in this

Downloaded from on March 18, 2015

Advances in Physiology Education VOL

AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS

Table 1. Examples of physiology concepts taught in the cases

The PNa is the ratio of Na to water in the ECF compartment.

Hyponatremia may be due to a loss of Na and/or a net gain of

the development required. Not only would this be essential for

Downloaded from on March 18, 2015

Fig. 4. The development process. From a PowerPoint wireframe, which was

Evaluation of the Electrolyte Workshop

target user population because their disciplines involve the

Table 2. Challenges in the development process

Very challenging communicating ideas about a complex

Expectations that the application

We expected the application to serve as a template,

Difficulties in the client effectively communicating the logic,

Project scope and iterations

Downloaded from on March 18, 2015

developers in the greater Cape Town area so that face-to-face

AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS

Table 3. Summary of the SUS results

Table 4. User satisfaction with the Electrolyte Workshop

Content scientifically sound

Downloaded from on March 18, 2015

soundness of the content, the ease of navigation, and whether

Table 5. Participant evaluation of the case length

n no. of particpants, with n 16 participants in total.

A multimedia application was successfully developed to

Deliberate practice in a specific domain is important in the

Table 6. Participant evaluation of the suitability of the application

n no. of particpants, with n 16 participants in total.

Downloaded from on March 18, 2015

cases (6 participants were neutral). It was considered difficult

AN E-LEARNING RESOURCE FOR ELECTROLYTE AND ACID-BASE DISORDERS

Table 7. Participant evaluation of their own computer literacy

n no. of particpants, with n 16 participants in total.