Академический Документы
Профессиональный Документы
Культура Документы
Yeong-Tay Sun
Center for HCI at Virginia Tech
2202 Kraft Drive
Blacksburg, Virginia 24060
540-231-7409
YTSun@vt.edu
Laurian C. Vega
Center for HCI at Virginia Tech
2202 Kraft Drive
Blacksburg, Virginia 24060
540-231-7409
LHoby@vt.edu
D. Scott McCrickard
Center for HCI at Virginia Tech
2202 Kraft Drive
Blacksburg, Virginia 24060
540-231-7409
McCrickS@cs.vt.edu
Abstract: The field of Human-Computer Interaction (HCI) is intimately ingrained with the
emerging development of educational resource tools. Case study libraries are one such tool that
have proven to be of great interest due to their adaptability as learning resources. This study
explores the ability of a collection of such case study libraries to support its users, gradually
revealing useful heuristics information derived from design characteristics affecting the usability of
each system. Though current results are preliminary, several interesting questions regarding
usability issues have been raised that may warrant additional assessment.
1. INTRODUCTION
Design, in general, is nebulous. In Human-Computer Interaction (HCI), we design first with the user in mind.
However, even within the domain on HCI there is debate on the 'correct' way to start designing [5]. Therefore, there
are many issues that teachers of design principles (the fundamental rules of interface design) can expect to address,
from choices in development methodologies (such as usability engineering or goal-directed design) to learning
objectives in the classroom. An emergent focus in education is on the design and implementation of useful learning
tools that may help students excel in their understanding of specific topics. Methodologies, while indispensable for
introducing an established curriculum from which to base course material and exercises, are necessarily complex
due to their nature and invariably retain their own deficiencies such as timeliness, lack of regular feedback and lack
of breadth [9]. In light of this, the introduction of tools that could sufficiently support and expedite the study of
different concepts being taught in design courses today would be immensely useful, particularly to novices of those
subjects [1]. The field of HCI, with its goals for improving interface design and implementation, is well-suited to the
development of such tools intended for improving the learning curriculum. Out of the many different tools that have
been proposed, the use of case study libraries have proven to be of some immediate interest.
A case study involves a persistent analysis of a single event, possibly over some period of time. The basic structure
of such a study provides relevant data with some degree of contextual information, while a subsequent analysis of
the case's events eventually arrives at any number of conclusions. As a result, the reader of the case study will be
able to better understand why these events happened as they did, and may also gain insight as to what elements
might warrant a more extensive analysis in future studies. Such cases lend themselves to both the generation and
testing of hypotheses and, where appropriate, have proven themselves to be an immensely useful learning tool due to
their ability to present problems in the form of an intuitive "story-like" narrative or structure [3]. Additional studies
suggest that case-based approaches to learning promote key meta-cognitive skills including cognitive elaboration,
error management, reflection, self-regulation, and transfer of knowledge [3]. Perhaps the most useful feature of a
case study is the ability to simulate problems in a practical, realistic manner. This makes it easier to draw parallels to
real-world situations and enhance the relevance and applicability of the material to both students and instructors
alike. With a solid understanding of the underlying concepts of one case, one might be encouraged to make a logical
leap of intuition to discover parallels between previously unassociated cases [4].
Obviously, a single use-case study would have severely limited applications in both scope and utility; thus we turn
our attention to large collections of multiple use cases, and how such a resource can be beneficial to students in an
academic setting. In the context of this discussion, we introduce an implementation of such a library called the
Usability Case Study (UCS) which serves as a link between collections of relevant case studies and related artifacts
[8]. A prevailing characteristic of each of these cases is their attention to the detailed documentation of the steps and
artifacts generated during a project's conception. These may include requirements stages, task analysis, user-
modeling, various forms of prototyping, evolutionary development, and usability evaluation methods involved in
real-world instances of design examples, ultimately allowing for a prototyping of cases at various levels of
abstraction [3]. In the original study, this tool was employed in the task of supporting specific topics of a usability
engineering course over the length of a semester by illustrating the different aspects of a project's design process. By
the semester's end, reactions towards the UCS were generally positive. In particular, it was reported that students,
following a natural period of adaptation to the system, developed a more sophisticated understanding of usability
engineering practices through their readings of homework assignments and participation in case-based activities [3].
The overall result was a better technical understanding of how human-centered software design works. It should be
noted that this system was not intended as a replacement for regular lecturing or class sessions; rather, it was meant
to support and enhance the comprehension of specific topics so that they might consume less time from other areas
to be covered over the course's duration.
While this experience has certainly formed a point of contention, it should be emphasized that at this time we are not
attempting to make any specific assertions regarding the ability of the UCS to support its users (as there is
insufficient evidence to make such a contention). Obviously, there are quite a few different plausible explanations
for this phenomenon. The fault may lie with some previously undetected differences between the two groups
exposed to the system, or it may be with the individuals conducting the studies. However, there is also the distinct
possibility that the issue may lie within the fundamental design of the system itself. With these results as inspiration,
there are several courses of action that we could take. For example, we could compile the lists of reactions and
arguments from the system's users and try to incorporate these suggestions into the revision of a newer, better
system. However, the problem with the UCS may be a problem in general with case study libraries. At this point in
our research we would like to reassess case study libraries - in general - to generate and collect some heuristics for
designing systems like the UCS. That is to say, based on information collected and analyzed on our own accord, we
would like to have some assurance that the grievances of the users of the system are justified (and perhaps correlated
in some manner).
To this end, a brief evaluation of three different online case study libraries incorporating useful functions or
distinguishing characteristics was conducted. Each of these libraries has satisfied our criteria for a successful
implementation of a use case library: they have either been integrated into the regular learning curriculum in an
academic setting, and/or they have been featured prominently as a section of a publicly accessible website with
relatively high visibility for some period of time. The focus of each library in itself is not especially significant, but
collectively they serve as a representative sampling of actual case study libraries currently in use. Despite their
differing subject matters, each website shares a certain number of similarities as well as unique differences, both of
which we would like to examine in detail.
http://path.upmc.edu/cases.html
This library incorporates a fairly large database with over 480 cases represented, providing entries from the last
decade or so. Photographs or diagrams of pathological data (cultures, etc.) are depicted, as applicable or appropriate.
Diagnoses compare impacted organs of a patient to healthy, normally functioning model. A wide variety of defects
requiring different degrees and techniques of diagnosis are presented. An interesting navigational element is
incorporated by splitting each case study into two parts. The first part includes all relevant details of a case, along
with a preliminary diagnosis and other relevant information. At the bottom of each page is a link to the second half
of the case study which presents a detailed "Final Diagnosis" of a patient's condition. Presumably, the separation of
these two sections is to encourage critical thinking from the user by giving them the opportunity to speculate on their
own diagnosis based on previously available information, rather than digesting all of the conclusions in the same
motion. This library lacks powerful search and categorization tools.
UCS Similarities: Complex Search w/ advanced parsing, Mainly sidebar-driven, High-level context provided as
initial view, Standard template format, Multimedia, Well-structured artifacts, Consistent formatting, Real-world
examples
http://www.american.edu/TED/class/all.htm
This is a 14-year archive of cases (consisting of two collections bi-annually) covering trade issues that affect the
environment on a global scale; many countries are covered. Case studies are presented as a rather lengthy articles of
text; multimedia is incorporated as applicable. The large amounts of information draw parallels to real-world
scenarios where it may not be immediately obvious as to what is relevant and what isn't. Each case is generally
divided into sections based on a unified template, while mnemonics are used for easy case identification. On closer
inspection, there seems to be a great diversity among the authors/submitters for the cases included in this library.
Most likely, this is a culmination of resources that was taken from local sources and adapted to the format of this
library. Although the issues of quality control and content review inevitably come into play at some point, this is an
excellent way of keeping the pool of case studies diverse and relevant.
UCS Similarities: Key scenarios essential to case comprehension appear in their own section, along with relevant
supporting materials, Standard template format, Multimedia, Artifact interpretation is open, Consistent formatting,
Real-world examples
http://www.cre.gov.uk/legal/casedatabase.html
These case studies span a number of years, with landmark cases being clearly delineated. Key points of the law that
factor heavily into the subsequent ruling of a case are outlined, while a summary of rulings appears towards the end;
this includes related laws and arguments influencing a decision. Cases are divided into seven sections according to
the most common groupings of cases. The body itself incorporates a combination of both bullet-style lists and
paragraphs of text, selecting the most suitable one for a given sub-section. Summaries, by nature, tend to emphasize
the notable points of an argument and would be better suited to a list-style layout. Case backgrounds, besides giving
specific information, are also meant to establish a case's context and are written in paragraph form to maintain their
"narrative" style. Besides contextual awareness this variety discreetly separates the text into manageable chunks,
resulting in an immensely more readable page.
UCS Similarities: Complex Search w/ advanced parsing, Mainly sidebar-driven, High-level context provided as
initial view, Standard template format, Well-structured artifacts, Real-world examples
2.1.4 Discussion
You may have noticed that none of the selected case study libraries are of a usability engineering-centric, or even
particularly technical (computer-wise) nature. This is mainly because the characteristics that we are attempting to
gather should be those that are common among all successful case study libraries. If we were to restrict our
observations to libraries that focus only on similar subject matters, we might very well be overlooking some
essential feature of a successful library, perhaps one with a predisposition towards not appearing in a technically-
oriented case library. Secondly, it is generally assumed that observations between similar libraries have already been
conducted to some degree (and likely in the original design stages of the UCS as well). By observing these common
characteristics we wish for the ability to recreate particular features from scratch based on their demonstrated worth,
rather than drawing on inspiration from a particular source.
3. EXPERIMENTAL PROCEEDINGS
3.1 Purpose
Such case libraries are only useful when they are both easy to use, and helpful for those individuals who will be
using them. The functional possibilities for access methods are many and varied. Certain preexisting notions of
optimal usability and interface layout certainly exist, but it is often difficult to discern exactly what elements in a
given implementation make one user interface inherently more usable over another. Additionally, a virtually
limitless number of factors may contribute to the success of a case study library such as access demographics, data
presentation, available tools, relevance of included material, quality of content, and so on. It is fairly self-evident
that even subtle changes to the user interface of a software program can have drastic effects on its subsequent
usability, for better or for worse. Alterations in the structure, length, emphasis, and many other components of an
individual use case can produce similar effects, and many parallels can be drawn between the two. Since we wish to
explore the basic components of a use case library that can shape a user's experience, both are equally relevant to the
goals of this study, and must be equally assessed to gain a comprehensive understanding of the system.
3.2 Basis
At this point, what would be the best method for gathering information that would provide useful insight into the
effective design and implementation of a use case library? Despite the fact that such skills may only be mastered
through practical experience, virtually all people tend to learn some strategies to improve their design approaches
through experience and feedback [6]. Assuming for a moment that we know nothing about designing a successful
case study resource, we can speculate that the relevant points of interest will most likely be reflected in the many
interactions between a resource and its users. In this way, these points may be revealed through the careful
observation of similar resources that are already successfully in use. A study of this nature would allow us to discern
the most influential and effective aspects of these libraries and incorporate them into our own work. In essence, this
study could be considered to be a momentary step back for a more complete system assessment (to "see the forest
for the trees", so to speak) before proceeding ahead. It is our expectation that the information gathered from this
study will eventually be used to support the development of an enhanced version of the original system that
produces a more desirable result.
Our experiments involved nine (9) participants, seven (7) male and two (2) female. They were not in any inclusion
or exclusion criteria; however, the intent was to seek participants who were novice designers in computer science.
This was to minimize the effects of participants with past specialization in interface design or usability engineering
which might skew results. Additionally, since the goal of developing this software was to help this audience design
better systems, it was likewise in our best interests to select our participants from this population. Each participant
was randomly assigned a case study library, then participated in a three-part experiment. The first part involved a
series of tasks written on separate sheets of paper, to be completed in order. These tasks, involving a specific use
case library, can be found below in Table 1 and were designed for probing specific aspects of the user's interaction
with the system.
Attempt to find the Which of the following According to the article, According to the article, how
answers to the following symptoms is not exhibited what island has stopped the many steps are there in
questions (within each by the patient in the initial hunt for seals? identifying race
case study): diagnosis? [thin appearance; discrimination?
impaired vision; increased
blinking; symmetrical
reflexes]
Attempt to find the "Chemistry" data What were two factors that What was the officially cited
Attempt to find the "Chemistry" data What were two factors that What was the officially cited
answers to the following measurements in this case contributed to the decline of reason for the termination of
questions (within each include which of the rug trades in Iran in the late the applicant's membership?
case study): following? [phosphorus, 1980s?
barium, fluorine, glucose,
potassium]
Identify the number of sections in this case that have appeared in other case studies explored thus far.
From a case study database's Homepage, identify the closest means of providing user feedback. This could be an e-mail
address or a scripted electronic form, for example. If none are available (or visible), navigate the site until one is found.
Table 1: Examples of Questions
Questions could range from something as simple as locating the front page of a case study library from a website's
main page to finding specific pieces of information within a specific case study. The emphasis should be made that
the primary goal was not to test the performance of the user, nor was it to specifically assess the correctness or
incorrectness of their answers, but to develop familiarity with a given case library. Once a participant completed all
of their tasks, they were asked to fill out a short likert-style questionnaire requesting feedback on various aspects of
the system. These questions, involving issues such as system usability and overall user satisfaction, were deemed
appropriate at this time since the participant would now be able to make informed responses based on their recent
experiences. Finally, an additional number of questions were administered verbally, expanding considerably upon
those in part two. These questions informally solicited opinions such as the sort of activities one might use a given
library for, or how certain characteristics of the case studies influenced the participant's impression of the library.
We generally considered this last portion of the experiment to provide the most useful information since we could
assess the participant's opinions of the system directly.
Interestingly enough, it has been observed that many participants tend to reserve the use of search functions as a last
resort. Rather, they initially prefer to scan the presented information visually, and in sequential order from top to
bottom until they decide that this method might be unfeasible. It may be that instead of trying to locate the specific
answer for a question, the participant is trying to assemble a contextual background for the question that might aid
them in locating the answer that much more quickly.
Another common complaint indicated that the overall hierarchy of information for a given case study library was
either so high-level as to be useless, or so dense as to be unusable. Users, apparently, are not adverse to different
views of information - that is, the ability to "drill-down" a particular piece of content to simplify or expand the data
to the desired level of complexity as necessary. At this point this highlights, perhaps, a way to apply Schneiderman's
mantra of “overview first, zoom and filter, and then details on demand” [7].
Most participants revealed that more often than not, the mere inclusion of multimedia elements (such as pictures)
seemed to have a marked effect on the user's reported usability of a case, regardless of whether or not the pictures
actually appeared to have any sort of tangible effect on the structure or flow of the case study. Interestingly, when
asked, participants said that the use of pictures allowed them to understand what the case was about. In a sense, it
allowed them to gain a high level understanding of what the case was about. Similarly, it made the participants feel
that the authors of the case study library took more time in crafting the case study. The participants then went on to
correlate this reaction to the case being better than others.
Other user reactions indicated, with surprising consistency, that they were somewhat skeptical of the legitimacy of a
piece of information retrieved from such a library because of the apparent lack of citations (the Employment Case
Law being the exception). The general sentiment was that there was only a limited trust of the library's information,
and that one could only safely use it in conjunction with a third-party search to confirm its validity; that is, they
wouldn't feel comfortable using it as a standalone resource, although some users decided to trust the given
information based on the subject matter alone.
5. CONCLUSION
This paper has presented a brief introduction to the appeal and usefulness of case study libraries in academic
settings, as well as some motivations and strategies for our heuristics gathering activities. We have also reviewed
some of the preliminary results from the experiments currently being collected for this study. The Case Law library
appears to have garnered the most positive impressions from participants by far, thanks to its concise narrative,
consistent layout and plain language. The "glance-ability" of this library was also improved by the subdivision of the
main body of text into a series of easily accessible sections. The pathology library elicited the most neutral responses
from experimental participants. It's use cases generally possessed the brevity shared with the Case Law library, but
were hampered by an abundance of technical jargon, with a similar effect on participant performance. Finally,
despite the extensive use of multimedia, the TED library cases proved to be less than popular to participants due to
their incredible length and extensive pretext, marginal search capabilities for a database consisting of well over five
hundred cases, and perpetually shifting page layouts, formats, colors, and virtually everything else from case to case.
6. FUTURE WORKS
By analyzing this information, we wish to build a set of heuristics that will be indispensable in our future studies of
educational resources, such as case study libraries, design and usability. Specific topics pending full analysis for
future research may include the notion of "browsing vs. searching", the use of multiple data hierarchies to organize
and streamline data access, the role of information density on a user's usability experience, and the role of trust as it
applies to use case resources. Eventually, we wish to augment the Use Case System to a state where its functionality
will be equivalent to the results found in the original study, and will serve as a useful learning tool for students in
their study of the principles of design. From there, we wish to continue building on the successful features of the
UCS, continuing to address issues that the original study may have overlooked.
7. REFERENCES
[1] Ardito, C., M. D. Marsico, et al. (2004). Usability of E-learning Tools. Proceedings of the Working
Conference on Advanced visual interfaces. Gallipoli, Italy, ACM Press: 80-84.
[2] Berry, B., L. Hobby, et al. (2006). Exploring Benefits of Visualization for Case Studies. World Conference
on Educational Multimedia, Hypermedia and Telecommunications 2006. P. Kommers and G. Richards.
Orlando, FL, USA, AACE: 485-492.
[3] Carroll, J. M. and M. B. Rosson (2005). "A case library for teaching usability engineering: Design
rationale, development, and classroom experience." Journal of Educational Resources in Computing 5(1):
3.
[4] Gick, M. L. and K. J. Holyyoak (1980). "Analogical problem solving." Cognitive Psychology 12: 306-355.
[5] Harrison, S., M. Back, et al. (2006). "It's Just a Method!": A Pedagogical Experiment in Interdisciplinary
Design. Proceedings of the 6th ACM conference on Designing Interactive Systems. University Park, PA,
USA, ACM Press: 261-270.
[6] Lewis, T. L., M. B. Rosson, et al. (2004). What Do The Experts Say?: Teaching Introductory Design from
an Expert's Perspective. Proceedings of the 35th SIGCSE Technical Symposium on Computer Science
Education. Norfolk, Virginia, USA, ACM Press: 296-300.
[7] Schneiderman, B. (1998). Designing the User Interface: Strategies for Effective Human-Computer
Interaction. Reading, MA, Addison Wesley.
[9] Wright, T., J. Noble, et al. (2005). Using a System of Tutorials and Groups to Increase Feedback and Teach
User Interface Design. Proceedings of the 7th Australasian Conference on Computing Education.
Newcastle, New South Wales, Australia. 42: 187-192.