Journal of Information Science

http://jis.sagepub.com

Approaches to User-Based Studies in Information Seeking and Retrieval: A Sheffield Perspective

Micheline Beaulieu
Journal of Information Science 2003; 29; 239
DOI: 10.1177/01655515030294002

The online version of this article can be found at:

http://jis.sagepub.com/cgi/content/abstract/29/4/239

Published by:

http://www.sagepublications.com

On behalf of:

Chartered Institute of Library and Information Professionals

Additional services and information for Journal of Information Science can be found at:

Email Alerts: http://jis.sagepub.com/cgi/alerts

Subscriptions: http://jis.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations (this article cites 28 articles hosted on the

SAGE Journals Online and HighWire Press platforms):
http://jis.sagepub.com/cgi/content/refs/29/4/239

Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008

© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies
in information seeking and
retrieval: a Sheffield perspective

Micheline Beaulieu reviews in the Annual Review of Information Science

and Technology which have appeared at regular
Department of Information Studies, University of Sheffield,
Sheffield, UK
intervals over the past four decades. Although the
importance of taking account of users of information
and information services either directly or indirectly
Abstract. has long been recognized, how to do so and how to
evaluate the impact of such studies remain problematic.
The paper traces the development of user-based research in
the Department of Information Studies at the University of
The aim of this paper is to trace the development of
Sheffield over the past 40 years, and highlights and assesses user-based research in the Department of Information
the different research questions addressed and main Studies at the University of Sheffield over the past 40
characteristics of projects undertaken. Four phases of user- years. This longitudinal review seeks to highlight and
based studies relating to information seeking and retrieval are assess the different research questions addressed and
identified including: preliminary studies on library users; the main characteristics of the projects undertaken
empirical studies on information user groups by the Centre for over that period of time. Four phases of research have
research on User Studies; the development of a theoretical been identified and are discussed in the following
framework for user studies; and the integration of sections.
information seeking and retrieval research. The review In the first phase, small-scale studies were primarily
concludes that user-based studies have evolved linking
concerned with users of libraries. The second phase
together studies on information use and provision,
information seeking behaviour and the design and
encompassed the work of the Centre for Research on
evaluation of actual retrieval systems. The unifying element User Studies, which addressed methodological issues
has been the emphasis placed on methodological issues. More and the need to focus on information needs and
recent work is transcending user-orientated and system information seeking more generally. This period was
perspectives giving way to a more integrated approach and followed by a third phase which concentrated on
emergent interaction paradigm for user studies. building a theoretical framework of models of informa-
tion behaviour to guide empirical studies. The fourth
current phase of research is characterized by a more
integrated approach to information seeking and retrie-
val research.
1. Introduction

User studies have constituted a core area of research in

library and information studies as manifested by the 2. First phase: studies on library users 1963–
1975
Correspondence to: Department of Information Studies,
University of Sheffield, Sheffield, UK. User studies have underpinned research at Sheffield
E-mail: m.beaulieu@sheffield.ac.uk since the Department of Information Studies was first

Journal of Information Science, 29 (4)Downloaded

2003, pp. 239–248 # CILIP
from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
239
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies
established in 1963. From the outset Masters student
dissertations explored different aspects of user infor-
mation-seeking behaviour principally in the context of
libraries. Studies in the first decade or so covered a
wide range of topics including the use of stock,
catalogue and online resources in both academic and
public libraries, as well as the communication pat-
terns, information needs and information-seeking
behaviour of specific library user groups [1] (e.g.
scientists, economists, ethnic groups, children). In
addition to these small-scale investigations, two major
projects were undertaken during this early period: one
on the role of foreign language materials in academic
research activity [2], and another on local library
cooperation issues [3]. Although the research questions
being addressed related to the performance and
effectiveness of library services, the approach included
a substantial user component. These studies broke new
ground in terms of the scale and combination of data
collection methods used and in-depth analysis of the
data derived from reliable samples of users.
User studies generally dominated library and infor-
mation studies research throughout the 1960s and
1970s, but their usefulness was repeatedly challenged
because the results were deemed to be neither
comparable nor applicable. This was largely due to
the lack of standard research methods used [4]. How-
ever, the initial larger-scale and more in-depth inves-
tigations undertaken at Sheffield were to lead the way
in the development of more systematic methods for
consulting users and indeed did inform on ways of
improving the information services under study.
3. Second phase: empirical studies on
information user groups 1976–1988
The need to develop more robust methods, in order to
provide results that were more directly applicable to
professional practice, was one of the factors which led
to the establishment of the Centre for Research on User
Studies at Sheffield (CRUS) in 1975. However it aimed
to achieve more than that, and the direction that the
research on user studies was to take was expressed as
follows:
We need to know more about the context in which
information needs arise: this may involve studies of
attitudes, temperament and total life situations so that
information-seeking behaviour may be better understood
. . . Perhaps the first goal of research in this field should be
to achieve convincing demonstrations of the necessity [5].
240
Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Thus there appears to be in this second phase of user
studies the recognition of the inherent value of
systematically studying users of libraries and informa-
tion as an independent primary focus in a broader
context than the immediate library use or service.
Between 1976 and 1988, CRUS was to play a leading
role as a national research centre funded by British
Library Research and Development Department
(BLR&DD) in the development of user studies research.
Firstly, CRUS conducted a series of large-scale empiri-
cal projects, which explored the information needs and
information-seeking behaviour of a wide range of
different information user groups from an information-
rather than a library-based perspective. This included
subject domains such as the humanities [6, 7], primary
education [8, 9], as well as community [10, 11],
medical [12] and business information services [13,
14]. Secondly, the projects adopted a combination of
primary quantitative and qualitative data collection
methods ranging from postal questionnaires, semi-
structured interviews and direct observation to the
analysis of complementary secondary sources such as
reference enquiries, inter-library loan records and
demographic data.
These investigations produced valuable outcomes
such as Stone’s seminal paper on the characteristics of
information seeking of humanities scholars [7] and
Beal’s standard work on community profiling [10].
Perhaps the most noteworthy comprehensive user
study to be conducted during this period was Project
INISS [15]. This was a five-year multi-phased piece of
action research, which aimed to establish guidelines
for the development of effective information services
for social services department in local authorities.
Such an in-depth analysis of information problems and
information-seeking behaviour within an organiza-
tional setting, which led to a number of practical
outcomes for service improvements, has not been
undertaken as far as we know anywhere else since.
3.1. Main contributions of CRUS
Generalization of findings. The studies undertaken
by CRUS came some way towards addressing the
recurring criticism of user studies in terms of the lack
of generalizability and applicability of results.
Findings demonstrated that information seekers in
different organizational or work contexts were
dependent on both informal and formal channels of
information sources and in some cases could operate
entirely outside the realm of libraries and information
professionals. Whilst different groups such as
Journal of Information Science, 29 (4) 2003, pp. 239–248 # CILIP
 M. BEAULIEU

humanities scholars were very dependent on libraries agenda for user studies which began to differentiate
to support their information needs, others such as between user studies that would constitute
social workers were not. The low level of information fundamental basic research in information science
skills encountered was another recurrent issue. The and those which could be defined as more practice-
coverage of academic, public and commercial based applied research. What is still little understood
exemplars thus provided a rich body of evidence to and remains open to debate is the relationship or
allow for some level of comparisons between different dependency between the two.
individual and group information behaviour. The
emergence of these general findings, by focussing on Methodological developments. In addition to the
information needs and seeking behaviour, thus firmly significance of its findings on user studies, CRUS also
established the validity of studying information effectively addressed the methodological issues
seekers and users in their own right. regarding the robustness of data collection methods
It is also worth noting that some of the research in by drawing on sound research practice from the social
the different information sectors undertaken during sciences. In addition to applying appropriate sampling
this period laid the foundation for the Department’s techniques for quantitative surveys, there was a strong
future research agenda and are continuing to play an emphasis on the development of instruments for the
important part in current activities: for example, the collection and analysis of qualitative data through
work on educational informatics, information literacy, semi-interviews and direct observation methods. As
reading, public library management, community infor- well as the innovative techniques developed for
mation and health information. eliciting information from users, other sources of data
were also analysed to corroborate primary evidence.
Applicability of user studies. In gaining a better CRUS was instrumental in improving the general
understanding of these complex behaviours and the standard of user studies research not only through its
various contributory factors, the challenge for the own work but also through its seminars and
information researcher or practitioner was then to publication guides on research methodology.
determine how to address or accommodate these
different approaches or situations. In some instances
it was possible to move from the problem analysis to 4. Third phase: developing a theoretical
actual solutions as in the INISS project [15], whereby
information resources would be provided within the
framework for user studies, 1989–1999
immediate workplace and not from a centralized
With the funding for CRUS coming to an end in 1988,
library service. In other instances, such as the study
research on user studies followed new directions,
of business information interests of metal production
which were to build on the body of knowledge and
firms, which identified barriers to information access,
experience that had accumulated over the previous
findings led to further research on acquisition
period. The research over the next 10 years was to raise
procedures and use of public sector business
different research questions and address more funda-
libraries [13]. By contrast other investigations, such
mental issues regarding the development of an appro-
as for example the survey of humanities scholars [7],
priate paradigm for conducting user-based studies. The
were not intended to lead to solutions as such, but
research activities of Wilson and Ellis, on the one
rather to provide useful insights into less tangible
hand, focussed on generating models of information
issues about information needs and information-
seeking and that of Ford, on the other hand, was
seeking behaviour which might require longer-term
concerned with the exploration of cognitive aspects of
monitoring. It can be argued that the applicability of
that behaviour.
research does depend on different factors including the
research questions being addressed, as well as the
different internal and external barriers or priorities
4.1. Building models of information behaviour
within an organizational setting for the implemen-
tation of recommendations. The assumption or Wilson’s model 1981. Whilst the numerous empirical
expectation that user studies must be applicable to studies at Sheffield and elsewhere revealed the
professional practice was probably too simplistic and interplay of different strategies and complex
somewhat ill-founded. One of the major contributions processes in information-seeking behaviour, a key
of CRUS was that it initiated a different research paper was published by Wilson in 1981 which

Journal of Information Science, 29 (4)Downloaded

2003, pp. 239–248 # CILIP
from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
241
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies
attempted to define and differentiate the concept of
information needs from that of information
seeking [16]. He observed that user studies to date
had primarily paid attention to how user sought
information but had ignored why they did so and
what use they made of the information found. He went
on to argue that a holistic view of the information user
was required which explored the role of information in
a user’s everyday life and work. To achieve this goal he
states ‘to uncover the determining factors of behaviour
we must do so by first undertaking in-depth studies of
well-defined categories of persons, developing
explanatory concepts and then testing these concepts
in related but different settings’. The model put
forward by Wilson presented information behaviour
in the context of an individual’s environment, social
role and physiological, affective and cognitive needs,
and regarded information-seeking behaviour as being
dependent or overcoming corresponding personal,
role-related and environmental barriers.
The model served to highlight both the behavioural
and organizational contexts of information seeking and
to take account of the factors influencing them,
including all types of information resources and
information transfer mechanisms. Wilson later
described the model as a macro-model, which could
suggest testable hypotheses on information
behaviour [17]. Although the model was generally
informed by previous empirical studies, it is an a-
priori conceptual or theoretical model that identifies
the essential elements (i.e. information need, informa-
tion seeking, information exchange and use). Wilson’s
general model thus provided a useful initial mapping
of the research terrain for subsequent empirical
exploration.
Ellis’s model 1989. Ellis’s model by contrast was
directly derived from empirical research and focussed
on the detailed processes of information behaviour to
detect common information-seeking patterns [18]. He
identified six characteristics or searching features
including starting (initial search activity), chaining
(following up citations), browsing (semi-directed or
semi-structured searching), differentiating (filtering
sources or content), monitoring (searching to keep up
to date), extracting (identifying and selecting relevant
material), verifying (checking for accuracy or
omissions) and ending (tying up loose ends).
Although the model seemingly encompassed a staged
process, other than starting and ending, no set
sequence for the different stages was implied and not
all individual features would necessarily be present in
242
Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
actual information seeking. As suggested by Wilson,
the explanatory concepts were then further tested in
follow-up studies of academic researchers in the
physical sciences and of engineers and research
scientists in an industrial environment [19, 20].
Minor variations in information-seeking patterns
were detected, such as the prominence of a search
strategy at a particular stage of research, but no
fundamental differences in information-seeking
behaviour were revealed. This confirmed the
robustness of the model in identifying generic
features of information-seeking behaviour at least for
these categories of information searchers.
In addition to the actual model generated, Ellis’s
work was also significant because of the methodologi-
cal approach adopted to derive the model. Influenced
by Wilson’s 1981 paper advocating the use of qualita-
tive research to develop concepts for user studies and
by the seminal ARIST review by Dervin and Nilan [21]
on the need for a paradigm shift for conducting
empirical research, Ellis turned to Glaser and
Strauss’s [22] grounded theory approach, a well
established inductive method for generating theories
and models in the social sciences. The aim was to
derive a more accurate model of information-seeking
patterns from empirical data. The application of the
approach is clearly described by Ellis in a paper on the
application of grounded theory [23]. The key was to
use informal semi-structured interviews whereby the
interviewer follows an interview guide to cover all the
issues to be addressed but has the freedom to explore
and probe those issues as appropriate with different
interviewees. The approach was detailed as follows:
The intention was for the researchers to talk about their
work and the sorts of things they did that might be
understood as having an information component, not for
them to end up talking rather artificially about their
‘information-seeking behaviour’ or some other such
abstraction or to talk in isolation about their use of the
library [23, p. 475].
The method of analysis and coding of the transcripts is
also vital. A constant comparative approach is used
consisting of four stages: recording incidents of each
category; determining properties of categories; differ-
entiating between core and subsidiary categories; and
finally deriving a coherent model based on the overall
relationships between the categories. Grounded theory
has been used in a number of other Sheffield doctoral
projects investigating the communication and use of
information of different groups in different environ-
ments including farmers [24], managers [25],
Journal of Information Science, 29 (4) 2003, pp. 239–248 # CILIP
 M. BEAULIEU

dentists [26] and health workers [27]. The method has behaviour in the outer circle, general more passive
proven to be a reliable and fruitful way of handling information-seeking behaviour in the middle circle
data analysis and has contributed to establishing the and more specific or active information searching in
validity of qualitative methods for user-based studies. the inner circle.
In an attempt to integrate the various models
Wilson’s revised models – 1994, 1996, 1999. Wilson’s previously proposed, Wilson finally offers a unified
1981 theoretical general model on information problem-solving model of the information seeking and
behaviour was revised in 1994 to directly incorporate searching process. Based on the notion of uncertainty
Ellis’s more specific empirical model on information and problem resolution, information-seeking beha-
seeking [28]. This is a good example of how empirical viour is viewed as a goal-directed process that
models can inform and also accommodate theoretical progresses through different stages to arrive at a
models and vice versa. The original model was then solution. Four stages are identified in uncertainty
further elaborated and refined in a new version in resolution: problem identification, problem definition,
1996, which drew on other information-related problem resolution and possible solution statement.
theories in the social sciences, e.g. communication, Whilst the successive stages should lead to the
decision-making and psychology [29]. Starting with resolution of more and more uncertainty, failure to
the same basic element of context of information do so would result in feeding back to the previous stage
needs, the barriers are now described as intervening and further iterations between stages. Wilson also
variables which are explicitly specified, e.g. points out that the notion of partial relevance supports
psychological, demographic, environmental, source the concept of uncertainty resolution.
characteristics, and serve to emphasize both their
supportive or inhibiting roles in information seeking.
4.2. Cognitive aspects of information behaviour
Information-seeking behaviour is also further
differentiated into different types to include, passive A second line of enquiry in this third phase of user-
and ongoing searching in addition to the category of based studies focused on cognitive aspects and
active searching, which now replaces Ellis’s more individual user characteristics of information beha-
detailed model. Lastly, two other more dynamic viour. The relationship between information needs,
components are introduced to complete the model, information and study skills and effective learning in
namely activating mechanisms, e.g. stress/coping an educational setting was explored in several papers
theory, social learning theory and information and preliminary studies by Ford in the early
processing and use. These constitute the underlying 1980s [34–38]. Ford was directly influenced by the
drivers, which motivate or inhibit searching activity emerging theories in cognitive science as they related
and the feedback loop relating outcomes to meeting to information processing and knowledge acquisition
information needs. more generally as well as their applicability to human
This more complete model is still described by interaction with actual information systems. The role
Wilson as a macro-behaviour model, and he claims in a and relationship between the physical Cranfield
later comprehensive review of information behaviour system-based paradigm and the more recent user-based
models that it can provide a richer source of hypoth- cognitive paradigm in information retrieval research
eses and potential for setting a research agenda than was the subject of a discussion paper by Ellis [39]. In
the previous versions [17]. In exploring the relation- this paper he argues the case for the development of
ship between different types of models, he reasserts the techniques for modelling the cognitive world of users
important distinction between information-seeking as part of the retrieval interaction.
behaviour models in the context of the broader Building on early studies, Ford and his colleagues
information environment and models of information conducted three empirical studies that explored the
searching from the perspective of an information correlation between cognitive and learning styles and
retrieval (IR) system, e.g. models of the IR interaction information access and searching behaviour [40–42].
process proposed by Belkin [30], Ingwersen [31], Based on constructs developed by educational psy-
Saracevic [32] and Spink [33]. He thus arrives at a chologists [43, 44], the studies found that searchers
classification of three different types of models forming with a global comprehension-learning field-indepen-
a single nested concentric hierarchical model repre- dent style adopted a broad more holistic strategy to
senting three related research areas. The representation searching. By contrast those with an operation-learning
incorporates the overall environment of information field-dependent and procedural style adopted a serial-

Journal of Information Science, 29 (4)Downloaded

2003, pp. 239–248 # CILIP
from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
243
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies
ist and more analytical strategy to searching. Global
comprehension learners used a greater number of
search tactics and tended to retrieve more references,
i.e. higher recall, than analytic operational learners.
However the higher precision results achieved by the
latter led to greater satisfaction. These findings high-
light the limitation in measuring retrieval performance
in terms of recall and precision, if considered solely
from a system perspective as opposed to a user
perspective.
Another Internet-based study revealed strong gender
differences in searching behaviour [45]. Poor retrieval
and negative perceptions of Internet searching were
associated with females who had feelings of not being
in control, of not being able to avoid irrelevant material
and of finding the Internet too unstructured. By
contrast browsing was much more appealing to men,
whereby they were happy to plough through material
and pursue personal interests without any specific
purpose or goal in mind. Browsing and a broad
approach to Internet exploration also correlated with
a holistic cognitive style, whereas keyword searching
and a more structured approach was associated with
an analytic cognitive style.
4.3. Role and significance of a research framework for
user studies
The framework for user studies research developed in
this third phase serves different purposes. Firstly, it
not only differentiates more clearly between different
contexts, purposes and types of user-based studies but
also provides a continuum for conducting investiga-
tions into different levels of information activity and
for taking into account different levels of abstraction.
Secondly, a key characteristic of such information
behaviour frameworks (the layered model proposed by
Saracevic [32] and Ingwersen [31] and the onion model
developed by Rasmussen et al. [46] are other exam-
ples), is the interdependency between the different
components or layers whereby the interaction between
the component parts can become the focus of the
research.
Thirdly, the building of models is a methodological
approach widely adopted in the social sciences to help
the understanding of complex social and behavioural
phenomena. The approach adopted here draws on
other areas of social sciences both in terms of the use of
theoretical underpinnings and constructs and in the
application of analytical methods for interpreting
observed behaviour and techniques for testing con-
tributory factors. Another important feature is the
244
Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
cyclic process of the model building. Initially the
theoretical model is partly informed by empirical
evidence. The proposed model is then tested by more
empirical studies, which in turn can further enrich the
original model and lead to the generation of other
hypotheses to be tested.
Finally and more importantly, the motivation in
developing a comprehensive framework for informa-
tion behaviour research was to provide an alternative
paradigm to that adopted in information retrieval
research, which would offer more insight on how to
improve and build better systems. Although some
behavioural features have been observed in initial
studies, it has proven to be much more difficult to
identify contributory factors or explanations for these
behavioural characteristics or to determine how they
can inform actual systems design. Thus the next phase
of user studies was to concentrate more specifically on
information behaviour in the context of computerized
information systems.
5. Fourth phase: bridging the gap between
information seeking and retrieval research,
2000 and beyond
Having established a theoretical framework over the
past decade, more recent and current work in
information behaviour at Sheffield has moved on to
encompass two related lines of enquiry. The first is
concerned with undertaking empirical studies of
searching behaviour in different online environments
to provide more robust user models to inform the
design of more effective systems. The second is
concerned with developing ways of involving users
in the design and evaluation of advanced experimen-
tal IR systems. This dual approach reflects the
argument put forward by Ford [47] for the need for
greater integration between user and system-orien-
tated research both in terms of methodological
approach (qualitative and quantitative) and research
focus (user and system).
5.1. Empirical studies of interactive searching
behaviour
Unlike a previous study which looked at the char-
acteristics of users searching the World Wide Web, a
second web-based investigation led by Ford [48]
sought to correlate search strategies with retrieval
performance or relevance scores, and in particular to
Journal of Information Science, 29 (4) 2003, pp. 239–248 # CILIP
 M. BEAULIEU

compare the effectiveness of Boolean vs best-match concept of uncertainty was recognized by searchers for
retrieval vs a combination of the two approaches. reporting progress in their research and that uncer-
Subjects were assigned both a fact-based finding task tainty is related to the problem stage and domain
and a research-based searching task and multiple knowledge [51]. Secondly, successive mediated
regression and factor analyses were applied to the searches often involved a refinement over previous
logged queries and searches to determine correlations searches but precision did not necessarily increase
between variables and cause and effect relationships. and the number of items judged as partially relevant
The results revealed that Boolean retrieval performed was substantially reduced [52]. Thirdly, searcher
worse than best match or a combination of the two and cognitive styles were correlated with their perceptions
that there was no evidence that the query terms in of aspects of their problem solving and information-
themselves were a contributory factor. seeking behaviour and with those of the search
Assigned searching tasks or topics were also used in intermediaries carrying out the searches and confirmed
another experimental study which examined searching previous findings regarding global field-dependent
behaviour in the context of the TREC-8 interactive holist researchers and field-independent serialist
track and considered the correlation between three researchers [53]. Lastly, interaction between research-
variables: searcher, system and task characteristics [49]. ers and search intermediaries in the pre-search inter-
Two versions of the Okapi best-match experimental view and during online searching was deemed helpful
retrieval system were used, one with relevance feed- in identifying search terms, and influenced users’
back and one without. Both qualitative and quantita- perceptions of the information problem, personal
tive data was collected using a combination of methods knowledge and relevance judgements [54].
including observations, verbal protocols, transaction Although information seeking and searching and
logs, questionnaires and structured post-search inter- information retrieval research in practice have been the
views. Search analysis revealed that searching beha- concerns of separate research communities, the above
viour was largely dependent on topic characteristics. studies clearly demonstrate that these areas address
The study differentiated between complex and simple interdependent issues. Beaulieu argues that interaction
topics and associated searching according to the level is the common element which links them together [55].
of search uncertainty and searcher engagement with She identifies three aspects of interaction. Task
the different topics. Overall best match ranking led to interaction relates to different interconnected activities
high precision searches and those that included or tasks, i.e. problem definition, query formulation etc.,
relevance feedback were marginally but not signifi- which can be undertaken in a sequential or iterative
cantly more effective. fashion. Interaction can be viewed as task sharing,
A third extensive and complex study led by Wilson whereby both user and system play active and/or
aimed to explore interactive searching and changing passive roles in performing different activities in the
information-seeking behaviour in the context of search process. Interaction can also be defined in terms
mediated online searching. Building on his Problem of the user interface and as a discourse or user dialogue
Solving Model discussed at the end of Section 4.1, the that enables task sharing.
study also draws on concepts derived from both IR and By identifying cognitive and affective factors, com-
information behaviour research to establish a theore- mon characteristics across different experimental set-
tical framework for the empirical investigation. Thus tings as well as correlations between different
the stated objective is to ‘characterize the progressive variables, these studies on searching behaviour are
changes that occur in users’ information seeking and starting to reveal new insights into the nature of
searching processes, including: user situational con- interaction. However much more research is required
text; user information problem; uncertainty reduction; to develop our general understanding of information
cognitive and affective states of users over successive behaviours and interaction in order to fully inform on
searches, and consequently in their queries and system design. In addition the development of appro-
cognitive styles’ [50]. priate experimental designs for interactive empirical
The results of the study are reported in a five-part studies is also of particular importance. For example
series of papers covering the theoretical framework and greater consideration needs to be given to the relative
research design, uncertainty and its correlates, succes- merits of laboratory vs operational settings, simulated
sive searching, cognitive styles in information seeking vs real search tasks, and how to combine quantitative
and user-intermediary interaction. The main findings and qualitative methods for data collection and
indicate firstly that the problem solving model and the analysis.

Journal of Information Science, 29 (4)Downloaded

2003, pp. 239–248 # CILIP
from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
245
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies

5.2. User-centred design and evaluation of advanced The latter applied a qualitative approach and took
interactive information retrieval systems account of the completeness and accuracy of the
summaries as assessed by users. The diagnostic
In parallel to the more in-depth observational studies
analysis on the reliability of the summaries provided
of interactive searching on current operational sys-
some indication, on the one hand, of what level of
tems, work is also being carried out into the design of
accuracy could be tolerated by users, and on the other
advanced retrieval systems which directly take
hand, what extraction rules should be refined to
account of users. User participation in the design
improve performance.
process is achieved in a series of steps. In the first place
The second ongoing project is developing a cross-
field studies are carried out to establish how users
language information retrieval system (CLIR) with
undertake particular information-seeking or search
participation from journalists and translators from
tasks in actual work contexts and in using current
two media organizations [57]. The system draws on
technology and services. The main approach is to draw
dictionary resources to translate queries from one
up work task scenarios through a combination of data
language to another using English as a pivot language
collection methods to assist in defining user require-
to retrieve news items from databases in multiple
ments for the initial specification of the user interface
languages. A field study indicated that potential users
for the proposed new system. The work task scenarios
of the system are polyglots who may combine different
are then transformed into discreet screen mock-ups,
languages to generate queries and wish to search
which are assessed by the users before the initial
different databases in parallel. Initial usability tests
system prototype implementation.
have compared the acceptability and effectiveness of
The main challenge in designing the user interface is
displaying translated query terms prior to or after the
to represent the underlying functionality of the system
system has undertaken a search on a translated query
in such a way as to enable the user to undertake what is
as well as different ways of displaying ranked retrieved
essentially a complex cognitive task in an uncompli-
sets in multiple languages. Clearly the successful
cated intuitive manner. Hence the goal of the initial
design of such complex interactive systems could not
prototype is to explore and start defining the system/
be achieved without the close involvement of actual
user interaction for the whole search process. This is
users.
an iterative process which aims to resolve the conflict
or tension between functionality and usability, effec-
tiveness and efficiency. The gradual refinement of the
6. Conclusions
user interface design and interaction is thus achieved
through successive formative user evaluations or
The approach to user-based studies in the Department
usability tests, usually involving a small number of
of Information Studies at Sheffield has evolved to form
users and focusing on specific aspects of the design.
a continuum linking together studies on information
The process may lead to one or more completed
use and provision, information-seeking behaviour and
prototypes, which then undergo a summative or
the design and evaluation of actual retrieval systems.
comprehensive evaluation in a more formal user-
The principal unifying element in the development of
experiment based on a larger sample of users.
the research agenda over this long period of time has
This approach to user-centred design has recently
been the emphasis placed on methodological issues.
been adopted in two major collaborative projects to
This includes: the development of a variety of both
design advanced information systems for specific user
qualitative and quantitative methods and combined
groups. The first was concerned with the design of a
instruments for collecting and eliciting information
current awareness service, which would alert users in
from users; the importation of theoretical concepts and
the pharmaceutical industry of drug approval deci-
constructs from other areas in the Social Sciences into
sions following clinical trials [56]. The system applied
the information discipline; and the interplay between
information extraction (IE) techniques to produce
theory and evidence in the theoretical underpinning of
summaries of news items relating to drug approval
empirical investigations.
events and track them chronologically and geographi-
The primary motivation for undertaking user studies
cally. The prototype was evaluated from both a system
research was to focus on user information needs and
and user perspective. The former measured the effec-
user seeking behaviour as an independent area of
tiveness of the information extraction methods using
research in its own right. This was largely in response
standard quantitative scores of precision and recall.
to the inadequacy and limitations of the systems

246 Journal of Information Science, 29 (4) 2003, pp. 239–248 # CILIP

Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
 M. BEAULIEU

approach of information retrieval research. However it Manufacturing Firms (CRUS, Sheffield University, Shef-
would appear that more recent work over the past five field, 1986).
years or so has started to transcend the respective user- [15] T.D. Wilson and D.R. Streatfield, You can Observe a Lot
orientated and system approaches. As more aspects of A Study of Information Use in Local Authority Social
Services Departments Conducted by Project INISS, 2
information seeking and richer information tasks are
vols (PGSLIS, Sheffield University, Sheffield, 1980).
being developed in online environments, the bound- [16] T.D. Wilson, On user studies and information needs,
aries between information seeking, searching and Journal of Documentation 37(1) (1981) 3–15.
retrieval are breaking down. This is thus giving way [17] T.D. Wilson, Models in information behaviour research,
to a more integrated approach and emergent interac- Journal of Documentation 55(3) (1999) 249–270.
tion paradigm for user studies. [18] D. Ellis, A behavioural approach to information retrieval
system design, Journal of Documentation 45(3) (1989)
171–212.
[19] D. Ellis, D. Cox and K. Hall, A comparison of the
References information seeking patterns of researchers in the
physical and social sciences, Journal of Documentation
[1] N. Roberts and T.D. Wilson, The development of user 49(4) (1993) 356–369.
studies at Sheffield University, 1963–88, Journal of [20] D. Ellis and M. Haugan, Modelling the information
Librarianship 20(4) 1988 270–290. seeking patterns of engineers and research scientists in
[2] W.J. Hutchins, L.J. Pargeter and W.L. Saunders, The an industrial environment, Journal of Documentation
Language Barrier: a Study in Depth of the Place of 53(4) (1997) 384–403.
Foreign Language Materials in the Research Activity of [21] B. Dervin and M. Nilan, Information needs and uses,
an Academic Community (PGSLIS, Sheffield University, Annual Review of Information Science and Technology
Sheffield, 1971). 21 (1986) 3–33.
[3] T.D. Wilson and W.A.J. Masterson, Local Library [22] B.G. Glazer and A.L. Strauss, The Discovery of
Cooperation: Final Report on a Project Funded by the Grounded Theory: Strategies for Qualitative Research
Department of Education and Science 2 vols (PGSLIS, (Aldine, Chicago, IL, 1967).
Sheffield University, Sheffield, 1974). [23] D. Ellis, Modeling the information-seeking patterns of
[4] J. Martyn, Information needs and uses, Annual Review academic researchers: a grounded theory approach,
of Information Science and Technology 9 (1974) 3–23. Library Quarterly 63(4) (1993) 469–486.
[5] G. Ford, User Studies: an Introductory Guide and Select [24] S. Vedi, Information and the awareness of leaching of
Bibliography, CRUS Occasional Paper no. 1 (CRUS, plant nutrients. PhD thesis (University of Sheffield,
Sheffield University, Sheffield, 1977). Sheffield, 1986).
[6] C. Corkhill and M. Mann, Information Needs in the [25] A.D. Brown, Information, communication and organisa-
Humanities: Two Postal Surveys (CRUS, Sheffield tional culture: a grounded theory approach. PhD thesis
University, Sheffield, 1978). (The University of Sheffield, Sheffield, 1990).
[7] S. Stone, Humanities scholars: information needs and [26] S. Soto, Information in dentistry: patterns of commu-
uses, Journal of Documentation 38 (1982) 292–313. nication and use. PhD thesis (The University of
[8] P. Heather, A Study of the Use of Books and Libraries by Sheffield, Sheffield, 1990).
Children in Primary Schools (CRUS, Sheffield Univer- [27] M. Musoke, Health information access and use in rural
sity, Sheffield, 1984). Uganda: an interaction-value model. PhD thesis (The
[9] P. Heather, Teaching Methods and the Use of Books and University of Sheffield, Sheffield, 2001).
Libraries in Primary Schools (CRUS, Sheffield Univer- [28] T.D. Wilson, Information needs and uses: fifty years of
sity, Sheffield, 1984). progress? In: B.C. Vickery (ed.), Fifty Years of Informa-
[10] C. Beal, Community Profiling for Librarians (CRUS, tion Progress: a Journal of Documentation Review
Sheffield University, Sheffield, 1985). (Aslib, London, 1994), pp. 15–51.
[11] D. White, The Provision of Community Information. [29] T.D. Wilson, Information behaviour: an interdisciplin-
What Does the Public Want? (CRUS, Sheffield Univer- ary perspective, Information Processing & Management
sity, Sheffield, 1987). 33(4) (1997) 551–572.
[12] G. Ford, The Use of Medical Literature: a Preliminary [30] N.J. Belkin, C. Cool, A. Stein and S. Theil, Cases, scripts,
Survey (British Library Research and Development and information-seeking strategies: on the design of
Department, London, 1980). interactive information retrieval systems, Expert Sys-
[13] C. Harris and C. Harrop, Metals Information Needs in tems With Applications 9(3) (1995) 379–385.
Industry: a Pilot Study in South Yorkshire (CRUS, [31] P. Ingwersen, Cognitive perspectives of information
Sheffield University, Sheffield, 1984). retrieval interaction: elements of a cognitive IR theory,
[14] N. Roberts and B. Clifford, Regional Variations in the Journal of Documentation 52(1) (1996) 3–50.
Demand and Supply of Business Information: a Study of

Journal of Information Science, 29 (4)Downloaded

2003, pp. 239–248 # CILIP
from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
247
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
Approaches to user-based studies
[32] T. Saracevic, Modeling interaction in information
retrieval (IR): a review and proposal. In: S. Hardin,
(ed.), 59th Annual Meeting of the American Society for
Information Science (American Society for Information
Science, Silver Spring, MD, 1996), pp. 3–9.
[33] A. Spink, Study of interactive feedback during mediated
information retrieval, Journal of the American Society
for Information Science 48(5) (1997) 225–230.
[34] N. Ford, Levels of understanding and the personal
acceptance of information in higher education, Studies
in Higher Education 5 (1980) 63–70.
[35] N. Ford, Relating ‘information needs’ to learner char-
acteristics in higher education, Journal of Documenta-
tion 36(2) (1980) 115–132.
[36] N. Ford, Recent approaches to the study and teaching of
‘effective learning’ in higher education, Review of
Educational Research 51 (1981) 345–377.
[37] N. Ford, Knowledge structures in human and machine
information processing: their representation and inter-
action, Social Science Information Studies 3(4) (1983)
209–222.
[38] N. Ford, Learning styles and strategies of postgraduate
students, British Journal of Educational Technology
16(1) (1985) 65–79.
[39] D. Ellis, The physical and cognitive paradigms in
information retrieval research, Journal of Documenta-
tion 48(1) (1992) 45–64.
[40] N. Ford and R. Ford, Towards a cognitive theory of
information accessing: an empirical study, Information
Processing and Management 29(5) (1993) 569–585.
[41] N. Ford, F. Wood and C. Walsh, Cognitive styles and
searching, On-line & CDROM Review 18(2) (1994) 79–
86.
[42] F. Wood, N. Ford, D. Miller, G. Sobczyk and R. Duffin,
Information skills, searching behaviour and cognitive
styles for student-centered learning: a computer-assisted
learning approach, Journal of Information Science 22(2)
(1996) 79–92.
[43] H.A. Witkin, C.A. Moore, D.R. Goodenough and P.W.
Cox, Field-dependent and field-independent cognitive
styles and their educational implications, Review of
Educational Research 47 (1977) 1–64.
[44] G. Pask, Styles and strategies of learning, British Journal
of Educational Psychology 46 (1977) 128–148.
[45] N. Ford and D. Miller, Gender differences in Internet
perception and use, Third ELVIRA Conference, Electro-
nic Library and Visual Information Research (Aslib,
London, 1996), pp. 87–100.
[46] J. Rasmussen, A.M. Pejtersen and L.P. Goodstein,
Cognitive Systems Engineering (Wiley, New York,
1994).
248
Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 14, 2008
© 2003 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.
[47] N. Ford, Improving the ‘darkness and light’ ratio in user-
related information retrieval research, Journal of Doc-
umentation 56(6) (2000) 624–643.
[48] N. Ford, D. Miller and N. Moss, Web-search strategies
and retrieval effectiveness: an empirical study, Journal
of Documentation 58(1) (2002) 25–43.
[49] H. Fowkes and M. Beaulieu, Interactive searching
behaviour: Okapi experiment for TREC-8. In: Proceed-
ings of the BSC-IRSG 22nd Annual Colloquium on
Information Retrieval Research, Cambridge, 2000 (Brit-
ish Computer Society Information Retrieval Specialist
Group, London, 2000), pp. 47–56.
[50] A. Spink, T.D. Wilson, N. Ford, A. Foster and D. Ellis,
Information seeking and mediated searching. Part 1.
Theoretical framework and research design, Journal of
the American Society for Information Science and
Technology 53(9) (2002) 695–703.
[51] T.D. Wilson, N. Ford, D. Ellis, A. Foster and A. Spink,
Information seeking and mediated searching. Part 2.
Uncertainty and its correlates, Journal of the American
Society for Information Science and Technology 53(9)
(2002) 704–715.
[52] A. Spink, T.D. Wilson, N. Ford, A. Foster and D. Ellis,
Information seeking and mediated searching. Part 3.
Successive searching, Journal of the American Society
for Information Science and Technology 53(9) (2002)
716–727.
[53] N. Ford, T.D. Wilson, A. Foster, D. Ellis and A. Spink,
Information seeking and mediated searching. Part 4.
Cognitive styles in information seeking, Journal of the
American Society for Information Science and Technol-
ogy 53(9) (2002) 728–735.
[54] D. Ellis, T.D. Wilson, N. Ford, A. Foster and A. Spink,
Information seeking and mediated searching. Part 5.
User-intermediary interaction, Journal of the American
Society for Information Science and Technology 53(11)
(2002) 883–893.
[55] M. Beaulieu, Interaction in information searching and
retrieval, Journal of Documentation 56(4) (2000) 431–
439.
[56] R. Gaizauskas, P. Herring, M. Oaks, M. Beaulieu, P.
Willett, H. Fowkes and A. Jonsson, Intelligent access to
text: integrating information extraction into text brow-
sers. In: Proceedings of Human Language Technology
Conference 2001, (HLT 2001) (San Diego, CA, 2001), pp.
189–193.
[57] D. Petrelli, P. Hansen, M. Beaulieu and M. Sanderson,
User requirement elicitation for cross-language Informa-
tion Retrieval. In: 4th International Conference on
Information Seeking in Context – ISIC 2002 (Lisbon,
2002), pp. 5–20.
Journal of Information Science, 29 (4) 2003, pp. 239–248 # CILIP