INTRODUCTION Academic research in the LIS field is thought, designed and carried out with the final aim

of satisfying users needs and improve their experience in the encounter with the information world. The structural distance between scholars and users is fulfilled by physical libraries, with its information resources, and librarians, with knowledge skills aimed at delivering specific services in the most sound and tailored possible way according to users needs. It is largely accepted that an effective dialogue between scholars and practitioners is of the outmost importance in order to provide users with services drawn by decisions based on the best available evidence (understood as research-based knowledge) (Hjorland 2011). In the case of information professional this means to focus upon using the best available evidence to make sound decisions on library and informatics issues (Eldredge 2002 p.342). Whatever are the norms and ideals in the philosophy of science we adopt to recognize which is the best evidence available, practitioners are required to appraise the evidences they will have gathered to make a decision. The kind of review placed at the top of the categories of evidence is the systematic review1, due to its capacity to minimize bias while integrating multiple research studies (Eldredge 2006). Nonetheless others can be performed (Petticrew and Roberts 2006) like narrative review (studies synthesized according to their argumentation rather than statistically) or critical review (highlighting the analysis of the social interests involved) (in Hjorland 2011). To support librarians skills to be competent and confident at assessing the quality of the research several checklists have been developed (Booth and Brice 2003; Glynn 2006; Koufogiannakis 2006; Host and Runeson 2007; Dilevko 2007) in the form of a set of criteria against which a piece of research can be assessed (Atkins and Sampson 2002). PURPOSE AND METHODS This essay is an attempt to practice this skill, which is one of the most important aspect to learn in order to be a critical scientist and reader. Another reason lies in the fact that a research carried out by scholars and published in a top journal not always guarantees its excellence (Little and Parker 2011). We have chosen a research whose results have been published on Journal of Documentation, Vol. 68. No.4, 2012: Modelling information-seeking behavior of graduate students at Kuwait University (Al Muomen et al. 2012). The reason of this choice lies in the fact that the broader topic in which this research can be framed (information related behaviors) has been object of our recent interest; therefore the requirement of being familiar with the subject area can be fulfilled (although partly) in order to carry out a more accurate evaluation (Little and 2011). We will analyze several aspects of a scientific paper by trying to integrate criteria extracted by several checklists (Booth and Brice 2004, Greenhalgh 2010, Keshav 2012, Little and Parker 2011) and online guides. In particular we will try to divide these criteria in three main sections corresponding) to the three pass approach proposed by Keshav. They refer to a three passes reading (each with his own goals): a quick scan, reading with greater care, virtually re-implementation of the paper. 1. A QUICK SCAN: Authority. The research has been carried out by three authors. N. Al Muomen is an Assistant Professor in the Department of Library and Information Science, Kuwait University. Despite belonging to an official Academic department, we failed in finding her main publications and research interests. A. Morris is
1

Systematic reviews are concise summary of the best available evidence that address precise defined questions; they use explicit and rigorous methods to identify, critically appraise and synthesize relevant studies (Mulrow 1987; Greenhalgh 2010). It is prompted by empiricism/positivism epistemological principles (Hjorland 2011).

Professor of Information Science at Loughborough University. Although, among her research interest is The Impact of Culture on Human Information Seeking Behaviour her previous publications are mainly oriented to the evaluation of user interface design and users interaction with metadata, a more technical aspect compared to the wider information behavior realm. Sally Maynard, who is a lecturer at the department of Information Science of the Loughborough university, has as a main interest analysis of childrens reading habits with several publications related to this topic. Although the profile of the authors is academically high, we can maybe not neglect the fact that their previous researches did not specifically focus on the topic of the article here analyzed. The journal where the results were published is, conversely, among the top journals of the LIS field, achieving every year the highest citation ratings in ISI Web of Science for comparable titles (Singh et al. 2011). This last point, conversely, suggests that authors background not always affect the quality of a research and that this piece of research has been conducted following strict scientific criteria and is likely to be accepted by the scientific community. Clearity of purpose and aims: the purpose of the research is to report the results of research conducted to model the information-seeking behavior of graduate studentsand the factors influencing that behavior. The aim of the research was to investigate the information seeking behavior of graduate students within the context of Kuwait, and to explore, in particular, the factors influencing the patterns of such information seeking. The main objectives are: to identify and consider existing models of information seeking behavior; to investigate the range of influences to which graduate students are subject when seeking information; to consider the effect of cultural factors on information seeking; to develop a model of the information seeking behavior of graduate students at Kuwait University. As we can see purpose, aim and main objectives are stated separately and quite clearly. The only objection we have refers to the objectives section: it seems that the authors will deliberately put particular stress on cultural factors that influence students information-behaviour at the outset of the research. This in itself surely is a remarkable objective, but we fear that it may lead to a misrepresented reality in the development of a objective model. Cultural factors may just be embedded in the range of influences the authors want to investigate so that the risk of being overemphasized can be avoided. Presentation of abstract and references: The house style of the journal facilitates a good presentation of the abstract because of its predefined structure in which several sections need to be followed: purpose; design, methodology, approach; findings; research limitations/implications; originality/value; keywords; paper type. References are presented clearly and consistently using the Harvard style; we have not been able to use a statistic software for a precise calculation, but it is evident at a first glance that most of the sources cited are not older than five years compared to the publication of this research, showing that the article takes into consideration contemporary studies. 2. Reading with greater care. Background literature review: the authors provide a literature review of both theories of information behavior and field researches. Information about theories of information seeking and behavior are only sketched and are drawn by a secondary resource (for ex. Case 2007). Then the focus shifts to information seeking behavior dealing with the use of online information resources with a list of studies related to graduate student in particular whose main results are not reported. In relation to field researches, authors provide some detail of the published researches on the information behavior of graduates creating three groups: information literacy (a short list of studies of which only implications are stated); use of resources approach (a detailed report of a few studies whose focus and results are shown); theoretical approach (a very short list of studies of which only the main topic is stated).

At the end of the paragraph they do not locate their study in none of these groups, leaving the reader wondering to which category this research can be associated. As far as we know new directions in the information behavior field also are being followed by scholars. They relate to the importance of contextual, psychological and personality traits influencing an information seeking behavior, and most of these researches have been carried out among graduate students (Heinstrom 2003, 2005). These studies could probably be omitted if aim of this article was a simple report on technical factors influencing information behavior of students. But going back to the aim statement, the range of influences the authors plan to take into consideration does not seem to be specified, leading the reader to think that the whole range of influences will be considered. We suppose therefore that quoting the above mentioned studies in this kind of research might have been appropriate. Clarity of theoretical position: it is stated that, in order to provide a theoretical background seven models will be considered because of their user-centred approach and because they explain factors and variables associated with information-seeking behavior process (p.433). It is not quite clear in what terms they are going to take into consideration these seven models or which aspects of them will be embedded in their research; at the end of the same paragraph though they say that the model they have developed is an extension of the models provided by Rowley and Urquhart (2007), which is listed among the 7 models, and Wilson (2005), which is not listed in that list, but it appears in the paragraph where their new extended model is being explained. A detailed explanation of the reasons why these two models have been selected is clearly provided: the necessity of combining the variables affecting information behavior (Wilson 2005) with micro and macro factors2 influencing students information behavior proposed by Urquhart and Rowley (2007) (a model developed to represent the information behavior of students when using electronic information resources). The authors state that in their new model demographics (gender, age, nationality) and cultural aspects (English proficiency) will be added. Their theoretical background is thus clearly explained. We only noticed that Wilsons (2005) and Leckies (Leckie et al. 1996) models already feature the above mentioned demographic factors. Research methods: Rarely researches in the information behavior field are either qualitative or quantitative (Bawden and Robinson 2012) and this case study is not either. For this reason it is suggested it should meet criteria of rigor3 (internal validity, external validity, reliability, objectivity); besides, in order to enhance credibility researchers should opt for a triangulation of the methods (Pickard 2007). The authors, in fact, have used several methods to retrieve data: questionnaire survey; semi-structured interviews; face to face and online interviews; focus groups. One weakness can be the fact that these are not weighted equally since they state that the paper focuses on the result of the questionnaire survey to students and where appropriate, these are supported by the findings from the other more qualitative approaches used (p. 436). By reading this statement we may think that data supporting questionnaire surveys will be taken into consideration whereas data not confirming the questionnaire results will be rejected. From a more technical point of view, methods are explained more clearly. The questionnaire is clearly shown in the ancillary materials; it was based on the Urquhart and Rowley questionnaire with the inclusion of statements used by the critical incident technique. Validated scales have been used for evaluation of information literacy and questions involving scales. Reliability and statistical significance was tested by
2

Variables proposed by Wilson are: psychological, demographic, role related, environmental, source characteristics. Micro factors isolated by Urquhart and Rowley are: information literacy, search strategy, support and training, pedagogy, academics information behavior, discipline and curriculum; macro factors identified are: information resources design, availability and constraint to access, information learning technology infrastructure, policies and funding, organizational knowledge and culture. 3 Pickard suggest that qualitative researches should meet criteria of trustworthiness (credibility, transferability, dependability, confirmability) whereas quantitative and mixed methods researches should meet criteria of rigor.

using SPSS; internal consistency was tested by using Cronbachs coefficient; logistic regression was used to identify the key predictors; construct extracted for a principal component analysis were conformed to the Scree test and Kaiser s criterion eigenvalue >1); varimax rotation identified three factors that combined the constructs. Not being staticians and having a look to academic articles available on the internet, we realized that while the scree test actually isolate important factors which account for the bulk of the correlations (http://luna.cas.usf.edu/mbrannic/files/tnm/factor/htm) there is a wide consensus in the literature that considering all factors with eigenvalues greater than 1.0 is among the least accurate methods for selecting the number of factors to retain (Velicer & Jackson, 1990 in Costello and Osborne 2005). Also, while principal components with varimax rotation and the Kaiser criterion are the norm, they are not optimal, particularly in the case of the social sciences (Costello and Osborne 2005). The authors later on say that in order to futher investigate factors related to library anxiety other statistical tests (Chi-square, MannWhitney, U test, Kruskal-Wallis, Fishers exact Test) were used, where appropriate (p. 439). In which case specific statistical test will be used is not specified. In general authors have explained clearly that microfactors will be analyzed by means of statistical tools, whereas macrofactors, because of their more qualitative nature, will be analyzed by means of other methods, as focus groups. In relation to focus groups, aspects of composition, questions and follow up questions, way of recording data (Gorman and Clayton 1997) are not provided. As well there is no hint of a negotiated outcome in relation to focus groups results, a measure often suggested for qualitative studies (Pickard 2007). Lack of clarity has been noticed in the relationship between investigation methods and sample4. The importance of sample seems to be paramount in order to be able to generalize findings (Pickard 2007, Greenhalgh 2010). Out of 1113 students 800 were sampled for the research. Students were selected using a clustered approach; departments were randomly selected from the ten school (p.438). It seems that no statistical examination of probability of the representativeness of sample was done. Details about samples of academic and library staff are not specified: numbers are provided but it s not clear if the sample of students, ,academic and library staff, in proportion, reflects actual figures of Kuwait University members. Again, if the proportion between students studying for a Master Degree, a Higher Diploma or a Phd reflect the reality is not specified. Furthermore, authors ignore students who did not respond (430 students) to the questionnaire, causing a potential bias in the result (Greenhalgh 2010). To whom results are useful? Results of this study are told to be twofold. On one hand, a new model developed for information seeking behavior in developing country can be applied in additional academic institutions in Kuwait; in addition, the model is open to examinationin other developing countries (p. 449). On the other hand, useful suggestions are offered to improve information services of the library at University Kuwait as providing a better IT infrastructure, reducing bureaucracy about gaining access to services and more in depth training about information literacy both for staff and students, improving English language training; reducing reliance on English online databases, improving the image and the status of library, services and staff. Originality, value and limitations of the research: halfway through the article we read: the study is of specific worth because it concentrates on graduate students, and on the particular cultural characteristics and influences of a developing country (p.431). The value of this article is actually to have considered (with the aim of offering an information seeking model) a part of population never investigated before: students
4

Questionnaire where sent to 800 graduate students and 180 academics; semi structured interviews with 8 academics; face to face and on line interviews with 11 university library staff; 4 focus groups with 24 students; 3 focus groups with 10 faculty staff.

from a developing country. Of value is also a brief review of differences between Uk/Western countries and developing countries students information behavior in the context of changes in electronic resources. We only wonder why in the abstract session no mention of developing countries, differences between Western and developing countries and the technical context of use of electronic resources (the three core elements of this research) is done. Missing in the abstract also are concrete limitations.

3. Re-implementing the article. Results analyzed appropriately. As already pointed out results from questionnaires and focus groups were analyzed. Tables reporting statistical tests and logistic regression results are provided as a supplementary material. Reported values of the statistical tests and logistic regression unfortunately imply the reader to be at least partially knowledgeable in the field as legends are not shown. According to the authors all microfactors (demographics, information literacy, IT skills, psychological factors, support and training, academic influence) affect some kind of information behavior. Not all results though are justified by the logistic regression. Among cultural aspects the effect of English proficiency and nationality are justified by the logistic regression, but internet censorship is a concern drawn by focus groups analysis. Support and training were not justified by the logistic regression as well, but only by focus groups results. Gender issues are presented as a problematic cultural aspect (segregation between male and females in particular) drawn by the focus groups sessions, although by having a look at the tables provided gender determine many other information behaviors, which are not treated in the paragraph. As already explained, macro factors (information resource design, availability and constraint to access, policies and funding) as well as organizational knowledge and culture and information technology infrastructure, were reported as results of focus groups only, whose data are not provided. Justification of conclusions. At the end of the article two main conclusions are drawn: the first one has a theoretical nature and it refers to the model the authors have developed; the second one refers to factors having an impact on the information seeking behavior. With regard to the first point the authors say that the use of the adapted model was also helpful in identifying differences between institutions, countries and continentsof particular interest was the comparison of models based on the situation in the West compared to practices in Kuwait (p.447). The importance of cultural factors actually have been stressed out several times through the article and, but the usefulness of the new model to explore differences between western and developing Countries seems to be slightly ambitious, due to the fact that differences between western countries and developing countries are not investigated systematically and results discussed related to this issue are extrapolated only by opinions of people attending focus groups. A further point which lacks of full clarity is the statement according to which all factors influenced student information behavior such as identifying needs, locating sources, searching (chaining, browsing, differentiating, monitoring, extraction and verifying) and providing feedback or ending the process (p. 447). Although few hints are offered about students changing theirs searches, it seems to us that all these issues are mentioned in the conclusions paragraph for the very first time. Conversely, very consistent with the results are suggestions for practitioners related to all those areas that would benefit from improvement.

CONCLUSIONS

In our opinion the article we have analyzed is of great value because introducing new elements to the LIS literature: the population studied and the model developed highlighting the importance of cultural factors affecting information behavior of students from developing countries. Also we think that results offered in the supplementary material have got much more potential than what stated from the authors. We believe that our criticisms may derive from the little capacity of processing the large amount of data the reader is offered in this article which focuses on several and complex aspects of information behavior simultaneously. As far as the importance and capability of evaluating scientific papers concerns, we firmly believe we have personally experienced the crucial importance of a deep reading of any papers reporting research results by adopting a scientifically skeptic attitude aimed at evaluating real facts and not only authors opinions; that we have achieved the awareness of the necessity of gaining as much experience as possible in this task in order to fully appreciate further academic researches; thirdly, that in order to thoroughly understand an article a great deal of time is necessary and the pass of virtually re-implement the article, if the article is of our interest, always recommended. This essay has been a first step toward gaining the skill of being competent at evaluating a piece of research. As such, we feel this experience has involved important processes and insights. and apologize with the authors of the article we have analyzed for our not justified criticisms.

BIBLIOGRAPHY
Al-Muomen, N., Morris, A., Maynard, S. 2012. Modelling information-seeking behaviour of graduate students at Kuwait University. Journal of Documentation, vol. 68 (4) , pp. 430-459.

Atkins, C., and Sampson, J. 2002. Critical Appraisal Guidelines For Single Case Study Research. ECIS 2002, June 68, Gdaosk, Poland. Available at: http://is2.lse.ac.uk/asp/aspecis/20020011.pdf Bawden, D. and Robinson, L. (2012). Introduction to information science. Facet: London. Booth, A. and Brice, A. 2004. Why evidence-based information practice? In A. Booth & A. Brice (Eds.). Evidence based practice for information professionals, pp. 1-12. London: Facet Publishing. Case, D.O. 2007. Looking for information: a survey of research on information seeking, needs and behaviour. 2nd ed. Amsterdam: Elsevier. Costello, A. B. and Osborne, J. W. 2005. Best practices in exploratory factor analysis: four recommendations for getting the most of your analysis. Practical Assessment, Research & Evaluation, vol. 10 (7), pp. 1-9. Dilevko, J. 2007. Inferential statistics and librarianship. Library & Information Science Research, Vol. 29 (2), pp. 209-229. Eldredge, J. 2002. Evidence-based librarianship levels of evidence. Hypothesis, vol. 16, pp. 10-13. Eldredge, J. 2006. Evidence-based librarianship: the EBL process. Library Hi Tech, vol. 24 (3), pp. 341-354. Glynn, L. 2006. A critical appraisal tool for library and information research. Library Hi Tech, vol. 24 (3), pp. 387-399. Gorman, G.E. and P. Clayton 1997. Qualitative research for the information professional: A practical handbook. London: The Library Association. Greenhalgh, T. 2010. How to read a paper: The basics of evidence-based medicine (4th edn), Chichester: Wiley- Blackwell. Heinstrm, J. 2003. Five personality dimensions and their influence on information behavior. Information Research, Vol. 9 (1). Available at http://InformationR.net/ir/9-1/paper165.html (visited: 25 November 2012). Heinstrm, J. 2005. Fast surfing, broad scanning and deep diving: The influence of personality and study approach on students' information-seeking behavior. Journal of Documentation, Vol. 61 (2), pp. 228 247. Hjrland, B. 2011. Evidence-Based Practice: An Analysis Based on the Philosophy of Science. Journal of the American Society for Information Science and Technology, vol. 62 (7), pp. 1301-1310. Hst, M. and Runeson, P. 2007. Checklists for Software Engineering Case Study Research. ESEM 2007, pp. 479-481. Keshav, S. 2012. How to read a paper, available at: http://blizzard.cs.uwaterloo.ca/keshav/home/Papers/data/07/paper-reading.pdf (visited: 24th November 2012) Koufogiannakis, D. 2006. Small steps forward through critical appraisal. In Evidence Based Library and Information Practice, vol. 1 (1), pp. 81-82.

Leckie, GJ, Pettigrew, KE and Sylvain, C. 1996. Modelling the information seeking of professionals: a general model derived from research on engineers, health care professionals and lawyers. Library quarterly, vol. 66 (2), pp. 161-193. Little JW and Parker R. 2011. How to Read a Scientific Paper. Available from: http://www.biochem.arizona.edu/classes/bioc568/papers.htm (visited 25th November 2012) Mulrow C.D. 1987. The medical review article: state of the science. Annals of Internal Medicine, vol. 106, pp. 485-488. Petticrew, M. and Roberts, H. 2006. Systematic reviews in the social sciences: A practical guide. Malden, MA: Blackwell Publishing. Pickard, A.J. 2007. Research methods in information. London: Facet Publishing. Rowley, J. and Urquhart 2007. Understanding student information behavior in relation to electronic information services: lessons from longitudinal monitoring and evaluation, part. 1. Journal of the American Society for Information Science and Technology, vol. 58 (8), pp. 1162- 1174. Singh, N. K., Sharma, J and Kaur, N. (2011). Citation analysis of Journal of Documentation. Webology, vol. 8 (1), Article 86. Available at: http://www.webology.org/2011/v8n1/a86.html

Velicer, W. F. and Jackson, D. N. (1990). Component Analysis Versus Common Factor-Analysis. Some Further Observations. Multivariate Behavioral Research, vol. 25 (1), pp. 97-114. Wilson, T. 2005. Evolution in information behavior modeling. Wilsons model, in Fisher, K., Erdelez, S. and Makechnie, L. (Eds), Theories of Information Behaviour, Information Today, Medford, NJ, pp. 31-36.