Medical Informatics and the Internet in Medicine March 2006; 31(1): 45 52

The effect of source credibility on consumers perceptions of the quality of health information on the Internet

BENJAMIN R. BATES1, SHARON ROMINA2, RUKHSANA AHMED1, & DANIELLE HOPSON3

School of Communication Studies, Ohio University, Ohio, USA 2The American Cancer Society Partnership at Ohio University, Ohio, USA 3The Appalachia Reads Project at Ohio University, Ohio, USA
(Received May 2005; accepted December 2005)
1

Abstract Recent use of the Internet as a source of health information has raised concerns about consumers ability to tell good information from bad information. Although consumers report that they use source credibility to judge information quality, several observational studies suggest that consumers make little use of source credibility. This study examines consumer evaluations of web pages attributed to a credible source as compared to generic web pages on measures of message quality. In spring 2005, a community-wide convenience survey was distributed in a regional hub city in Ohio, USA. 519 participants were randomly assigned one of six messages discussing lung cancer prevention: three messages each attributed to a highly credible national organization and three identical messages each attributed to a generic webpage. Independent sample t-tests were conducted to compare each attributed message to its counterpart attributed to a generic webpage on measures of trustworthiness, truthfulness, readability, and completeness. The results demonstrated that differences in attribution to a source did not have a signicant effect on consumers evaluations of the quality of the information. Conclusions. The authors offer suggestions for national organizations to promote credibility to consumers as a heuristic for choosing better online health information through the use of media cochannels to emphasize credibility.

Keywords: Online health information, e-health, source credibility, information quality

1. Introduction The Internet is a rapidly growing and increasingly used source of health information. In 2004, 4.5% of all Internet searches worldwide were for health or health-related information [1]. Although a majority of US residents have not sought on-line health information [2], a large and growing number have stated that they would like to use the Internet to nd their own information about disease, treatment, and prevention [3]. The Internet is often promoted as

Correspondence: Benjamin R. Bates, School of Communication Studies, Lasher Hall, Ohio University, Athens, OH 45701, USA. Tel: 740.593.9163. Fax: 740.593.4810. E-mail: batesb@ohio.edu ISSN 1463-9238 print/ISSN 1464-5238 online 2006 Taylor & Francis DOI: 10.1080/14639230600552601

46

B. R. Bates et al.

an open, anonymous, and democratic means of accessing information. Nevertheless, because anyone with a computer and rudimentary skills can erect a website, there is a concern that the factors that make the Internet a potentially powerful means of disseminating good information are the very factors that may allow inaccurate and misleading information to become widespread [4 6]. To ensure that persons who access health information on the Internet receive the best information possible, advocates of quality health information have urged that consumers be aware of the variable worth of this information. One of the primary markers that advocates name for judging the quality of Internet-based health information is source credibility [2,7,8]. That is, a source that is a recognized expert, that keeps information current, and that has no competing interests in providing the information should be judged a more credible source than one that is not an expert source, that provides outdated information, or that has commercial interests in providing the information. Using these standards, several researchers have categorized sites offering health information by their level of credibility and have found that misleading and low-quality sites are as common as, if not more common than, highly credible sites [5,9 15]. To provide sources with additional source credibility, advocates of high-quality health information have listed model websites [16], named best design practices [4,17 19], and argued for external agencies to enforce systems of quality control [12,20]. The most important condition to ensuring that consumers access high-quality health information may not be internal to the sites. The best way to ensure that the Internet is used to gather high-quality health information may be to educate consumers about how to evaluate the credibility of the source of a website [1,6,13,21,22]. Efforts to educate consumers about judging the source credibility of health information on the Internet may be having some effect. Internet users often report that source credibility is a signicant consideration when they judge the reliability of health information [2,3,5,10,18,21,22,24 28]. Indeed, users claim to judge Internet-based health information in the same way that they judge other mediated messages, primarily by considering source credibility [29]. Although Internet users report that they consider the source, and despite limited backing for this claim [22], observational studies nd that Internet users often pay little attention to source credibility when seeking out health information on the web [1,5,17,21,27,31]. There are mixed ndings when self-report studies are compared to observational studies of consumers use of source credibility in evaluating health information on the Internet. To negotiate these differences, our study group conducted a study that manipulated source credibility as a variable for consumers evaluation of health related materials on the Internet. After describing our methods and results, we offer some implications that our study has for educating consumers about the use of source credibility as a factor for judging health information on the Internet. 2. Methods An underlying assumption of present research is that source credibility should matter to Internet users when they evaluate the quality of health information. When a highly credible source is named, consumers should judge that information to be of high quality. When no source or a low-credibility source is named, consumers should judge that information to be of low quality. Thus, we set out to identify specic differences, if any, in the perception of information quality when highly credible sources are compared to low- or no-credibility sources. Based on existing research, we expected persons who received information from a high-quality source to judge that information more credible than persons who received

Source credibility and online information

47

information from a no-credibility source. More specically, we speculated that when a source is named for health information on the Internet, that information will be judged to be more trustworthy, more truthful, more readable, and more complete than information that does not name a source. H1: Persons who are told the source of health information on the Internet will report signicantly higher evaluations of information quality than will persons who are not told the source of the health information.

2.1. Research instrument To evaluate the effect of source credibility on consumers perceptions of health information on the Internet our study group compared consumers perceptions of information that was taken from three high-credibility sources on a familiar health issue. These sources were the National Cancer Institutes (NCI), the American Lung Associations (ALA), and the American Cancer Societys (ACS) webpages on preventing lung cancer. These sources were then manipulated to either state the source of the information or to state that information simply came from a webpage about preventing lung cancer. Thus, participants were randomly assigned one of six messages: a message from NCI, an identical message from a webpage, a message from ALA, an identical message from a webpage, a message from ACS, or an identical message from a webpage. The participants then evaluated these messages on four dimensions of quality using a seven-point Likert-type survey instrument: the trustworthiness of the information (two items, a 0.80), the truthfulness of the information (three items, a 0.85), the readability of the information (two items, a 0.73), and the completeness of the information (a single item). 2.2. Sample population The survey was administered to a community-wide convenience sample through intercept survey methods. Participants were recruited at high-trafc areas in a regional hub city in southeastern Ohio, USA. These high-trafc areas included the parking lot of a major-chain discount retailer, the lobby of a local health and community center, and the intersection of two streets in the walkable downtown area of the city. Participants were entered into a drawing for a $50 gift card to a major-chain discount retailer as an incentive for participating. Six gift cards were awarded. Each participant provided informed consent. The study was approved by a University Institutional Review Board. 2.3. Analysis The survey data were analyzed using the Statistical Package for the Social Sciences software (SPSS 11.5 for Windows). Independent sample t-tests were employed to measure differences in the evaluation of the quality of the information from a highly credible source versus a nocredibility source. Participants evaluations of the named NCI message were compared to evaluation of the NCI message attributed to a webpage. Participants evaluations of the named ALA message were compared to evaluation of the ALA message attributed to a webpage. Participants evaluations of the named ACS message were compared to evaluation of the ACS message attributed to a web page. P-values less than 0.05 were considered signicant. A 95% condence interval was used.

48

B. R. Bates et al.

3. Results Five hundred and nineteen people participated in the study. Participants ranged from 18 to 80 years of age (m 28.25). A majority of the participants were White/European-American (*85%). Educational levels ranged from less than an eighth grade education to possession of a terminal degree; the median educational level was some college. Although this population is skewed whiter, younger, and more educated than the general population of the United States, these numbers are concordant with the United States Census Bureaus description of the population of the county where the study was conducted [32]. Table I reports the full demographics of the population studied. Our hypothesis proposed that persons who were told the source of health information on the Internet would report signicantly higher evaluations of information quality than would persons who were not told the source of the health information. This hypothesis was not supported. Participants who received the NCI message did not evaluate the message to be more trustworthy, truthful, readable, or complete when they were told that NCI was the source of the message than did participants who were told that the message came from a webpage. See Table II for means of each variable for the two groups. Participants who received the ALA message did not evaluate the message to be more trustworthy, truthful, readable, or complete when they were told that ALA was the source of the message than did participants who were told that the message came from a webpage (see Table III).
Table I. Demographics. Number Race Black/African-American White/European-American Native American Hawaiian/Pacic Islander Asian-American Bi/Multiracial Other Refused Ethnicity Of Hispanic descent Not of Hispanic descent Sex Male Female Refused Educational attainment Less than 8th grade Some high school High school graduate Some college 2 year degree 4 year degree Some graduate school Terminal degree Refused Age Note: N 519. %

24 441 15 2 8 6 18 5 13 506 222 284 13 5 10 76 256 50 61 31 22 8 Range: 18 80

4.6 84.9 2.9 0.4 1.5 1.1 3.5 0.9 2.5 97.5 42.8 54.7 2.5 1.0 1.9 14.6 48.3 9.6 11.8 6.0 4.2 1.5 Mean: 28.25

Source credibility and online information

49

Participants who received the ACS message did not evaluate the message to be more trustworthy, truthful, or complete when they were told that ACS was the source of the message than did participants who were told that the message came from a webpage (see Table IV). There was one exception to this overall trend. Participants who were told that ACS had authored the message did rate the message to be more readable than did participants who were not told the author. Overall, however, the manipulation of a highly credible source versus a no-credibility source had no main effect on the perception of the quality of the message. 4. Discussion The primary goal of this study was to evaluate whether source credibility has a signicant impact on consumers perceptions of the quality of health information on the Internet. Our results indicate that presenting high-credibility sources of health information on the Internet has little or no effect on consumers perceptions of quality when these sources are compared to a no-credibility source. We offer two reasons specic to these messages that may explain

Table II. Credibility differences for named versus unnamed source, National Cancer Institute message. Source named Mean Trustworthiness Truthfulness Readability Completeness 4.21 2.26 1.92 3.62 Source unnamed Mean 4.09 2.52 2.08 3.47 t 0.975 71.675 70.946 0.622 sig. n.s. n.s. n.s. n.s.

Note: lower scores indicate greater reader perception of trustworthiness, truthfulness, readability, and credibility of the message; d.f. 176; N 178 (91, named; 87, unnamed).

Table III. Credibility differences for named versus unnamed source, American Lung Association message. Source named Mean Trustworthiness Truthfulness Readability Completeness 4.22 1.99 1.94 3.51 Source unnamed Mean 4.22 2.12 1.93 3.69 t 0.038 70.894 0.087 70.590 sig. n.s. n.s. n.s. n.s.

Note: lower scores indicate greater reader perception of trustworthiness, truthfulness, readability, and credibility of the message; d.f. 172; N 174 (85, named; 89, unnamed).

Table IV. Credibility differences for named versus unnamed source, American Cancer Society message. Source named Mean Trustworthiness Truthfulness Readability Completeness 4.11 2.60 1.70 3.53 Source unnamed Mean 4.10 2.97 2.17 3.43 t 0.156 71.22 72.94 0.315 sig. n.s. n.s. .004 n.s.

Note: lower scores indicate greater reader perception of trustworthiness, truthfulness, readability, and credibility of the message; d.f. 165; N 167 (84, named; 83, unnamed).

50

B. R. Bates et al.

this nding and some advice for encouraging consumers to take source credibility into greater account when evaluating health information on the Internet. One reason that our participants may not have taken source credibility fully into account is that consumers may not be invested in seeking out high-quality information on the Internet. As reported by Morahan-Martin [1], users generally consult only the most popular site or sites of health-related information found through general Internet search engines. As such, users may believe that there are few, if any, distinctions in the quality of health information available from different sources. If users are not used to comparing different sources of health information on the Internet because they believe that one webpage is as good as any other webpage, it would be surprising to nd signicant differences in the perceptions of the quality of health information based on different sources of Internet-based health information. Moreover, because of low health literacy and health information-seeking skills, consumers may not know how to evaluate the quality of health information on the Internet [5,33]. Moreover, patient consumers exhibit lower evaluation skills than do physician and expert consumers. As such, users may nd it difcult to tell the difference between high-credibility and no-credibility sources of health information. If users lack health literacy and informationseeking skills, the lack of distinction in the evaluation of good and poor sources would represent part of a larger trend of general consumer propensities related to health. Future research may want to measure levels of health literacy alongside and with measures of quality and credibility to account for the possible intersection of these variables. Alternatively, the ndings of our study could support the claim that consumers often use source credibility to evaluate the quality of health information on the Internet. Each message employed in the study addressed prevention of lung cancer. The three named sources, NCI, ALA, and ACS each have a vested interest in reducing the rate of lung cancer. Moreover, each organization is known for its efforts to reduce tobacco use rates, promote smoke-free workplaces, and to implement other measures. Readers who were told that NCI, ALA, and ACS authored messages may have marked these messages as less credible because of perceptions of bias. These markings of bias may have reduced the trustworthiness, truthfulness, readability and completeness of the messages such that they were equivalent to the levels of trustworthiness, truthfulness, readability and completeness of messages that appear on generic websites. Although this possibility is a less elegant explanation, future research should includes measures of perceived bias of the source to account for the possibility that bias has a negative effect that counters the positive effect of naming the source. Although there are at least two possibilities for the lack of difference in the evaluation of high-credibility versus no-credibility sources, the implications that these ndings have for health-information providers on the Internet are largely the same. To make their messages appear to be of higher quality than generic websites, i.e. to promote their websites as being more trustworthy, truthful, readable, and complete than other web pages, NCI, ALA, and ACS may wish to promote themselves as higher-quality brands than generic websites. Although much current research promotes skill-building for consumers so that consumers can judge for themselves the quality of a website [1,6,13,21,22], these organizations (and others) may want to emphasize brand awareness over self-direction. A patients personal physician is still the most trusted source of health information [22,25]. As such, NCI, ALA, and ACS may wish to promote themselves to health care providers so that, when a patient asks for more information, the provider directs the patient to the organizations website specically rather than giving less directive advice that may lead the patient to consult generic webpages. Alternatively, organizations like NCI, ALA, and ACS may want to take a page from commercial health-information websites. Trafc to commercial health websites may result from advertising campaigns that promote the credibility of these websites. For example,

Source credibility and online information

51

DrKoop.com leveraged the authority of former US Surgeon General C. Everett Koop to attract Internet users and WebMD.com promoted itself as a comprehensive source of health information in national advertising campaigns. Because NCI, ALA, and ACS already engage in several media campaigns, they could emphasize their websites as highly credible sources for additional information. Were they to do so, then Internet users may remember these sources and visit them directly when seeking additional health information instead of relying on general Internet search engines. Because consumers have so many options for health information on the Internet, they may have difculty telling good information from poor information. To ensure that the best, most accurate, timely, and relevant information is used by consumers, organizations devoted to health have an obligation to attract users to this information. Only when organizations commit to these strategies will Internet users best be able to avoid misleading and inaccurate information and obtain information online that best encourages health and healthy behaviors.

References
1. Morahan-Martin JM. How Internet users nd, evaluate, and use online health information: A cross-cultural review. Cyberpsychology and Behaviour 2004;7:497 510. 2. Tu HT, Hargraves JL. Seeking health care information: Most consumers still on the sidelines. Issue Brief, Center for Studying Health System Change 2003;61:1 4. 3. Escoffery C, Miner KR, Adame DD, Butler S, McCormick L, Mendell E. Internet use for health information among college students. Journal of American College Health 2005;53:183 188. 4. Leadbetter M. Internet support for people who are affected by breast cancer. Nursing Times 2003;99(14): 26 27. 5. Benotsch EG, Kalichman S, Weinhardt LS. HIV-AIDS patients evaluations of health information on the Internet: the digital divide and vulnerability to fraudulent claims. Journal of Consulting and Clinical Psychology 2004;72:1004 1011. 6. Travers RL. Information consumerism on the world wide web: Implications for dermatologists and patients. Seminars in Cutaneous Medicine and Surgery 2002;21:223 231. 7. Kunst H, Groot D, Latthe PM, Latthe M, Khan KS. Accuracy of information on apparently credible websites: Survey of ve common health topics. British Medical Journal 2002;324:581 582. 8. Kim P, Eng TR, Deering MJ, Maxeld A. Published criteria for evaluating health related websites: A review. British Medical Journal 1999;318:647 649. 9. Latthe PM, Latthe M, Khan KS. Quality of information on female sterilisation on the Internet. Journal of Obstetrics and Gynaecology 2000;20:167 170. 10. Feathers AS, Charron-Prochownik D, Siminerio LM, Manthei ER, Dorman JS. Genetics and type 1 diabetes: Online resources for patients. Diabetes Education 2004;30:972 979. 11. Kunst H, Khan KS. Quality of web-based medical information on stable COPD: A comparison of noncommercial and commercial websites. Health Information and Libraries Journal 2002;19:42 48. 12. Mashiach R, Seidman GI, Seidman DS. Use of mifepristone as an example of conicting and misleading medical information on the Internet. British Journal of Obstetrics and Gynaecology 2002;109:437 442. 13. Diering CL, Palmer MH. Professional information about urinary incontinence on the world wide web: It is timely? Is it accurate? Journal of Wound, Ostomy, and Continence Nursing 2001;28:55 62. 14. Latthe M, Latthe PM, Charlton R. Quality of information on emergency contraception on the Internet. British Journal of Family Planning 2000;26:39 43. 15. Latthe PM, Latthe M, Khan KS. Quality of medical information about menorrhagia on the worldwide web. British Journal of Obstetrics and Gynaecology 2000;107:39 43. 16. Junghans TB, Sevin IF, Ionin B, Seifried H. Cancer information resources: Digital and online sources. Toxicology 2002;173:13 34. 17. Walther JB, Wang Z, Loh T. The effect of top-level domains and advertisements on health web-site credibility. Journal of Medical Internet Research 2004;2:e24. 18. Nahm ED, Preece J, Resnick B, Mills ME. Usability of health web sites for older adults: A preliminary study. Computers, Informatics, Nursing 2004;22:325 334. 19. Roberts JM, Copeland KL. Clinical websites are currently dangerous to health. International Journal of Medical Informatics 2001;62:181 187.

52

B. R. Bates et al.

20. Dyer KA. Ethical challenges of medicine and health on the internet: A review. Journal of Medical Internet Research 2001;3:e23. 21. Peterson G, Aslani P, Williams KA. How do consumers search for and appraise information on medicines on the Internet?: A qualitative study using focus groups. Journal of Medical Internet Research 2003;5:e33. 22. Jacob J. Consumer access to health care information: Its effect on the physician patient relationship. Alaska Medicine 2002;44:75 83. 23. Gray NJ, Klein JD, Noyce PR, Sesselberg TS, Cantrill JA. Health information-seeking behaviour in adolescence: The place of the Internet. Social Science and Medicine 2005;60:1467 1478. 24. Stavri PZ, Freeman DJ, Burroughs CM. Perception of quality and trustworthiness of Internet resources by personal health information seekers. American Medical Informatics Association. Annual Symposium Proceedings, Bethesda, MD; 2003: pp. 629 633. 25. Dutta-Bergman M. Trusted online sources of health information: differences in demographics, health beliefs, and health-information orientation. Journal of Medical Internet Research 2003;5:e21. 26. Stella N. Credibility and consistency earn users trust. Internet Healthcare Strategy 2003;5:8 9. 27. Eysenbach G, Kohler C. How do consumers search for an appraise health information on the world wide web?: Qualitative study using focus groups, usability testing, and in-depth interviews. British Medical Journal 2002;324:572 577. 28. Casebeer L, Bennett B, Kristofoc R, Carillo A, Centor R. Physician Internet medical information seeking and online continuing education use patterns. Journal of Continuing Education in the Health Professionals 2002;22: 22 42. 29. Menon AM, Deshpande AD, Perri M III, Zinkhan HM. Trust in online prescription drug information among Internet users: The impact on information search behavior after exposure to direct-to-consumer advertising. Health Marketing Quarterly 2002;20:17 35. 30. Dutta-Bergman M. The impact of completeness and web use motivation on the credibility of e-health information. Journal of Communication 2004;54:253 269. 31. Shon J, Marshall J, Musen MA. The impact of displayed awards on the credibility and retention of web site information. Proceedings of the American Medical Informatics Association Symposium, Los Angeles, CA; 2000: pp. 794 798. 32. QuickFacts from the US Census Bureau [Internet]. Washington (DC): United States Census Bureau. 2005 Feb. 1 [cited 2005 May 1]. Available from: http://quickfacts.census.gov/qfd/states/39/39009.html 33. Groot D, ter Riet G, Khan KS, Misso K. Comparison of search strategies and quality of medical information on the Internet: A study related to ankle sprain. Injury 2001;32:472 476.