Secondary Logo

Journal Logo


Proposed Curricular Objectives to Teach Physicians Competence in Using the World Wide Web

McGowan, Julie J. PhD; Berner, Eta S. EdD

Author Information


In this article, we examine the educational implications of the World Wide Web as a primary source of health information in medical practice and offer curricular objectives to educate the next generation of physicians to be competent in using the Web.

The Need

The growth of the field of medical informatics has created an awareness of the need to teach medical students and residents computer skills, but the incorporation of medical informatics instruction into undergraduate medical education has been as varied as the world's curricula leading to the Doctor of Medicine degree. Evidence-based medicine has been embraced by most medical schools, as has instruction in its basic precepts.

However, few schools are offering specific instruction in management of information found on the Web, even though the increasing availability of information there means that a competent physician must:

  • understand the Web and realize its importance;
  • develop Web competencies in his or her undergraduate, graduate, and continuing medical education;
  • have a working knowledge of the structure and the function of the Web and be able to use Web-based tools to acquire information through the Web;
  • be able to critically review Web sites and the content of those sites for application in problem-solving; and
  • be able to use the Web for patient education and be able to advise patients about the level of quality of patient-acquired Web information and the value and potential dangers of accessing information on the Web.

A search of the complete Medline database on April 1, 2003, using the search strategy outlined in Table 1, yielded 516 citations. A thorough review of titles, abstracts if available, and full text articles as appropriate, found that the vast majority were about using the Web for teaching. Fewer than 5% of the articles dealt with education in which Web resources were the focus of the teaching. Most of these articles were devoted to providing resources available on the Web in support of other education efforts. Only a handful discussed the issues of teaching how to perform quality review of Web information.

Table 1
Table 1:
The Authors’ Search Strategy of the Medline Database to Find Articles about the World Wide Web in Medical Education

Information presented in print format frequently acquires a status approaching that of generally accepted human knowledge. That status is often raised if the print information appears on a worldwide network. Unlike the products of the traditional print medium, however, that are expensive to produce and are usually subjected to editorial oversight, it is relatively easy to publish anything on the Web.1 This ease of publishing raises questions about both the content and the security of health information on the Web.

Web Problems and Attempts to Solve Them

With the Web becoming so pervasive, physicians and physicians-in-training must have a healthy skepticism of its content. They must understand the differences between the reliability of material retrieved from the U.S. National Library of Medicine's Entrez databases and that garnered through a general Google search. They must also be able to counsel patients on the quality (or lack of it) of Web information.

Web security is another problem about which physicians and physicians-in-training need education. Web-based electronic medical record systems are generally safe, as are reputable e-health commerce systems.2 However, patients’ confidentiality may be compromised in the Web environment in many ways, not the least of which is through patients’ ignorance. Specifically, patients’ interaction with commercial health Web sites, sharing as little information as birth date, gender, ethnicity, and zip code, when linked to readily available census data, enables personal identification at over a 95% level.3

There have been attempts to provide prefiltered access to health information sites, to rate health information on the Web, and to develop tools to automatically filter Web-based information. MEDLINEplus (at 〈〉) and HealthWeb (at 〈〉) are examples of sites that provide filtered health information. They have achieved their aims and are in widespread use. Attempts to rate Web sites, such as the Health on the Net Foundation Code of Conduct and American Medical Society Library Choices, have been only marginally successful.4

Researchers at Stanford5 and Oregon Health Sciences University6 have investigated problems involved in the design of medical informatics tools to provide quality filtering of Web sites. Building on the work of Health on the Net, both groups have identified core elements for automated filters; however, lack of standardization of metadata elements (i.e., data about the data) for Web sites has presented significant obstacles. MedSifter, a medical version of BioSifter,7 offers a customizable tool for continuous filtered access to information from the Web, but, for reliability, initial human selection of approved Web portals is required.

Although these efforts show promise, they also underscore the need for physician education about practicing “information-astute medicine.” The Association of American Medical Colleges’ Medical School Objectives Project (MSOP) recognizes the need for medical informatics in the curriculum. In the MSOP's Lifelong Learning Role objective, educators are challenged to facilitate student learning in information retrieval, including the identification and acquisition of documents from the Web, and in filtering, evaluating, and reconciling that information.8 However, traditional issues, such as lack of curricular time and inadequate expertise of faculty, have contributed to the slow adoption of the MSOP recommendations.

Another issue is the significant difference in the perceptions of medical educators and medical students of the need for instruction in critically evaluating electronic resources. Approximately half of matriculating medical students feel that they are already competent in this skill, while medical educators disagree that such a high percentage are competent.9 Similar results are found in the Association of American Medical Colleges’ graduation questionnaire responses. Data from 2002 show that 55% of student respondents felt confident in their ability to use a variety of telemedicine applications that are not widely used in practice, much less incorporated into medical school curricula. Over 93% also felt confident in their ability to perform sophisticated searches of medical information databases, and 78% felt confident in their ability to critically analyze a research report.10 Again, these data, besides illustrating the difficulties in self-reporting for assessment, point to the discrepancy between students’ definition of competency in critically evaluating electronic resources and medical educators’ expectations.

Proposed Learning Objectives for Web Competencies

To be conversant with all of the issues involved in information acquisition over the Web, physicians need to understand its underlying structure and how information is communicated. They need facility with the tools for acquiring and filtering information and must be able to critically analyze the content. A key component of these learning objectives involves patient education and how physicians can leverage Web use to improve communication. Our proposed basic Web competencies and their associated learning objectives are listed in Table 2; these are explained in more detail below.

Table 2
Table 2:
Proposed Curricular Model for Basic Web Competencies and Their Associated Learning Objectives

Structure and Function of the Web

The World Wide Web is complex in both its structure and function. Understanding the fundamental structure, with the use of domains and domain types, will enable the user to employ a first critical element in filtering. Some domain types have inherent controls. Government sites attempt to protect the world's health while facilitating access to high-quality information. Educational sites facilitate access to knowledge while protecting freedom of speech within loosely monitored faculty and student Web environments.

In addition to the Web as a network of knowledge, it is also a marketing tool. E-commerce is growing rapidly. Commercial Web sites promote products and services, but they can also contain useful information. Internet service providers exercise the least amount of control but are responsible for a number of patient support chat rooms. Domain and domain types (e.g., .com, .gov, .edu) should not automatically exclude information, but should frame the reference standards through which the information is viewed.

Web Tools to Access Health Information

There are many tools on the Web that provide access to health information. The least effective of these are the Web search engines, such as Google, Alta Vista, and others. These search engines can, however, provide a starting point, especially for information problems needing a less traditional approach. Most Web search engines have generic and advanced search functions. Facility with the advanced search tools can provide a more efficient and effective search with a greater likelihood of finding needed information.

Beyond the use of Web search engines, knowing how to use tools employing a Web front end, such as those that provide access to the U.S. National Library of Medicine's databases or those that facilitate entry into online educational environments, are essential skills for the information-astute physician.

Quality Filtering of Web-Based Information

Students in many medical schools are taught how to critically analyze research articles as part of evidence-based medical practice. Similar skills should be applied to the critical review of Web sites. In March 2000, the American Medical Association developed and published a list of guidelines for association Web sites containing medical and health information. These guidelines provide a basis for quality filtering of Web sites.11 A similar set of guidelines was recently published on the Web by the American Accreditation HealthCare Commission.12 A summary of the topics of both these sets of guidelines is found in List 1; ready knowledge of these criteria should be part of the armamentarium of the information-astute physician.

List 1 Topic Areas of World Wide Web Standards Created by the American Medical Association and the American Accreditation HealthCare Commission
List 1 Topic Areas of World Wide Web Standards Created by the American Medical Association and the American Accreditation HealthCare Commission:
List 1 Topic Areas of World Wide Web Standards Created by the American Medical Association and the American Accreditation HealthCare Commission

Patient Issues Concerning the Web

The Web offers unique opportunities and unique challenges to the physician–patient relationship.13 Patients get a substantial amount of health care information from the Web, but a substantial amount of it is inaccurate. It is known that patients lack knowledge of techniques to critically review Web sites they select for information; even in cases where they have that knowledge, they sometimes do not apply it.14 Physicians need to develop the skills necessary to review Web-based information brought to them by patients. They also need to counsel patients regarding the accuracy of Web information, based on standard criteria for analysis. This process could, with appropriate skill, significantly enhance both the physician–patient relationship and the education of the patient. It could also offer the opportunity to discuss issues of patient confidentiality and the Web. Information-astute physicians must extend skills in this area to their patients.

Evaluation of Content and Tools

Evaluation is central to ensuring that learning objectives are met. Evaluation of the objectives for safe medical information practice poses particular challenges, given the vastness and variability of information on the Web. These challenges can lead to either absent or informal evaluation, or evaluation that is separated from the “real” content of medicine. There is a need for systematic, reproducible assessment models for these competencies that can complement other methods of assessment. We developed a prototype of such an assessment method and evaluated it at two institutions.15 The project, funded by the National Board of Medicine Examiners, developed scoring criteria for various information acquisition and application skills in clinical problem solving. The Web component required students to identify appropriate Web sites to address specific information problems and to formulate responses to patients based on the findings. The researchers manually scored the sites selected using some of the criteria shown in List 1.

Automating scoring of student performance would make such assessments easier to administer. However, issues involved in defining discrete variables for automated scoring are as dynamic as the Web itself. Domain types are being added; Web-filtering parameters are becoming varied based on type of information supported; and information security and confidentiality are constantly challenged by changes in networking and operating system architectures. However, even if a large-scale examination that required automated scoring were not feasible, assessing this component of the curriculum along with other key content areas could serve to underscore its importance.

Summing Up

The World Wide Web continues to grow and offers information of varying quality to both physicians and patients. The complexity of the Web continues to hinder the development of automated tools to provide any but the most rudimentary level of filtering. Physicians and physicians in training need to learn to apply a knowledge of Web structure and function to targeted information acquisition. They also need to skillfully use Web-based search tools and standardized filtering techniques to identify high-quality information resources, and they should counsel patients about the benefits and risks of using the Web for personal health decision making. To ensure that medical students develop the knowledge and skills necessary to become information-astute physicians, medical schools must accept responsibility for this learning, develop defined, Web-focused learning objectives, promote responsible use of Web-based information, and evaluate the success of these endeavors in the continually changing Web environment.


1. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et wiewor—let the reader and viewer beware. JAMA. 1997;277:1244–5.
2. Frisse ME. What is the Internet learning about you while you are learning about the Internet? Acad Med. 1996;71:1064–7.
3. Sweeney L. Guaranteeing anonymity when sharing medical data, the Datafly system. In: Masys DR (ed). The Emergence of ‘Internetable’ Health Care. Systems That Really Work. Proceedings of the 1997 AMIA Annual Fall Symposium, 1997 Oct 25-29, Bethesda, MD. Philadelphia: Hanely and Belfus, 1997:51-5.
4. Jadad A, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA. 1998;279:611–4.
5. Shon J, Musen MA. The low availability of metadata elements for evaluating the quality of medical information on the World Wide Web. In: Lorenzi NM (ed). Transforming Health Care Through Informatics: Cornerstones for a New Information Management Paradigm. Proceedings of the 1999 AMIA Annual Symposium, 1999 Nov 6-10, Washington, DC. Philadelphia: Hanely and Belfus, 1999:945-9.
6. Price SL, Hersh WR. Filtering web pages for quality indicators: an empirical approach to finding high quality consumer health information on the World Wide Web. In: Lorenzi NM (ed). Transforming Health Care Through Informatics: Cornerstones for a New Information Management Paradigm. Proceedings of the 1999 AMIA Annual Symposium, 1999 Nov 6-10, Washington, DC. Philadelphia: Hanely and Belfus, 1999:911-5.
7. Mostafa J, Quiroga L, Palakal M. Filtering medical documents using automated and human classification methods. J Am Soc Inf Sci. 1998;49:1304–18.
8. Medical Informatics Advisory Panel and Population Health Perspective Advisory Panel. Contemporary issues in medicine: medical informatics and population health. Report II of the Medical School Objectives Project. Acad Med. 1999;74:130-41.
9. Scott CS, Schaad DC, Mandel LS, Brock DM, Kim S. Information and informatics literacy: skills, timing, and estimates of competence. Teach Learn Med. 2000;12:85–90.
10. Association of American Medical Colleges. 2002 Medical School Graduation Questionnaire. All School Report. Washington, DC: AAMC, 2002.
11. Winker MA, Flanagin A, Chi-Lum B, et al. Guidelines for medical and health information sites on the Internet: principles governing AMA web sites. JAMA. 2000;283:1600–6.
12. URAC. American Accreditation HealthCare Commission. Health Web site standards 〈〉. Accessed 1 April 2003.
13. Jadad AR. Promoting partnerships: challenges for the Internet age. BMJ. 1999;319:761–4.
14. Eysenbach G, Kohler C. How do consumers search for and appraise health information on the World Wide Web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324:573–7.
15. Berner ES, McGowan JJ, Hardin JM, Spooner AS, Raszka WV, Berkow RL. A model for assessment of information retrieval and application skills. Acad Med. 2002;77:547–51.
© 2004 Association of American Medical Colleges