Western University, Canada - London Ontario

| Tuesday October 21, 2014
4:45 PM

Evidence Based Neurology:
An Innovative Curriculum for Post-Graduate Training in the Neurological Sciences

Bart M. Demaerschalk, MD, MSc, FRCPC
Samuel Wiebe, MD, MSc, FRCPC
Mayo Clinic Scottsdale, USA and Western University
Copyright 2001, Western University

Contents:

INTRODUCTION 

Evidence based medicine (EBM) is the conscientious, explicit, and judicious use of current best external evidence in making decisions about the care of individual patients.[1] Practicing EBM requires the integration of individual clinical expertise, proficiency, judgment, and compassion with the best available evidence from systematic and relevant basic science and clinical research. The practice of EBM is really life-long and self-directed learning in health care.

Examining evidence from clinical research in the neurological sciences is important for the following reasons: (1) despite neurological practice having its history deeply rooted as a rich descriptive discipline, it has shifted to one of increasing diagnostic and therapeutic interventions. (2) Clinicians continue to face growing pressure to justify their decisions in practice with scientific evidence. (3) Wide variations exist in neurological practice due largely to sub-optimal interpretation and application of the best external evidence. (4) The explosion in biomedical literature makes it very difficult for clinicians to keep updated.

Practicing EBM requires unique skills that allow clinicians to identify knowledge gaps, formulate questions, search the literature, appraise, apply it to their patients, and keep track of their progress. These skills must be taught to clinicians in training through medical education.

In this article, an innovative curriculum for evidence based neurology (EBN) will be presented. The whole of EBN in the current context of postgraduate neurological training will be defined. Current problems and obstacles impeding EBM in the neurosciences will be outlined. An innovative proposal to address solutions is described. The design, architecture, development, implementation strategy, impact, and evaluation of the curriculum will be revealed.

PUTTING THE NEW CURRICULUM INTO CONTEXT 

Neurological nodes can be viewed as consisting of two large domains, i.e., background and foreground learning. The background is occupied by the basic neurosciences, neuroanatomy, neurophysiology, neuropharmacology, and neuropathology. Foreground learning refers to clinical decision-making, history taking, examining, diagnosing, and therapeutically intervening, and is the niche of EBM. With time, foreground learning occupies a larger portion of a clinician continuing education, but both play complimentary rather than mutually exclusive roles.

Despite the proliferation of clinical trials and a growing base of biomedical literature, many physicians on the front lines of patient care do not seek, appraise, or apply this external evidence.[2,3,4] Perhaps this is because of lack of access, lack of special skills, or lack of acceptance.[5,6,7,8,9] In response to these needs, medical educators, advisory groups (Accreditation Council for Graduate Medical Education), Association of American Medical Colleges, and the Royal of College of Physicians and Surgeons of Canada have called for the introduction of clinical epidemiology, biostatistics, critical appraisal, and medical informatics into medical school and post-graduate medical education curricula.[10,11,12]

EBM teaching and practice vary widely in amount, depth, and structure among Canadian postgraduate neurology training programs.[13] Canadian neurology program directors, teaching neurologist, and trainees surveyed indicate that EBM is considered highly relevant, valuable, and necessary in the training and practice of neurology, however, self-reported proficiency in EBM principles is low and no formal educational EBM neurological curricula exists.[14]

There was a perceived need for EBM training in the neurological sciences because existing proficiency was low. Educational bodies began to demand the introduction of such training in undergraduate and postgraduate medical education. Therefore, the ground was fertile for the development and implementation of the first evidence-based medicine curriculum in neurological training.

OBSTACLES TO OVERCOME 

Despite a consensus on the need to cultivate EBM skills in neurological practitioners, such training at the postgraduate level has proven difficult. Problems have been identified in the following realms: learning outcomes, learner needs, resources, content, teaching and learning strategies, and evaluation.

Most existing journal clubs and critical appraisal sessions in postgraduate neurological training do not have explicit intended learning outcomes. This suggests that neither the teachers nor the trainees really have an idea about what the objectives are. Learners are rarely consulted regarding their particular and unique needs in neurological training, but trainees surveyed at Western University indicate that lack of time and lack of teaching neurologists interest are the two main obstacles to effective learning and practice of EBM.[15] Self-reports from trainees demonstrate low proficiency in EBM principles and application to neurological practice.[16]

There was no neurological training program, which included EBM teaching in a systematic fashion, and therefore, neurological educators have no directly applicable model to adopt. Although multiple examples of instruction and exercises in critical appraisal exist at the post-graduate medicine level, this training largely takes place in the form of journal clubs or literature-based curricula.[17,18,19,20,21,22] There is a paucity of published literature describing comprehensive EBM curricula in internship in residency training. Only in the disciplines of internal medicine, community family medicine, and physiatry are there publications in this arena.[23,24,1]

Resources other than helpful literature are also lacking. There is a lack of faculty participation in EBM at neurological training institutions. Opposition to EBM still exists. There is a missing infrastructure and environment necessary for effective EBM practice. Trainees often lack a compilation of EBM resources, an EBM library, a resource room, computers, software, and informatic support. Establishing even the bare minimum for success is expensive. Securing a financial commitment from neurology departments, hospitals, universities, and government is challenging. At most neurological training centers, faculties do not have formal training in necessary prerequisites such as clinical epidemiology, biostatistics, and teaching EBM.

Content of traditional journal club and clinical appraisal exercises are inadequate. They are literature driven and not patient or problem focused. There is no systematic development of clinical content. Topics are generated merely because the available literature is handy or because it is the interest of a single trainee or faculty member. There is little emphasis on training in informatics, database searching, and librarian skills. Evidence from one source is rarely compared to another. The sessions promote anilism and "paper bashing" by over-emphasizing a strict appraisal of study methodology in accepting nothing short of the perfect randomized, controlled trial. Teaching critical appraisal skills, although essential, is not sufficient. As important, are the principles of beginning with a clinical problem, formulating an answerable question, searching the literature, interpreting the evidence, establishing bottom lines, applying the evidence to the original patient or problem, and effectively collecting and storing the work. Existing journal clubs and appraisal exercises are given a low priority in the current neurology educational hierarchy with didactic lectures still holding a disappointingly high position. Critical appraisal exercises are viewed as an add-on as opposed to a scheduled component of training.

A systematic review on effectiveness of teaching critical appraisal skills reports that such training can result in significant gains in knowledge of epidemiology in undergraduate programs, but conversely, the evidence to date indicates that teaching such skills in residency programs is surprisingly ineffective.[26] The authors surmise that the difference may lie in the trainee evaluation system. While in clerkships, it is possible to ensure that part of course credit is based on performance in critical appraisal, existing residency programs do not link performance in EBM with overall trainee evaluation. Although educational bodies have recommended EBM training in postgraduate medical education, the same bodies have not yet emphasized these skills at the level of the fellowship examinations. These differences are critically important because of evaluation has been frequently identified as a major determinate of learning.[27] The authors postulate that the integration of EBM as an essential and continuing component of a residency training program will be preferable to the "add-on" nature of the reported interventions to date. Finally, another problem in the area of assessment is the lack of existing evidence supporting the notion that gains in EBM knowledge demonstrate at the undergraduate or postgraduate level can be sustained into practice and eventually translated into improved patient care.

THE INNOVATIVE CURRICULUM PROPOSAL 

Needs Assessment

Based on a review of the current literature, independent pre-curriculum surveys of neurology program directors, teaching neurologists, and neurology trainees, and experienced practicing and teaching EBM in neurology over the last five years, the key curriculum needs are: (1) explicit-intended learning outcomes; (2) increased proficiency in the principles and practice of EBM; (3) faculty-qualified, able, willing, and rewards in teaching practice EBM; (4) resources including time, physical space, and tools necessary for EBM; (5) alternative and nontraditional sources of funding; (6) systematically developed content; (7) integration with other elements of neuroscience training; (8) better transposition of EBM skills from the classroom to the bedside; (9) novel teaching strategies and more learner-centered education; and (10) evaluation scheme and assessment tools.

OVERALL PURPOSE 

By teaching neurology residents to practice EBM, the global purpose is to foster life-long self-teaching, self-evaluation, and to promote improvement of the care they provide to their neurological patients.

INTENDED LEARNING OUTCOMES 

Skills

By the end of postgraduate neurological training, residents will be able to: (1) constructive focused, answerable question when faced with uncertainty in a clinical scenario; (2) devise a strategy and conduct efficient literature searches on a variety of available databases; (3) critically appraise a published report of a clinical study; (4) apply the appraised evidence to their clinical problem; and (5) incorporate the evidence into their decision-making for individual patients.

Attitudes

By the end of postgraduate neurological training, residents will: (1) appreciate the advantages of practicing EBM; and (2) find the EBM curriculum to be a valuable educational experience.

Behavior

By the end of postgraduate neurological training, residents will: (1) increase their use of evidence from clinical research to help solve the neurological problems they may encounter; and (2) continue to practice and teach EBM throughout their neurological careers.

Knowledge

By the end of postgraduate neurological training, residents will have: (1) acquired knowledge of the most current best evidence for neurological practice in all the various sub-disciplines of the clinical neurosciences; and (2) the skills (see above), to update this knowledge, as it evolves, over the time span of their careers.

RESOURCES 

Financial

In addition to seeking support from traditional sources (Department of Clinical and Neurological Sciences, Hospital, University) funding was also sought from granting agencies including the Educating Future Physicians of Ontario (EFPO). Additionally, proposals were drafted and delivered to multiple interested pharmaceutical companies. Pharmaceutical funding is only accepted if it is in the form of unrestricted educational grants. To ensure ethical practice, guidelines for pharmaceutical company sponsorship of evidence based neurology sessions are written and approved by key participants (program director, postgraduate educational committee, and EBM faculty).

Faculty

There is only one full-time teaching neurologist in the department qualified to teach evidence-based medicine, clinical epidemiology, and biostatistics. One fellow, completing graduate training in medication education and clinical epidemiology and biostatistics, helped to facilitate the evidence-based neurology sessions. Interested senior informer residents, having been exposed to evidence based medicine training and practice in the program, become highly skilled and have a graduated level of responsibility regarding the teaching role. Affiliations are formed with faculty outside the neurosciences with expertise in various related spheres of study including medical library sciences, epidemiology and biostatistics, and health economics. Teaching physicians through other departments, for example, internal medicine, with EBM training have also agreed to participate. The evidence-based neurology trainees vary in number from a total of 10-14 each year.

Time

The EBN tutorial sessions are one hour in duration, held twice monthly, twelve months per year, throughout the five-year training program. The sessions fit into protected educational half-day time for neurology trainees. Preparatory time for trainees and faculty is informal and unscheduled.

Space

The twice-monthly tutorials are held in an available hospital conference room. There is also one room, named Evidence-Based Neurology Resource Room, which houses the evidence-based medicine library, computers, links to require databases, all necessary software, in addition to desks, tables, book shelves, and chairs.

Educational Material

A resource binder containing the most valuable evidence-based medicine literature is compiled and distributed to the trainees. A compilation of critically appraised topics are processed and printed in pocket size form to allow trainees to have easy access to the fruits of their labor during subsequent and clinical encounters.

Computer and Informatics

There is a dedicated personal computer and printer for the evidence-based neurology resource room. Additionally, faculty has access to a laptop computer and projection unit to enable facilitators to use Power Point technology during presentations and teaching sessions. At least once per year, a university computer laboratory is reserved for training sessions with medical librarians concerning medical literature database searching.

Content

The clinical topic content (24 topics per year) is annual assembled by first surveying all neurology trainees and teaching neurologists in the department and having them list what they consider to be clinically important and relevant neurological themes or topics. These are then rank ordered by EBM participants and faculty according to frequency of occurrence, level of relevance, clinical importance, amount of interest and enthusiasm, and question answer ability. The 24 most highly ranked topics become the basis for the clinical evidence-based neurology content for the upcoming academic year. The topics are screened additionally to ensure that they fit with educational recommendations of the training program (postgraduate education committee) and Royal College of Physicians and Surgeons of Canada Advisory Committee. The content is flexible and modifications are made periodically to better satisfy trainee needs. Annual and semiannual meetings of participants are held to review what has been done and what still needs to be addressed. The entire curriculum content is designed to be responsive to results of evaluation and feedback. The 24 topics were equally divided amongst the neurology trainees, and schedules indicate dates, times, locations, names of topics, responsible resident, and faculty facilitator.

Pre-tutorial

  • Responsible resident chooses a personal clinical experience, which falls under the category indicated by the scheduled topic.
  • A focused clinical question is constructed based on a real patient encounter.
  • A literature search strategy is developed to best answer this question.
  • The literature search is conducted and refined.
  • Available literature screened according to predefined criteria.
  • The 1-4 best pieces of literature of selected.
  • Copies of the clinical scenario, question, search strategy, and literature are circulated to all trainees and faculty facilitators at least one week in advance of tutorial.
  • Opportunity exists for resident and facilitator to meet and prepare for tutorial.
  • If time allows, trainees attempt independent searches and compare results.
  • All trainees arrive at tutorial prepared to participate.

60-Minute Evidence-Based Neurology Tutorial Format

5 minutes : Presentation and discussion of clinical scenario and focus question

5 minutes : Presentation and analysis of literature search strategy and yield

20 minutes : Critical appraisal of the evidence

15 minutes : Interpretation of results and application to the case at hand

5 minutes : Define the clinical bottom lines

5 minutes : Creating the critically appraised topic

5 minutes : Evaluation

Post-tutorial

The evidence-based neurology facilitator and resident edit and submit the final CAT product to the hospital graphics department for publication and circulation in print and electronic form.

Inter Tutorial

All trainees are encouraged to use their acquired evidence-based medicine practice skills in other clinical venues, including clinics, wards, rounds, teaching sessions, and in dialogue with their peers and the teaching neurologists. In order to facilitate this, neurology faculty must be "on board" with the development of evidence-based medicine in neurology teaching and practice.

TEACHING AND LEARNING STRATEGIES 

Adult Learning Theory

In as many ways as possible, the educational strategy strives to satisfy the assumptions that underlie adult learning theory.[28,29,30] Because adult learners need to know why they need to learn something before they undertake learning it, evidence supporting the practice and teaching of evidence-based medicine is presented. Because adult learners prefer responsibility for their decisions and desire self-direction, the trainees play instrumental roles in the development of learning outcomes, assembly of educational content, facilitation of teaching sessions, education of their peers, evaluation, storage, and updating of their work. They also participate in the ongoing modifications of the curriculum itself. Individualization of learning and teaching strategies is endorsed to allow for diversity of learners. Because adults become ready to learn things when they need to know them in order to cope effectively with real-life situations, the trainees own clinical experiences serve as the substraight for the clinical topic content. Adult learners thrive in an environment of life-centered learning as opposed to subject-centered learning, and therefore, the EBM sessions are constructed around real clinical scenarios instead of traditional neurological topics. Adults most potent motivators for learning are internal, and therefore, external dictatorial approaches are discouraged.

Favorite Approaches of Evidence-Based Medicine Teachers

Literature reviews of approaches to EBM training and personal communication with numerous evidence-based medicine educators reveal several attributes favored by teachers of EBM: small group, learner centered format for seminars, one-on-one residents-faculty opportunities, immediate clinical relevance, residents as both educators and learners, integration of EBM into mainstream clinical work, and faculty role modeling of EBM. Examples of all of these approaches are illustrated in this EBM curriculum.

Collaborative Learning

The five basic elements of collaborative learning are heavily incorporated in the EBM curriculum. Trainees act as both learners and educators and each is responsible for segments of the overall content (positive interdependence). The educator trainee must ensure that each of the learner trainees receive all of the necessary material, that there is appropriate preparedness for a session, and that the work is appropriately summarized and stored (individual accountability). The sessions are held in small groups and seating is around a table to ensure face-to-face interaction amongst participants. The same group of core neurology trainees (approximately 12), meet regularly over the entire training period to promote interpersonal relations. Within the larger group, often, small group breakout sessions will ensue to accomplish complimentary but independent task learning (group process). Research reveals that, in contrast to learning only 50% of what we see and hear (the traditional didactic lecture), we learn 95% of what we teach to others in collaborative learning.[31]

To facilitate collaborative group work in the EBM tutorial sessions, participants are sometimes assigned roles to play. This works well when there are two or more papers to critically appraise in the allotted time period. The group of 12 divide into two groups of six. Most often there are two EBM faculty facilitators in attendance. Each subgroup member plays a role. (1) process person makes sure everyone contributes to the discussion, (2) gofer sees that all necessary supplies (articles, appraisal checklist, pens, paper, flip chart, etc.) are available, (3) scribe records results of literature appraisal, bottom lines, and critically appraised topic, (4) timekeeper watches the clock, (5) relevance monitor brings participants attention on task and discourages side track discussions, (6) reporter verbally summarizes the article, appraisal, application to the case at hand, and clinical bottom lines to the larger group.

Cooperative Learning

Group dynamics, described above adopted jigsaw cooperative learning approach whereby a volume of material prescribed for any individuals session may be cut into sizes made more manageable for smaller groups of individuals who then complete the assigned task and contribute their piece to the group at large.[32] This method places responsibility of learning and teaching on trainees shoulders, de-emphasizes competition, makes the goal information sharing and promotes active learning.

What makes cooperative learning distinct from group work is that there is an atmosphere of interactive learning, a set of learning objectives which are social and affective in addition to academic, a theme of cooperation, and a central role for communication,[33] cooperation outperforms other instructional modes for higher and lower order thinking.[34]

Creating Teachable Moments

By means of pre-tutorial questionnaires, an effort is made to non-threateningly uncover trainees misconceptions, misunderstanding, or ignorance of a clinical topic in neurology with the goal of creating a teachable moment and enhance learning.[35]

Multimedia Instruction and Informatics

A wide array of media are utilized for instructive and facilitative purposes from the simple (chalkboard, flip chart) to intermediate (overhead and slide projector) through to the complex (projected Power Point presentations via laptop and web-based medical literature database searching in computer laboratories). Technology is only chosen if it will fulfil objectives to enhance learning or compliment the tutorial.

Evaluation

To non-threateningly uncover and determine trainees misconceptions, misunderstanding, or ignorance of a clinical topic in neurology, a pre-tutorial questionnaire is utilized. Questions include: (1) how often do you see this problem; (2) how confident are you in managing this problem; (3) are you confident that you are using the best available evidence for this problem; (4) how familiar are you with the evidence on this topic; (5) how valid do you believe the evidence to be on this topic. Questions are answered using 7-point Likert scales.

To evaluate trainees learning and application of evidence-based medicine principles in neurology, again using 7-point Likert scales (1 = very poor/low/little, and 7 = very good/high/much), trainees self rate the following after each session: question formulation/methods/efficiency, results of literature search, familiarity with EBM principles, confidence in knowledge of existing evidence, ability to apply evidence to practice, change in prior clinical concepts and foreseeable practice. Preliminary data reveal that ratings are high/good for question formulation and literature searching and moderate-to-high for familiarity with EBM principles, confidence in knowledge of existing evidence, ability to apply evidence, and changes in concepts in foreseeable practice.[36]

Opportunities for trainee evaluation of EBM faculty, resources, and tutorial format are necessary. Although overall objectives are being evaluated currently by anonymous trainee self-rating questionnaires, there is room to introduce both formative and summative evaluation for individual trainees. These results can then be utilized by the program director and postgraduate education committee and link to overall trainee evaluation.

IMPLEMENTATION STRATEGIES 

  • Faculty development of EBM workshops
  • Boosting moral and reducing skepticism of EBM
  • Ensuring the trainee participation is mandatory
  • Ensuring the trainee participation and performance is assessed and will form a component of overall trainee evaluation
  • Royal College of Physicians and Surgeons of Canada (Neurology) examine committee to include evidence-based medicine component.
  • Offering national EBM workshops for neurology program directors and residents
  • Collaborate with 1-2 people at each of the Canadian Neurology Training Centers who are interested in order to develop a national critical mass.
  • Offer EBM workshops at national and international neurology meetings
  • Recruit new neuroscience faculty with training in EBM.

CONCLUSION 

The direction toward which postgraduate neurological education must move is clear. Ensuring that residents acquire the skills to articulate answerable questions, search and appraise literature, and interpret and apply the results to the care of individual patients is essential. This curricula for the introduction of EBM into neurological training is clearly a work in progress. A curriculum, by its very nature, is a never-ending winding path. This project voices the current context of postgraduate training in the neurological sciences, the problems faced by previous EBM curricula innovators, new solutions for overcoming the majority of recognized barriers, and a detailed description of its purpose, learning outcomes, required resources, content, teaching strategies, and assessment tools. Just as it may take ten years for results of clinical research to find their way from the literature to routine use at the bedside, the diffusion of educational innovation is recognizably slow.

REFERENCES 

  • Sackett, D.L. et al. (1997). Evidence-Based Medicine, How to Practice and Teach EBM. London: Churchill Livingstone.
  • Ayanian, J.Z. et al. (1994) Knowledge and Practices of Generalist and Specialists Physicians Regarding Drug Therapy for Acute MI. NEJM Vol. 331, 1136-42.
  • Brand, D.A. et al. (1995). Cardiologists' Practices Compared with Practice Guidelines: Use of Beta-Blockade after Acute MI. J Am Coll Cardiol Vol.26, 1432-6.
  • Soumerai, S.B. et al. (1997). Adverse Outcomes of Under use of Beta-Blockers in Elderly Survivors of Acute MI. JAMA Vol. 377, 115-21.
  • Covell, D.G. et al. Information Needs in Office Practice: Are They Being Met? Ann Intern Med Vol. 103, 586-9.
  • Osiobe, S.A. et al. (1985). Use of Information Resources by Health Professionals: A Review of the Literature. Soc Sci Med Vol. 21, 965-73.
  • McKibbon, K.A. et al. (1990). How Good are Clinical MEDLINE Searches? A comparative Study of Clinical End-User and Librarian Searches. Vol. 23, 583-93.
  • Weiss, S.T. et al. (1980). An Assessment of Physician Knowledge of Epidemiology and Biostatistics. J Med Educ Vol. 55, 692-7.
  • Berwick, D.M. et al. (1981). When Doctors Meet Numbers. Am J Med Vol. 71, 991-8.
  • Barondess, J.A. (1981). The Future Physician: Realistic Expectations and Curricular Needs. J Med Educ Vol. 56, 381-9.
  • Rafuse, J. (1994). Evidence Based Medicine Mean M.D.s Must Develop New Skills and Attitudes, CMA Conference Told. CMAJ Vol. 150, 1479-80.
  • Accreditation Council for Graduate Medical Education. (1996-97). Program Requirements for Residency Education in Internal Medicine: Special Educational Requirements. In: The Graduate Medical Educator Directory. Chicago, Il: AMA.
  • Wiebe, S., and Demaerschalk, B. (1998). Evidence Based Neurology Comes of Age: Introduction to Residency Programmes. Canadian Journal of Neurological Sciences, Vol. 25, (Suppl 2):S80 Abstract.
  • Demaerschalk, B., Wiebe, S. (1998). Evaluating the Relevance of Evidence Based Medicine in a Neurology Residency Programme. Canadian Journal of Neurological Sciences, Vol. 25(Suppl 2):S79-S80 (Abstract).
  • Linzer, M. Et al. (1986). Role of a Medical Journal Club in Residency Training. J Med Educ Vol. 61, 471-3.
  • Sidorow, J. (1996). How are Internal Medicine Residency Journal Clubs Organized and What Makes Them Successful? Arch Intern Med Vol. 155, 1193-7.
  • Woods, J.R. et al. (1982). Journal Club Format Emphasizing Techniques of Critical Reading. J Med Educ Vol. 57, 799-801.
  • Kitchens, J.M. et al. (1989). Teaching Residents to Read the medical Literature: A Controlled Trial of Curriculum in Critical Appraisal/Clinical Epidemiology J Gen Intern Med Vol. 4, 384-7.
  • Gehlbach, S.H. et al. (1980). Teaching Residents to Read to Medical Literature J Med Educ Vol. 55, 362-5.
  • Inui, T.S. (1981). Critical Reading Seminars for Medical Residents: Report of a Teaching Technique Med Care Vol. 29, 122-4.
  • Green, M.L. and Ellis, P.J. (1997). Impact of an Evidence-Based Medicine Curriculum Based on Adult Learning Theory. J Gen Intern Med Vol. 12, No. 12, 742-50.
  • Kenney, A.F. et al. (1998). Introducing Evidence-Based Medicine into a Community Family Medicine Residency. J Miss State Med Assoc. Vol. 39, No. 12, 441-3.
  • DeLisa, J.A. et al. (1999). Evidence-Based Medicine in Physiatry: The Experience of One Department's Faculty and trainees. Am J Phys Med Rehabil. Vol. 78, No. 3, 228-32.
  • Norman, G.R. and Shanon, S.I. (1998). Effectiveness of Instruction in Critical Appraisal (Evidence-Based Medicine) Skills: A Critical Appraisal. CMAJ, Vol. 158, No.2., 177-181.
  • Swanson, D.B., Norman, G.R., and Linn, R.L. (1995). Performance-Based Assessment: Assessment: Lessons From the Health professions. Educ Res, Vol. 24, 5-11.
  • Knowles, M. (1984). The Adult Learner: A Neglected Species. Houston, TX: Gulf Publishing Co.
  • Brookfield, S.D. (1995) Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass Publications.
  • Cross, K.P. (1981). Adults as Learners. San Francisco: Jossey-Bass
  • Glasser, W. (1986). Control Theory in the Classroom. New York: Perennial Library.
  • Fortner, R. (1999). Using Cooperative Learning to Introduce Undergraduates to Professional Literature. Journal of College Science Teaching. 261-265.
  • Lee, C. et al. (1998). Co-operative Learning in the Thinking Classroom: Current Research. Educational Practice and Theory, Vol. 20, No. 1, 59-73.
  • Rolheiser-Bennet, N. (1986). Four Models of Teaching: A Meta-Analysis of Student Outcome, Unpublished Ph.D. dissertation, University of Oregon, Eugene.
  • Hansen, E.J. (1998). Creating Teachable Moments and Making Them Last. Innovative Higher Education, Vol. 23, No. 1, 7-26.
  • Demaerschalk, B., Wiebe, S., and Jenkins, M. (1999). Evaluating the Impact of an Evidence Based Medicine Curriculum in a Neurology Training Programme. Canadian Journal of Neurological Sciences Vol. 26, (suppl 1):S52 (Abstract).
Wai P. Ng