Defning competence for the purposes of certifcation by the College of Family Physicians of Canada: The evaluation objectives in family medicine 2630 Skymark Avenue, Mississauga ON L4W 5A4 2630, avenue Skymark, Mississauga ON L4W 5A4 905 629•0900 Fax 905 629•0893 www.cfpc.ca 905 629•0900 Téléc 905 629•0893 www.cfpc.ca Defining competence for the purposes of certification by the College of Family Physicians of Canada: The evaluation objectives in family medicine Working Group on the Certification Process ©Copyright 2010 The College of Family Physicians of Canada (CFPC) Contents Preface ............................................................................................................................................ 1 Acknowledgements ......................................................................................................................... 2 Executive Summary ......................................................................................................................... 3 Part I—The evaluation objectives: what are they, where do they come from, and how can they be used? .......................................................................................................................................... 7 Part II—The evaluation objectives for daily use: the operational level, for assessing competence ....................................................................................................................................................... 31 Bibliography ................................................................................................................................ 163 Preface The evaluation objectives presented here will guide the College’s Board of Examiners in the choice of criteria for the testing and certification of competence in family medicine, during both in- training evaluation and the Certification Examination in Family Medicine. They will serve to ensure that the certification decisions maintain acceptable validity and reliability. To do this, the evaluation objectives have been designed to clearly describe the domain of competence, as well as the skills and behaviours to be tested within each topic and within each dimension of competence. The topics and essential skill dimensions of competence were initially identified through surveys of family physicians in practice in different settings and communities across Canada. Further definition of the evaluation objectives was done by a series of focus groups, using various structured approaches and methods, both qualitative and quantitative. A detailed description of this process is presented here. This document will be added to and enhanced over time, but the most current working document will always be posted and available on the College’s website for the reference of teachers and learners. You can view the different components of the evaluation objectives by following the links provided on this page. We have posted a searchable key feature database of the priority topics and key features, cross- referenced to the skill and phase dimensions. A more usable database is being prepared, but while you are waiting, this current document will let you search and select key features according to the skill dimensions and phase dimension—this is particularly useful when you wish to concentrate on certain skills or phases. If you wish to see the key features clearly, individually or by topic, go directly to the Priority Topics and Key Features for Assessment in Family Medicine. 2010 1 | P a g e Acknowledgements The College of Family Physicians of Canada would like to acknowledge the many members who have contributed to the development of this comprehensive set of evaluation objectives. These individuals have participated in various ways, at various times. The Board of Examiners, the Committee on Examinations, and the Working Groups on the Certification Process and on Procedure Skills have played major roles. About 500 members helped by responding to the surveys that were done to establish and validate our basic priorities for the assessment of competence to practice family medicine. Many others in our community have provided insight and feedback, and continue to do so, as the components of these objectives have been presented and tried “in the field” across the country. The principal authors of this report are: Tim Allen, Cheri Bethune, Carlos Brailovsky, Tom Crichton, Michel Donoff, Tom Laughlin, Kathrine Lawrence, Stephen Wetmore. 2 | P a g e 2010 Executive Summary In 1998, the Board of Examiners started a review of the certification process with particular emphasis on developing a competency-based definition of family medicine for the purposes of assessment. There had been challenges, both internal and external, to the significance of certification from the College (the CCFP): what does it mean; is it still necessary? The Four Principles of Family Medicine provided a framework to describe the practice of family medicine in Canada, describing the broad and complex role of the family physician, and informing our understanding of how a competent family physician should function. They did not provide, however, specific details on how the family physician achieves and demonstrates this competence. The Board of Examiners decided to develop a new definition of competence for the purposes of assessment, and chose to ground the definition in the experience of practicing family physicians. It began the process with a postal questionnaire asking how family physicians would define competence at the start of independent practice. This led to a series of working groups tasked with further developing the components of this definition, with the aim of establishing competency-based evaluation objectives for the certification process. This document is the ultimate result of that undertaking. It is not intended to redefine the specialty of family medicine in Canada. Rather, it articulates, for the purposes of assessment, the specific skills required by physicians to effectively function within the context of family medicine in Canada. Part I of the document describes the components and structure of these evaluation objectives, explaining and justifying the methods used in their development. Part II lays out, in detail, the operational level of all the evaluation objectives. These objectives do not contain test instruments, examinations, scoring scales, performance levels, or forms, or any prescription for developing these tools. This omission is intentional. As these tools are developed, these evaluation objectives will direct both the content and the format of the instruments used and increase the chances that any evaluation will perform well according to the characteristics of successful evaluations. This summary concludes by providing a brief outline and description of the main components of the evaluation objectives and this definition of competence in family medicine. For more detailed information, go to the appropriate sections in Parts I and II of the main body of the document. 2010 P a g e | 3 The evaluation objectives in family medicine The Domain of Competence in Family Medicine: There are four major components in this definition: 1. Skill Dimensions of Competence 2. Phase of the Clinical Encounter 3. Priority Topics, Core Procedures, and Themes 4. Key Features and Observable Behaviours Together they constitute the domain of competence in family medicine. Overall competence is determined through a process of continuous sampling, observation, and reflection of learner performance with respect to key features and observable behaviours for a series of problems (priority topics, procedures, themes), throughout the phases of the clinical encounter, until evaluators are satisfied that the physician is competent in all the skill dimensions. The Components of the Evaluation Objectives: 1. Skill Dimensions of Competence: There are six essential skills that enable the family physician to deal competently with problems in the domain of family medicine. The competent family physician has the potential to use all the skills for any problem, but competence is also characterized by adapting the choice of the skills used to the specific needs of the problem at hand. a) Patient-centred approach: This is a hallmark of family medicine and represents one of the most efficient and effective methods for dealing with problems. The details of the method are well established in the literature, and the evaluation objectives for this dimension of competence are derived directly from this information. b) Communication skills: Certain skills and behaviours facilitate communication, and good communication is essential for competence. Communication can be written or verbal, with patients or colleagues; it also involves listening and watching as much as or more than talking and showing. c) Clinical reasoning skills: This dimension focuses on the problem-solving skills used to deal with the “medical aspects” of a problem. Although obviously knowledge dependent, many of the difficulties in this dimension are related to poor process (the how and why). Assessment of these processes is more important than assessing the final results or answers. d) Selectivity: This dimension has not, to our knowledge, been described with respect to physician competence. It describes a set of skills cited as characterizing the competent family physician: such a physician does not do things in a routine fashion, but is selective in their approach, adapting it to the situation and patient. This physician sets priorities and focuses on the most important, knowing when to say something and when not to, gathering the most useful information without losing time on less contributory data, or doing something extra when it will be helpful. It is perhaps a subset of all the other dimensions, but it was used frequently enough to merit its own dimension. e) Professionalism: This dimension was the most frequently cited in the descriptions of competence. It includes all the responses that dealt with respect and responsibility to patients, to colleagues, to oneself, to the profession, and to society. It includes ethical issues, as well as lifelong learning and the maintenance of quality of care. It also includes attitudinal aspects such as caring and compassion. f) Procedure skills: In the initial survey, specific procedures themselves were not often cited as being characteristic of competence. It was recognized, however, that an individual about to enter independent practice should be able to competently perform certain procedures. A working group on procedure skills identified 65 core procedures; assessment of competence in 4 | P a g e 2010 Executive Summary this dimension will be based on these as well as general key features developed for procedure skills. 2. Phase of the Clinical Encounter: This component plays an essential role in directing assessment toward the cognitive processes most critical to the competent resolution of a specific problem or situation. It covers the steps or phases from the beginning to end of a clinical encounter. It includes the processes usually identified with a hypothetico-deductive model of clinical problem solving, and with clinical decision making. a) Hypothesis generation (preliminary differential diagnosis) b) History (gather the appropriate information) c) Physical examination (gather the appropriate information) d) Investigation (gather the appropriate information) e) Diagnosis, including problem identification (interpret information) f) Treatment (or management) g) Follow-up h) Referral 3. Priority Topics, Core Procedures, and Themes: These constitute a list of the problems or situations that the competent family physician should be able to deal with at the start of independent practice. This list sets out and limits the content of competence in family medicine for the purposes of certification. The limits permit all concerned to concentrate their efforts, and the scope reassures us that overall competence can be reasonably inferred if assessment has been based on an adequate sampling of this content. a) Priority topics: This list was generated from the responses in the original survey. It includes diagnoses, symptoms, presentations, and tasks; there are also roles (periodic health/screening), groups (immigrants, newborn, elderly), issues (lifestyle), situations (family issues, difficult patients), and even some topics (antibiotics). b) Core procedures: Competence in this dimension is not limited to the technical skills required for the 65 core procedures. Other aspects, such as indications and contraindications, deciding to do or not to do a procedure, and choosing among several possible approaches should also be assessed. With this in mind, a key feature analysis was undertaken to identify the critical aspects of competence applicable to all procedures. c) Themes: The dimensions of the patient-centred approach, professionalism, and communication skills were not sufficiently defined by the key feature analysis of the priority topics. An additional iterative, a focus-group approach, using information from a variety of sources as inspiration, was used to develop a series of themes to organize the description of competence for each of these three dimensions. 4. Key Features and Observable Behaviours: These are the operational evaluation objectives describing competence in relatively objective and observable terms. This component is most useful for the assessment of competence during daily clinical supervision. a) Key features: Each priority topic underwent analysis to generate the key features for the topic. Key features are the specific situations most determinant of competence within a topic and the critical processes involved in dealing competently with each situation. They are determined by a group of practicing peers using a reflective, iterative process. Each key feature identifies the skill dimensions and phases of the clinical encounter that are to be used in assessing the competence for the situation and task in question. b) Observable behaviours: For each of the themes identified for communication skills and professionalism, an iterative process analogous to the key feature analysis was used to identify behaviours indicative of competence, or lack of it, for each theme. While the key feature 2010 5 | P a g e