PUBLICATIONS & REPORTS
Key Takeaway
Connecting education and assessment to a publicly accessible competency document will build public confidence by clarifying expectations for policy makers, regulators, educators, and examinees.
Why do you need a competency-based assessment?
In our home province of Ontario The Regulated Health Professions Act, 1991 governs 29 regulated health professions' and 26 affiliated Colleges. Similar legislation exists in other jurisdictions to set out the rules under which professions can oversee their own members and regulate themselves. Health profession self-regulation is in the public interest and influenced by factors such as patient safety, policy changes, workforce demands, and training programs. Typically, the behavioural expectations of a profession are documented in a blueprint or competencies document that can inform the curricula of professional training programs, and serve as a framework to assess individual competence. Recent trends in health professions education and assessment in Europe and North America have connected these competency documents directly to milestones that must be completed. In medicine for example, the Royal College of Physicians and Surgeons of Canada has worked with individual medical specialty programs and schools to develop a set of workplace tasks or Entrustable Professional Activities (EPA) that medical residents must attempt throughout their training. Medical residents must show some level of proficiency in each identified EPA prior to graduation towards independent practice. Similarly, the American Council for Graduate Medical Education (ACGME) has developed a list of common program requirements or milestones in addition to specialty specific competencies.
This trend is driven by a general call to have more explicit and transparent expectations for health care professionals in order to build public trust. By connecting education and assessment to a publicly accessible competency document, it is hoped that the public can build confidence in the process from curriculum planning, to training, assessment, supervised and finally independent practice. Additionally, linking assessment to a profession’s competency document can clarify expectations for policy makers, regulators, educators, and trainees.
It may seem a daunting task to begin the process of building a competency based exam. So we have outlined a series of steps below to outline this process. We have expertise at Touchstone Institute to help a profession with all steps, with test development generally starting at Step 5. Steps are not necessarily sequential, and are customized for the specific profession and the purpose of the assessment. Please contact us for information on how this model can be applied to meet your profession’s requirements.
- Step 1 – Professional Competency Analysis
- Step 2 – Refine Analysis into a professional competencies document
- Step 3 – Identify the purpose of the competency assessment
- Step 4 – Identify the key competencies to be assessed
- Step 5 – Determine ideal components of the assessment
- Step 6 - Create a blueprint for the assessment
- Step 7 – Develop items according to blueprint
- Step 8 – Review items, validation process
- Step 9 – Define/revise qualitative standard of competence to be demonstrated
- Step 10 – Define/revise scoring tools to align with standard
- Step 11 - Consider cut-score
