In   on September 20, 2019

Building a Competency Assessment

Key Takeaway

Connecting education and assessment to a publicly accessible competency document will build public confidence by clarifying expectations for policy makers, regulators, educators, and examinees.

Why do you need a competency-based assessment?


In our home province of Ontario The Regulated Health Professions Act, 1991 governs 29 regulated health professions' and 26 affiliated Colleges. Similar legislation exists in other jurisdictions to set out the rules under which professions can oversee their own members and regulate themselves. Health profession self-regulation is in the public interest and influenced by factors such as patient safety, policy changes, workforce demands, and training programs. Typically, the behavioural expectations of a profession are documented in  a blueprint or competencies document that can inform the curricula of professional training programs, and serve as a framework to assess individual competence. Recent trends in health professions education and assessment in Europe and North America have connected these competency documents directly to milestones that must be completed. In medicine for example, the Royal College of Physicians and Surgeons of Canada has worked with individual medical specialty programs and schools to develop a set of workplace tasks or Entrustable Professional Activities (EPA) that medical residents must attempt throughout their training. Medical residents must show some level of proficiency in each identified EPA prior to graduation towards independent practice. Similarly, the American Council for Graduate Medical Education (ACGME) has developed a list of common program requirements or milestones in addition to specialty specific competencies.


This trend is driven by a general call to have more explicit and transparent expectations for health care professionals in order to build public trust. By connecting education and assessment to a publicly accessible competency document, it is hoped that the public can build confidence in the process from curriculum planning, to training, assessment, supervised and finally independent practice. Additionally, linking assessment to a profession’s competency document can clarify expectations for policy makers, regulators, educators, and trainees.


It may seem a daunting task to begin the process of building a competency based exam. So we have outlined a series of steps below to outline this process. We have expertise at Touchstone Institute to help a profession with all steps, with test development generally starting at Step 5. Steps are not necessarily sequential, and are customized for the specific profession and the purpose of the assessment. Please contact us for information on how this model can be applied to meet your profession’s requirements.


  • Step 1 – Professional Competency Analysis
  • Step 2 – Refine Analysis into a professional competencies document
  • Step 3 – Identify the purpose of the competency assessment
  • Step 4 – Identify the key competencies to be assessed
  • Step 5 – Determine ideal components of the assessment
  • Step 6 - Create a blueprint for the assessment
  • Step 7 – Develop items according to blueprint
  • Step 8 – Review items, validation process
  • Step 9 – Define/revise qualitative standard of competence to be demonstrated
  • Step 10 – Define/revise scoring tools to align with standard
  • Step 11 - Consider cut-score


on March 12, 2024
Responding to a Changing Landscape.
on April 11, 2023
Have you ever wondered if the specific set of factors you are assigned to on OSCE exam day has an impact on your performance? As with any exam that is in high demand and will be attempted by many examinees, we want to accommodate as many examinees as possible and this is only possible by having multiple exam tracks, including rest stations in addition to the scored stations, and holding multiple exam times and days. The Research and Analysis department at Touchstone Institute wanted to answer this question to ensure examinees are offered a fair, valid and equivalent assessment. Watch the video to learn more about the investigation.
on November 30, 2022
Healthcare professionals are in short supply in Ontario and globally, a situation exacerbated by the pandemic. Ontario has one of the lowest ratios of doctors to population in Canada and continues facing shortages in health human resources due to retirements, fatigue, and dissatisfaction. Strategies to increase the supply of healthcare workers are vital to building a resilient system to cope with the health sector challenges that Ontario will face in the future. There has been a historical reliance on internationally educated health professionals (IEHPs) to fill the gap. They are here but are often underutilized or face lengthy pathways to acceptance and registration. More effective strategies to integrate them into the workforce are needed. Competency assessments are a proven method that is efficient and effective.
Scroll To Top