INNOVATION

Assessment Design and Administration

Introduction

Since our incorporation in 2006 we have partnered with health professionals and their regulators to develop and administer high-stakes competency assessments applying trusted methods and innovative technology.

Key Links

See Our Assessments

Contact Us

To inquire about our assessment design

Send an inquiry

Assessment Design

Exams created by Touchstone Institute are constructed and validated by our in-house experts, who work in collaboration with practising health professionals and experts in healthcare from Ontario’s universities, professional colleges and regulatory bodies.

Our Process

Scoping and blueprinting workshops

Through focused workshops the subject matter experts along with examination development experts define the scope of the project and decided on the competency document to be used.

Next, a framework for the examination is created that we refer to as the blueprint.
In this process decisions are made on how to appropriately test the competencies that are needed to be demonstrated, and ultimately what the final exam format will look like.

Creating the exams

Subject matter experts are recruited to develop items that have been predefined by the blueprint. Orientation sessions familiarize the SMEs with the format of the exam, the competency documents, the exam blueprint, and are trained on how to successfully create exam content. All content created is peer reviewed by SMEs, lead examiners, simulated client trainers, and exam administrators before being submitted for item testing as potential exam content.

Assessment Technology

Touchstone Institute is a member of the Umbrella Consortium for Assessment Networks, which develops and provides support for leading-edge exam technology. UCAN is a non-profit umbrella organization for various assessment and education organizations that create and administer examinations. Their team of educational experts and software developers has more than 10 years of experience in developing solutions for educational exams. With more than 70 Partners from 8 countries, UCAN is a leader in providing leading-edge solutions for high stakes exams.

Item Management System

Touchstone Institute utilizes UCAN's Item Management System (IMS) in order to streamline content creation. It allows us to generate new exam forms using a highly sophisticated competency classification system. IMS also serves as a database to capture information on our items regarding authoring, reviewing, performance, and activity history to ensure that our exams generated match our blueprint specifications.

Tablet-Based Exam Applications

Touchstone Institute has implemented UCAN exam technology in several of our current
assessment programs, with new assessments being developed with the UCAN tools. Our assessment programs utilize over 100 tablets for our OSCE and MCQ examinations. Tablet-based examinations have streamlined our workflow, automated the process to reduce human error, and expedited reporting to stakeholders.

Some of the key features of tablet scoring include:

QR code registration allows quick enrollment of examinees into tablet scoring system
Examinations are hosted in a secure environment within Touchstone Institute
Tablets are portable allowing for administration at sites across Canada
Exams are created quickly and efficiently
Multifactor backup of exam scoring for increased security
Customizable exam formats
Intuitive user interface

tOSCE: Tablet- Based Objective Structured Clinical Exam (OSCE) Scoring Application

The tOSCE application has allowed us to eliminate the use of scantron test sheets in our OSCE exams. The application has been designed to mimic the format of a scantron sheet, so examiners are familiar with the layout. We have integrated station information into the tOSCE application to make station criteria and marking information accessible to examiners.

The application allows several optional features including predefined comments, keyboard access, video playback and audio recording of comments.

tEXAM: Tablet-Based Written Exam Application

Assessment Administration

Our Toronto Exam Centre

Touchstone Institute is the largest assessment centre of its kind in Canada. Exams administered at Touchstone Institute benefit from our comprehensive facilities, expert full-time staff, and experienced standardized clients. Many regulatory colleges entrust Touchstone Institute with the design and administration of their high-stakes examinations.

Our purpose based facility was designed for high-stakes exams and provides a secure and well controlled environment. Touchstone Institute routinely manages large groups of examinees, examiners, standardized clients and support staff on examination days. Our 52 exam rooms, and all common areas, are monitored and video recorded. Tablet technology is used for most assessments (both written and clinical), providing secure and complete data capture.

See Our Facilities

In Other Provinces:

Touchstone Institute also has the capacity, infrastructure and expertise to administer assessments across Canada through a network of vetted third-party examination sites.

Exterior of Touchstone Institute building

Validity and Reporting

Touchstone Institute is experienced in a range of validation strategies that yield information on the accuracy of testing procedures and the calculation of results. These studies also test exam logistics and include feedback from examinees, examiners, and other participants and observers. All this data is used to enhance the performance of the exam.

We work alongside regulators to deliver exam results through reports that meet their needs and requirements. Reports are generated by our psychometrics team, who ensure ongoing reliability and validity of all measurement tools. We also offer benchmarking, standard setting and competency assessment reviews to help regulators ensure fair and defensible processes for their applicants.