INNOVATION
Assessment Design and Administration
Assessment Design
Exams created by Touchstone Institute are constructed and validated by our in-house experts, who work in collaboration with practising health professionals and experts in healthcare from Ontario’s universities, professional colleges and regulatory bodies.
From development to administration, Touchstone Institute's roster of full time staff has experience with high stakes registration exams, certification exams, and assessment programs for individuals with an international education. Our dedicated staff recruit subject matter experts (SME’s) to work with us to facilitate exam creation.
Through focused workshops the subject matter experts along with examination development experts define the scope of the project and decided on the competency document to be used.
Next, a framework for the examination is created that we refer to as the blueprint.
In this process decisions are made on how to appropriately test the competencies that are needed to be demonstrated, and ultimately what the final exam format will look like.
Subject matter experts are recruited to develop items that have been predefined by the blueprint. Orientation sessions familiarize the SMEs with the format of the exam, the competency documents, the exam blueprint, and are trained on how to successfully create exam content. All content created is peer reviewed by SMEs, lead examiners, simulated client trainers, and exam administrators before being submitted for item testing as potential exam content.
Assessment Technology
Touchstone Institute is a member of the Umbrella Consortium for Assessment Networks, which develops and provides support for leading-edge exam technology. UCAN is a non-profit umbrella organization for various assessment and education organizations that create and administer examinations. Their team of educational experts and software developers has more than 10 years of experience in developing solutions for educational exams. With more than 70 Partners from 8 countries, UCAN is a leader in providing leading-edge solutions for high stakes exams.
Touchstone Institute utilizes UCAN's Item Management System (IMS) in order to streamline content creation. It allows us to generate new exam forms using a highly sophisticated competency classification system. IMS also serves as a database to capture information on our items regarding authoring, reviewing, performance, and activity history to ensure that our exams generated match our blueprint specifications.
Touchstone Institute has implemented UCAN exam technology in several of our current
assessment programs, with new assessments being developed with the UCAN tools. Our assessment programs utilize over 100 tablets for our OSCE and MCQ examinations. Tablet-based examinations have streamlined our workflow, automated the process to reduce human error, and expedited reporting to stakeholders.
Some of the key features of tablet scoring include:
- QR code registration allows quick enrollment of examinees into tablet scoring system
- Examinations are hosted in a secure environment within Touchstone Institute
- Tablets are portable allowing for administration at sites across Canada
- Exams are created quickly and efficiently
- Multifactor backup of exam scoring for increased security
- Customizable exam formats
- Intuitive user interface
The tOSCE application has allowed us to eliminate the use of scantron test sheets in our OSCE exams. The application has been designed to mimic the format of a scantron sheet, so examiners are familiar with the layout. We have integrated station information into the tOSCE application to make station criteria and marking information accessible to examiners.
The application allows several optional features including predefined comments, keyboard access, video playback and audio recording of comments.
tEXAM is UCAN's application for delivering written examinations, including multiple-choice question exams. tEXAM utilizes QR code to quickly enroll examinees into an exam. The application allows examinees to easily bookmark and tag questions for referencing. The user friendly interface has been well received by examinees.
Assessment Administration
Our Toronto Exam Centre
Touchstone Institute is the largest assessment centre of its kind in Canada. Exams administered at Touchstone Institute benefit from our comprehensive facilities, expert full-time staff, and experienced standardized clients. Many regulatory colleges entrust Touchstone Institute with the design and administration of their high-stakes examinations.
Our purpose based facility was designed for high-stakes exams and provides a secure and well controlled environment. Touchstone Institute routinely manages large groups of examinees, examiners, standardized clients and support staff on examination days. Our 52 exam rooms, and all common areas, are monitored and video recorded. Tablet technology is used for most assessments (both written and clinical), providing secure and complete data capture.
In Other Provinces:
Touchstone Institute also has the capacity, infrastructure and expertise to administer assessments across Canada through a network of vetted third-party examination sites.
Validity and Reporting
Touchstone Institute is experienced in a range of validation strategies that yield information on the accuracy of testing procedures and the calculation of results. These studies also test exam logistics and include feedback from examinees, examiners, and other participants and observers. All this data is used to enhance the performance of the exam.
We work alongside regulators to deliver exam results through reports that meet their needs and requirements. Reports are generated by our psychometrics team, who ensure ongoing reliability and validity of all measurement tools. We also offer benchmarking, standard setting and competency assessment reviews to help regulators ensure fair and defensible processes for their applicants.
