NEWS
The Canadian English Language Benchmark Assessment for Nurses (CELBAN) is a made-in-Canada exam that reflects nursing practice in Canada. It assesses language proficiency within authentic healthcare scenarios, and is delivered in two parts: the CELBAN Speaking Test and the CELBAN Computer Based Test, consisting of the Listening, Reading, and Writing Tests.
The Centre for Canadian Language Benchmarks (CCLB) and Touchstone Institute have committed to a partnership that strengthens support for CELBAN, Canada’s internationally respected, nursing-specific language exam.
Earlier this year, Touchstone Institute administered its first virtual exams. The digitization of the CELBAN exam aimed to ensure accessibility of the exam to current and future Canadians pursuing a career in nursing.
Touchstone Institute has successfully administered exams to 800 examinees since the relaunch earlier this year. Evaluation results indicate that IENs accept the new exam format and psychometric analysis has confirmed reliability and validity. This new format is also accepted as proof of language proficiency by nursing regulators across Canada.
New Delivery Method
CELBAN is now a virtual, computer-based exam. CELBAN exam delivery has changed from a paper-based exam offered at 8 licensed CELBAN Test Sites across Canada, to a virtual exam that can be completed from home, or at one of 18 approved exam centres across Canada.
Although the delivery model is now virtual, CELBAN’s construct and format remain unchanged. The Listening, Reading, and Writing Tests (previously known as “The Group Test”) are now delivered through Prometric as a remotely proctored exam or in-person at a Prometric test centre. The Listening, Reading, and Writing Tests comprise multiple choice items for the comprehension of nursing-related video interactions and readings, and two samples of written communication.
The interactivity is the most valuable feature of the Speaking Test as it is an authentic representation of nurses’ engagement with clients and colleagues. The Speaking Test was previously delivered as an interview between an examinee and two examiners who took turns with questions, role-playing and rating. The current model remains a live interaction between an examinee and an examiner through a video conferencing system.
Validity Considerations
Validity refers to the inferences made based on the data collected through an assessment process. Validity exists along a continuum and requires continuous attention. Test users – those who receive the test results and apply them to make decisions – must be confident that the assessment can help them make good decisions and that the decisions made will be defensible.
In planning the CELBAN transition to online delivery, the team referenced an extensive existing research literature on computer-based and online language testing based on work from the last 20 years (Chapelle & Voss, 2016).
Sources of validity evidence include:
- Construct validity: Construct validity means the test measures the skills/abilities that should be measured. In digitizing CELBAN we ensured the blueprint was adhered to and the appropriate language skill areas are represented and being assessed with the same rigor as the paper-based exam.
- Content Validity: Content validity means the test measures appropriate content. In digitizing CELBAN we worked to match the specification of the CELBAN test blueprint and retain the communication tasks identified by nursing subject matter experts and language researchers as the target language functions.
- Reliability: Reliability refers to dependable and consistent results. Our examiner calibration and ongoing psychometric analyses tracks the exam. The exam is consistent internally, and ensures that the results are consistently accurate.
| Construct Validity | Interview model and rating rubrics remain the same and are aligned with CELBAN Test Specifications. | Listening, reading and writing tasks are aligned with CELBAN Test Specifications. Writing task and rating rubrics remain the same and are aligned with CELBAN Test Specifications. |
| Reliability of Test Results | Comparison of live and video-mediated ratings. Examiner training and calibration. Double rating process. | Psychometric analysis of Listening and Reading Test results. Examiner training and calibration as well as double rating process implemented for Writing Test. |
| Standardization | Pilot Testing Process of virtual interview delivery. Video recordings reviewed by peers and quality assurance specialist. | Pilot testing of the Prometric Platform. Raw score conversion algorithms validated, applied and psychometrically reviewed on a continuous basis. Computer-based Writing Test format is standardized. |
| Benchmarks Alignment | Double rating to ensure consistent benchmark assignments. Speaking examiner calibrations. | Double rating of writing test to ensure consistent benchmark assignments. Writing rater calibrations. Alignment of results to CLB Subject Matter Expert item calibration (Reading and Listening). |
| Test Form Equating | Monitor results psychometrically across forms. | Develop parallel test forms. Monitor results psychometrically across forms. Results equating across test forms (paper vs online; various online forms to each other). |
CELBAN Examinee Testimonials
"Online was better because I did not need to worry <about> traveling to the venue, hence I did not need to wait at the waiting area for my speaking test which I find it more nervous."
"I was more comfortable with doing it online compared to having it in person."
"Overall, I had a comfortable experience during the exam."
"At first, I had encountered a couple of technical issues but the technical support representatives were so helpful and I was able to proceed with my test as scheduled."
"Before the exam i was really concerned and confused about the proproctor, but everything worked really well."
