Can a test using virtual reality effectively supplement or replace a performance test?
Hands-on performance testing often ties up equipment and personnel. This costs money and may also limit where and how often the test can be delivered. If VR testing could reliably predict the outcome of the performance test, it could be used as a funnel: candidates who are found unlikely to pass the performance test could train and practice more before taking the performance test.
In certain circumstances, a VR test could even substitute for a performance test.
The National Commission for the Certification of Crane Operators (NCCCO) certifies crane operators using a performance test (supplemented by selected response). The NCCCO commissioned a study using ITI VR Crane Simulators developed in partnership by Industrial Training International (ITI) and Serious Labs.
The NCCCO Foundation has released a report prepared by Wallace Judd, Ph.D., president of Authentic Testing Corp. on that study. The study sought to determine the extent to which a virtual-reality crane operator test could predict the results of a test conducted on an actual crane.
The results were encouraging.
“The likelihood of a candidate passing the VR test but failing the subsequent parallel test on an actual crane was 5.5%,” said Dr. Judd. “And the correlation of the score on one Actual Crane test with another actual crane test score taken by the same candidate was 0.88.”
Dr. Judd cautioned, however, that further studies would likely be needed.
“This study has significant implications for the way virtual reality is viewed in the professional assessment community,” Dr. Judd added, “and in that respect it is truly ground-breaking. However, additional studies will be required to determine conclusively whether it would be appropriate to certify a candidate VR technology as a replacement for an actual crane.”