Exploring the Relationship Between Scoring Accuracy and Predictive Validity in Risk Assessment Using the Service Planning Instrument (SPIn)

Public Deposited
Resource Type
  • Accurate scoring is an essential component of a risk assessment's reliability, which in turn contributes to its predictive validity. This study tested a new method for evaluating the accuracy of completed risk assessments, described as intra-rater consistency. Pairs of cross-consistency items in the Service Planning Instrument were used to identify scoring errors. Participants were 31,460 adults (20.5% female; 20.7% Indigenous) on community supervision in Alberta. The overall base rate of error was low; Indigenous participants had higher rates of error than non-Indigenous participants. In the full sample assessments with errors had slightly higher predictive accuracy than assessments without errors (error AUC = .70, no error AUC = .68). Similar results were found among non-Indigenous sub-groups, however, there were no significant differences in predictive accuracy based on errors among Indigenous participants. Sex- and race-based differences in predictive accuracy were observed in assessments with errors, but not in assessments without errors.

Thesis Degree Level
Thesis Degree Name
Thesis Degree Discipline
Rights Notes
  • Copyright © 2022 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, research, scholarship, and teaching. Theses may only be shared by linking to Carleton University Institutional Repository and no part may be used without proper attribution to the author. No part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
Date Created
  • 2022


In Collection: