What is Coordinated Assessment?
Communities across the globe are adopting coordinated assessment in an effort to more efficiently allocate scarce housing resources based on the support service needs of single adults, youth, and families experiencing homelessness. Through coordinated assessment, all people experiencing homelessness in a given continuum of care or local jurisdiction participate in a standardized assessment of their mental health, medical, and social vulnerabilities. Based on assessment scores, individuals are triaged to housing services offering varying levels of support services, and centralized waitlists for housing resources in the community are prioritized accordingly.
Why is Evidence-based Coordinated Assessment Important?
Coordinated assessment instruments are at the foundation of centralized housing waitlists. Without a strong foundation, the whole house may come crumbling down. Instruments that do not accurately assess an individual’s housing support service needs may unintentionally reduce a highly vulnerable individual’s opportunity for housing. Alternatively, inaccurate assessment may inappropriately prioritize a person with lower support service needs for costly housing interventions like permanent supportive housing. In some cases, a difference of one point on an assessment instrument could determine whether a person is prioritized for a particular type of housing service. Ineffective housing prioritization has major implications for the wellbeing of vulnerable people and for the systems serving them.
Due to the complexities associated with homelessness, what it means to be “vulnerable” or “self-sufficient” cannot be objectively measured. Therefore, we are left developing assessment tools composed of items that we believe—or research would suggest—measure these concepts. However, there are numerous considerations to take into account when it comes to developing effective measurement tools. For example, the types of questions, the response options, and the overall assessment approach can influence whether people respond accurately on an assessment.
Psychometric research is used in many fields that lack objective measurements of concepts of interest. In the field of clinical psychology, psychological assessment measures of depression, anxiety, and other mental health conditions undergo rigorous research and vetting before they are recommended for use in research and clinical application. Unfortunately, the same process did not occur before coordinated assessment unfolded, despite the high stakes involved. There is currently very limited evidence to support the psychometric properties of instruments available for coordinated assessment. In other words, we do not know whether we are building our houses on shaky foundations.
Overview of the Vulnerability Index-Service Prioritization Decision Assistance Tool
The Vulnerability Index-Service Prioritization Decision Assistance Tool (VI-SPDAT), now on its second version, has been widely adopted for coordinated assessment. The VI-SPDAT was designed for rapid, interview-style administration that can be applied with minimal training, making it a desirable choice for communities tasked with assessing a large homeless population. The instrument primarily relies on the self-report of those assessed; the original version also included four observer-rated items indicating a subjective evaluation of the extent of impairment related to daily living skills, physical health conditions, substance use, and mental health observed by the assessor. VI-SPDAT items are grouped under four subdomains: History of Housing and Homelessness, Risks, Socialization and Daily Functions, and Wellness. The VI-SPDAT triages people for three housing types, or housing assessments, based on their score: a) permanent supportive housing (i.e., permanent housing subsidies with housing support services) for those reporting the greatest range of vulnerability, b) rapid rehousing (short-term housing subsidies or other financial support and temporary support services) for those scoring in the moderate range, and c) mainstream affordable housing (i.e., individuals directed toward mainstream affordable housing options) for those scoring in the minimally vulnerable range.
In addition to its ease of use and other potential strengths, the VI-SPDAT has gained traction due to the developers’ assertions that it is “evidence-informed” and the strongest tool available based upon its evidence and testing. However, the evidence base for the VI-SPDAT versions 1 and 2 is largely unclear. Indeed, there has been little to no independent research conducted on the tool’s reliability and validity.
Our Research
We recently published a study in which we used Homeless Management Information System (HMIS) data from a continuum of care in a Midwest county in the United States to assess several types of reliability (i.e., the extent to which an assessment tool gets consistent results) and validity (i.e., the extent to which an assessment tool measures what it intends to measure) of the first version VI-SPDAT for single adults. Here is an overview of our main findings:
- The VI-SPDAT did not produce consistent results. In terms of reliability, we found individuals that were administered the VI-SPDAT twice did not produce consistent scores. In fact, 89% produced either higher or lower scores during their second administration. The observer-rated items were not reported consistently across administrations, suggesting the omission of these items in the VI-SPDAT version 2 likely improved the tool.
- The questions did not fully measure the concept of “vulnerability”. In terms of the validity, we found that several questions on the VI-SPDAT were not strongly related, or were related in an unexpected way (e.g., the presence of a health condition was associated with lower vulnerability), with the concept of vulnerability and/or with the VI-SPDAT subdomains. The Socialization and Daily Functions domain and health-related items on the Wellness domain demonstrated particularly poor validity.
- The type of housing support a person had was a better predictor of returning to homelessness than their VI-SPDAT score. It is reasonable to expect that individuals who are more vulnerable are at higher risk of housing instability or homelessness than those who are less vulnerable. However, among individuals in this study who were permanently housed after taking the VI-SPDAT, higher scores were only marginally associated with the likelihood they would re-enter homeless services (a proxy measure for housing stability). Rather, the type of housing a person obtained (i.e., rapid rehousing or unsubsidized housing) was a stronger predictor of increased risk of homeless service re-entry.
Main Takeaways
It is important to acknowledge that this study was conducted in the context of community implementation and not under standardized or experimental conditions. While the psychometric properties of the instrument in “ideal” circumstances remain unclear, our findings are likely more representative of the tool’s effectiveness when being used in practice. Findings suggest the VI-SPDAT has weaknesses in its reliability and validity. These weaknesses may result from problems with the tool itself, how it is used in real-world practice, or individuals’ tendencies to inaccurately disclose sensitive information.
Coordinated assessment and centralized housing prioritization procedures vary widely. Communities that have adopted the VI-SPDAT as the primary method for making housing recommendations should integrate other assessments that are more comprehensive in nature and that include other sources of information in addition to self-report. Additional research and development of the VI-SPDAT and other coordinated assessment instruments, such as the Vulnerability Assessment Tool, can help improve these tools and ensure people are prioritized appropriately and given the support services and housing they need.
You can read the full study here