The current paper discusses an approach to measuring treatment integrity of Dialectical Behavioral Therapy (DBT) when implemented within two programs providing services to street-involved youth in the community. Measuring treatment integrity is a critical component of effective implementation of evidence-based interventions in clinical practice, since sound treatment integrity increases confidence in client outcomes and intervention replicability. Despite being an essential part of implementation science, few studies report on treatment integrity, with limited research addressing either measurement tools or maintenance of treatment integrity. To address the lack of available treatment integrity measures, researchers in the current study developed and piloted a treatment integrity measure which pertain to the individual and group components of DBT. A total of 20 recordings were assessed using the treatment integrity measures. Results indicate that the community agency staff (e.g. youth workers, social workers & nurses) implemented the intervention as intended; increasing confidence in the outcome variables, the staffs’ training and the replicability of the intervention. This article offers one approach to addressing treatment integrity when implementing evidence-based interventions, such as DBT in a community setting, and discusses the need for effective and feasible integrity measures that can be adopted in order to strengthen mental health practice in community settings.