Runaway youth are 6-12 times more likely to become infected with HIV than youth would stay at home. Using a quasi-experimental design, the authors evaluated the effectiveness of an HIV prevention program during a 2 year study among 2 groups of runaways: (1) those at 2 shelters who received Street Smart, an intensive HIV intervention program, and (2) youth at 2 control shelters. Youth in the Street Smart program had access to health care and condoms and received a 10-session skill-focused prevention program based on social learning theory. Before analysis of the intervention's outcomes, propensity scores were used to identify comparable subgroups of youth in the intervention (n = 101) and control conditions (n = 86). Compared to females in the control condition, females in the Street Smart program significantly reduced their unprotected sexual acts at 2 years and alcohol use, marijuana use, and the number of drugs used during 12 months. Male adolescents in the Street Smart program showed significant reductions in marijuana use during 6 months compared to control youth. Adolescent HIV prevention programs must proactively identify mechanisms for maintaining behavior change for the long-term, and innovative research designs are needed to allow examination of agency-level interventions.