Leveraging Confirmatory Program Evaluation to Statistically Assess the Effectiveness of the Upward Bound Math and Science Program at Temple University, 2008–2021

Abstract

The Upward Bound Math and Science (UBMS) program at Temple University (TU) seeks to guide first-generation, low-income high school students from Philadelphia in their quest to obtain postsecondary STEM degrees that lead to related careers. This study aims to evaluate the efficacy of the program by analyzing its students’ performance in light of its own goals and also in view of those of its sister program, The Upward Bound (UB) program. Primarily, this study uses Confirmatory Program Evaluation to ascertain which programmatic elements predict outcomes associated with indicators of STEM interest. Program leaders in TU’s College of Education and Human Development provided data for the study. Data came from 2015–2021 annual performance reports, containing information on 374 former UBMS student participants as well as 483 former UB students. Findings indicate that UBMS students pursued postsecondary enrollment at a significantly higher rate than UB students. Results also suggest that UBMS students’ grade level upon program entry was a significant direct negative predictor for how long they were affiliated with the program. In addition, students’ grade level upon program entry was a direct positive predictor for whether they obtained a rigorous course of study. Community service involvement was a significant negative predictor for whether students completed advanced math courses. None of the predictors yielded a significant effect in either of the two outcomes by way of a third, mediating variable. Implications of the study suggest that alliances between principal investigators, program administrators, and statisticians—ultimately through mixed-methods approaches—may offer valuable insight regarding the evaluation of UBMS and programs like it.