PROGRAM EVALUATION OF GRADUATE EDUCATION PROGRAMS IN AN AMERICAN UNIVERSITY IN JAPAN

Abstract

Program evaluation in the fields of second language acquisition (SLA) and teaching English to speakers of other languages (TESOL) has a history dating from the 1960's. The focus of previous program evaluations has been on language achievement at the end of the program of study (Lynch, 1996). However, to improve or maintain program quality, teacher education of future foreign language instructors is essential; thus, improvements in the quality and/or control of teacher education programs is also important. The primary purpose of this study is to propose and test a multi-faceted approach to program evaluation that originates from the administrative side of two graduate programs in Applied Linguistics. This marks the first time in the field that this approach has been implemented. First, time series enrollment models were examined to investigate the overall stability of the two graduate education programs for non-traditional students from 1993 to 2010 (for the Master's programs) and 1995 to 2010 (for the doctoral programs). Second, logistic regression models were examined to investigate the characteristics of the students who did and did not graduate from the programs. Third, event history Cox regression models were examined to investigate the amount of time spent by the graduate students to complete the degree using potential demographic and enrollment pattern factors as predictors. Finally, a dynamic model was formulated and tested to simulate the program's potential future enrollments based in part on the results of the prior model analyses and publically available data. The results indicated the potential for developing a relatively strong time series enrollment prediction model for near future enrollments using the data available in the administrative database. However, the student success/failure models and event history program study time prediction models were relatively weak and the results indicated the difficulty of predicting whether students will successfully complete the graduate programs using data typically collected from the students when they enter the program. The simulation results showed a potential decline in enrollments over the next decade mainly due to the decline of the 18-year-old student population in Japan, the decreasing size of the foreign language teacher population, the long, gradual recession in Japan, and the revision of university faculty employment status by the Japanese Ministry of Education.