A comparison of language sample elicitation methods for dual language learners
Genre
Thesis/DissertationDate
2017Author
Toscano, JacquelineAdvisor
Reilly, JamieCommittee member
Reich, JodiGarcía, Felicidad M.
Department
Communication SciencesSubject
Speech TherapyCulturally and Linguistically Diverse
Dual Language Learners
Language Assessment
Language Sample Analysis
Permanent link to this record
http://hdl.handle.net/20.500.12613/4139Metadata
Show full item recordDOI
http://dx.doi.org/10.34944/dspace/4121Abstract
Language sample analysis has come to be considered the “gold standard” approach for cross-cultural language assessment. Speech-language pathologists assessing individuals of multicultural or multilinguistic backgrounds have been recommended to utilize this approach in these evaluations (e.g., Pearson, Jackson, & Wu, 2014; Heilmann & Westerveld, 2013). Language samples can be elicited with a variety of different tasks, and selection of a specific method by SLPs is often a major part of the assessment process. The present study aims to facilitate the selection of sample elicitation methods by identifying the method that elicits a maximal performance of language abilities and variation in children’s oral language samples. Analyses were performed on Play, Tell, and Retell methods across 178 total samples and it was found that Retell elicited higher measures of syntactic complexity (i.e., TTR, SI, MLUw) than Play as well as a higher TTR (i.e., lexical diversity) and SI (i.e., clausal density) than Tell; however, no difference was found between Tell and Retell for MLUw (i.e., syntactic complexity/productivity), nor was there a difference found between Tell and Play for TTR. Additionally, it was found that the two narrative methods elicited higher DDM (i.e., frequency of dialectal variation) than the Play method. No significant difference was found between Tell and Retell for DDM. Implications for the continued use of language sample for assessment of speech and language are discussed.ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.eduCollections
Related items
Showing items related by title, author, creator and subject.
-
Diagnosing L2 English Learners’ Listening comprehension abilities with Scripted and Unscripted Listening TextsL2 listening research has moved toward a focus on understanding the process of listening. However, there are still few detailed studies of L2 listening that reveal learners’ comprehension processes when listening to scripted and unscripted listening texts. Studies in which such processing has been discussed have lacked detailed diagnoses of how bottom-up and top-down processing interactively affect listeners’ comprehension. This study was designed to show how listeners’ process and comprehend texts, with a focus on how their bottom-up and top-down processing either assist or impede their comprehension. In this study, a group of 30 L1 Japanese university English language learners’ listening abilities were diagnosed. The 30 participants were at three listening proficiency levels—high, mid, and low—based on TOEIC listening proficiency scores. The diagnostic procedure involved participants listening to two scripted and two unscripted listening texts and then reporting what they comprehended through three tasks—L1 oral recalls, L2 repetitions, and verbal reports. Other data was also collected in the study to relate the comprehension of listening texts to other important listening-related variables including listening proficiency, lexical knowledge, listening anxiety, study abroad experience, short-term phonological memory, and working memory. The main finding of the study was that miscomprehension of listening texts was invariably multi-causal, with a combination of both bottom-up and top-down factors leading to comprehension difficulty. Although not a new finding, the study offered more detail than current research about how bottom-up and top-down processing occur interactively. Regarding the overall difficulty of the listening texts, unscripted texts were more difficult to comprehend than scripted texts, and high-proficiency participants had fewer listening difficulties overall than mid- and low-proficiency participants. Quantitative and qualitative results revealed common processing difficulties among all participants due to L1-related phonological decoding issues (e.g., /l/ vs. /r/), connected speech, unknown lexis, and a lack of familiarity with unscripted speech hesitation phenomena (e.g., um, like). Qualitative transcript examples showed how top-down knowledge influenced misinterpretations of words and phrases interactively with bottom-up information, making inaccurate understandings of listening difficult to overcome. In addition to revealing participants’ difficulties and the severity of their comprehension difficulties, the diagnostic procedure showed common strengths—key words and phrases understood well by participants. High-frequency vocabulary and shorter utterances were both shown to be comprehended well. Finally, quantitative results in the study revealed relationships of participants’ listening comprehension with other important listening related variables. Listening proficiency and listening anxiety had strong relationships with listening comprehension of the listening texts. Working memory and short-term phonological memory had no relationship with listening text comprehension. Finally, study abroad experience showed a relationship with comprehension, but with many caveats, and listening vocabulary knowledge was not related with comprehension, but again, with numerous caveats to consider. Based on the results, theoretical and pedagogical implications were posed. Theoretical implications from the study relate to the understanding of four concerns in L2 listening research. Mainly, data in the study will aid researchers’ understanding of how L2 English listeners process speech interactively (i.e., with bottom-up and top-down information) for comprehension, how L2 English listeners experience connected speech, how L2 listeners deal with unknown lexis, and how L2 listeners experience difficulties with features of unscripted speech. Pedagogical implications of the study include the need for increased teacher and learner awareness of the complexity of L2 listening, the need to have learners to track their own listening development, and the need for teachers to expose learners to unscripted listening texts and make them familiar with features of unscripted speech. Finally, suggestions for further research are posed, including conducting diagnostics assessments of L2 listening with listeners of different L1s and with more varied proficiency levels, using different diagnostic procedures to examine L2 listening comprehension, and using more instruments to understand listening-related variables’ relationships with L2 listening comprehension.
-
DECLINE OF A HERITAGE LANGUAGE, PALAUAN: THE INTERPLAY OF LANGUAGE POLICIES, PLANNING, PRACTICES AND OPINIONS IN PALAUThis case study investigates the language policies and planning (LPP) implemented in Palau since the occupation by Japan before and during World War II, and by the United States of America under the United Nation's Trusteeship after the war. Palau is an island country in the Pacific with a population of 17,500, including 4,600 foreign-born citizens. The society is multilingual as a result of a 150-year occupation by other countries, including Japan and the United States, before its independence in 1994. In this study I also explore the effects of LPP during that time, including the policy regarding a standard writing system, practices at pedagogical institutions, and Palauans' opinions about languages, especially the two official languages, Palauan and English. Data were gathered through interviews, historical document study, observations of classes, and a questionnaire administered in Palau, by visiting the country more than 20 times, for one- to two-week stays beginning in 2001. Hornberger (2006) stated that the terms language policy and language planning have been used interchangeably or as a single concept in many previous studies. Her suggestion was to use the two terms as a set, as the relationship between them has been ambiguous in the past (p. 25). I agree with Hornberger that the two terms fundamentally form a single concept, and therefore, they are used as a set in this study. The theoretical framework proposed by Taylor (2002) is used to analyze the current LPP in Palau: that is, (1) language planning composed of (1.1) status planning, (1.2) corpus planning, and (1.3) acquisition planning; (2) language-in-education policy; and (3) aspects of language-in-education implementation program that consist of (3.1) curriculum policies, (3.2) personnel policies, (3.3) material policies (methods, content), (3.4) community policies, and (3.5) evaluation policies (p. 318). He stated, "[t]he process of devising a new national language policy" affects "language-in-education implementation programs" (p. 318). Major LPP studies were reviewed chronologically based on three phases suggested by Ricento (2000, pp. 10-22). It was helpful to consider the history of LPP "as a dynamic interplay between academic concerns... and political/bureaucratic interests" (Wee, 2011, p. 11). Also, some previous researchers have noted that localized studies of language goals, language use, and language change are needed. According to Kaplan and Baldauf (2003), who studied languages and language-in-education planning in the Pacific Basin, it is rare for Pacific Basin countries to have a language policy: "... [L]anguage planning is frequently undertaken by the education sector in the absence of any such higher-level policy or in the light of such a policy so vaguely articulated as to be quite incapable of implementation" (p. 6). Although their study provided a great deal of valuable information, they did not investigate the language policies of Palau. In this study I describe the government's policies, and real life situation of the policies. To describe the real life situation of the policies, interviews, and a questionnaire survey were used. I interviewed Palauans, such as those who had experienced the occupation(s) and postwar period to better understand the historical background of the current LPP. I also interviewed incumbent teachers after observing their classes. Most of them described various problems in teaching the compulsory Palauan Studies Course, on Palauan language, history, tradition, and culture. I also interviewed officials of the Ministry of Education, who provided a great deal of information about the educational system in Palau and the curriculum of the Palauan Studies Course. The 62-item questionnaire provided data concerning people's language use in various social contexts, as well as the effects of language policies and planning on people's opinions about languages. The 137 respondents were divided into five groups according to their year of birth, considering the years when important transitions had occurred in the LPP. Their responses were compared, and some of the respondents were interviewed to illuminate the questionnaire results. I interviewed eight Palauans in March and September 2012 and asked why they had selected certain responses to the questionnaire items. The questionnaire results indicated that there is a tendency for the younger generation to use English more than the older generations in various contexts, and that the efforts Palauans have made, such as making the new writing system a compulsory part of the school curriculum, have yielded positive effects on the opinions of the younger generation, who learned the Palauan writing system at school. Overall, the results showed that Palauan is not in danger of extinction at present, but it might lose its status as the primary language in the future. I suggest strategies for preserving Palauan as the primary language.
-
Coherence in Quantitative Longitudinal Language Program EvaluationIn recent years, foreign language program evaluation has gained greater attention among language educators, program administrators, and evaluators. Increased demands for demonstrated program performance, often motivated by external forces, such as accreditation pressures and decisions regarding the allocation of funding, have led to heightened focus on foreign language program evaluation practices, methodologies, and results. Despite this increased attention, there are few published evaluation studies within the field of foreign language learning that have examined foreign language program effectiveness over time. This longitudinal study was designed to quantitatively investigate the performance of one Japanese university English for Academic Purposes (EAP) program over the 20-year span of the program’s existence. Quantitative evaluation methodologies and advanced statistical procedures were utilized to examine changes in student English proficiency, as measured by the Institutional Test of English as a Foreign Language (TOEFL ITP) and English achievement, as measured by four semesters of EAP course grades, as students progressed through the two-year program. Twenty cohorts of students (cohort n-sizes ranging from approximately 250 to 550 students) were included in this study. The comprehensive data set included three repeated-measures of the TOEFL ITP and four English achievement grade point averages (GPAs) for each of the 20 cohorts. The research questions for this expansive longitudinal study addressed two levels of inquiry. First, at the program-global level, this study sought to investigate patterns of English proficiency change within and between cohorts across the life of the program, and the extent that programmatic events and external influences might have impacted those patterns. For this investigation, TOEFL ITP results for three proficiency domains—listening, grammar, and reading—were chronologically charted for the 20 cohorts and time-series analyses were conducted. The results indicated that all cohorts demonstrated significant gains in the three proficiency domains by the end of the two-year program. However, the overall trends across the program’s 20-year history revealed gradual negative trajectories for grammar and reading proficiency. Events that were hypothesized to have influenced proficiency patterns were tested, including (a) the addition of a new department specialization, (b) changes to department admissions, (c) the entrance of students who experienced new national reforms at the secondary education level, and (d) department expansion. While listening proficiency patterns were unaffected, grammar and reading proficiency trends were negatively impacted by the start of the new specialization and changes to admissions procedures. The entrance of students who had experienced secondary educational changes had an initial negative impact on the grammar trend, but positive grammar and reading proficiency trends emerged from that point onward. It was speculated that these events, as well as larger population trends impacting Japanese universities, led to gradual shifts in program student demographics, which contributed to the observed changes in proficiency patterns. Also of interest was an examination of the concept of English achievement coherence—or the extent that student English achievement, as measured by English course grade point averages (GPAs)—can be used to assess course interrelatedness. English course GPA data was used to statistically derive three rival achievement coherence metrics. These metrics were then tested separately, using hierarchical linear modeling techniques, to examine the extent that achievement coherence might serve to mediate any proficiency variation observed across the 20 cohorts. There were no significant findings for two of the metrics tested, while the third metric was found to have a significant negative effect for reading proficiency. This finding directly contradicted the hypothesized outcome that a greater amount of coherence would serve to facilitate proficiency development. Given the significant negative reading trend that emerged across the life of the program, this result might suggest that larger influences affecting student demographic changes could outweigh any potential facilitative effects of coherence on proficiency outcomes. Following the program-global analyses, the second level of inquiry was at the cohort-specific level. Individual cohorts that had demonstrated comparatively high and low listening and reading proficiency gains were selected for follow-up analyses. The aim was to examine if differences in coherence at the cohort level might account for the contrastive proficiency gains attained. For each target cohort, a recursive path model, including the program’s 16 English courses and final proficiency outcome, was tested to examine English achievement interrelatedness and contributions to the final proficiency outcome. A greater number of significant paths and larger final model R2 coefficient would suggest more coherence. Additionally, for each target cohort, grade residuals analyses using linear regression methods were conducted to investigate grading consistency at the course level. A greater number of outlying grade cases could indicate that the course assessment schemes were not followed, which would suggest less cohort coherence. The results of these analyses for the pairs of contrastive listening gain and reading gain cohorts were compared, but no significant differences were found. While these analytical methods were determined to be useful for ongoing formative evaluation processes, the resulting measures were likely too broad to capture any meaningful differences in coherence between cohorts at the program-global level.
