Show simple item record

dc.contributor.advisorDuCette, Joseph P.
dc.creatorMargolis, Melissa J.
dc.date.accessioned2020-10-27T15:14:24Z
dc.date.available2020-10-27T15:14:24Z
dc.date.issued2011
dc.identifier.other864885797
dc.identifier.urihttp://hdl.handle.net/20.500.12613/1832
dc.description.abstractBackground: Providing examinee performance data is a common procedural modification in modified-Angoff standard setting exercises. Results of research examining the impact of the practice are mixed, and despite the frequency with which this procedural modification is done, the overall impact on the outcomes is not well understood. This research presents a large-scale evaluation of the impact of providing performance data in the context of operational standard setting exercises for a medical licensing examination program. Methods: Data from a total of 18 independent standard setting panels across three different examinations were analyzed to evaluate whether and how the provision of performance information impacted the resulting cut scores. Results: Significant cut-score changes were found between initial and final judgments for all panels. A decrease in standard deviation between initial and final judgments for all panels was indicative of the convergence of judgments following the provision of data, and a lack of systematic changes in final cut scores suggested that judges were not simply changing their content-based judgments to align with the normative data. Both findings provide important procedural validity evidence for the standard setting process. Implications: The current data set (replications of the procedure within a single cycle, across cycles for an individual examination, and across multiple examinations) allowed for the most comprehensive evaluation of this topic that has been described to date and provides valuable insight into an issue that is relevant to many testing contexts.
dc.format.extent124 pages
dc.language.isoeng
dc.publisherTemple University. Libraries
dc.relation.ispartofTheses and Dissertations
dc.rightsIN COPYRIGHT- This Rights Statement can be used for an Item that is in copyright. Using this statement implies that the organization making this Item available has determined that the Item is in copyright and either is the rights-holder, has obtained permission from the rights-holder(s) to make their Work(s) available, or makes the Item available under an exception or limitation to copyright (including Fair Use) that entitles it to make the Item available.
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/
dc.subjectEducational Tests & Measurements
dc.subjectLicensing
dc.subjectStandard Setting
dc.titleThe Impact of Examinee Performance Information on Judges' Cut Scores in Modified-Angoff Standard Setting Exercises
dc.typeText
dc.type.genreThesis/Dissertation
dc.contributor.committeememberFarley, Frank
dc.contributor.committeememberFullard, William
dc.contributor.committeememberThurman, S. Kenneth
dc.contributor.committeememberClauser, Brian E.
dc.description.departmentEducational Psychology
dc.relation.doihttp://dx.doi.org/10.34944/dspace/1814
dc.ada.noteFor Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu
dc.description.degreePh.D.
refterms.dateFOA2020-10-27T15:14:24Z


Files in this item

Thumbnail
Name:
Margolis_temple_0225E_10914.pdf
Size:
1.163Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record