Show simple item record

dc.creatorLandy, Justin F.
dc.creatorJia, Miaolei
dc.creatorDing, Isabel L.
dc.creatorViganola, Domenico
dc.creatorTierney, Warren
dc.creatorDreber, Anna
dc.creatorJohannesson, Magnus
dc.creatorPfeiffer, Thomas
dc.creatorEbersole, Charles R.
dc.creatorGronau, Quentin F.
dc.creatorLy, Alexander
dc.creatorvan den Bergh, Don
dc.creatorMarsman, Maarten
dc.creatorDerks, Koen
dc.creatorWagenmakers, Eric-Jan
dc.creatorProctor, Andrew
dc.creatorBartels, Daniel M.
dc.creatorBauman, Christopher W.
dc.creatorBrady, William J.
dc.creatorCheung, Felix
dc.creatorCimpian, Andrei
dc.creatorDohle, Simone
dc.creatorDonnellan, M. Brent
dc.creatorHahn, Adam
dc.creatorHall, Michael P.
dc.creatorJimenez-Leal, William
dc.creatorJohnson, David J.
dc.creatorLucas, Richard E.
dc.creatorMonin, Benoit
dc.creatorMontealegre, Andres
dc.creatorMullen, Elizabeth
dc.creatorPang, Jun
dc.creatorRay, Jennifer
dc.creatorReinero, Diego A.
dc.creatorReynolds, Jesse
dc.creatorSowden, Walter
dc.creatorStorage, Daniel
dc.creatorSu, Runkun
dc.creatorTworek, Christina M.
dc.creatorVan Bavel, Jay J.
dc.creatorWalco, Daniel
dc.creatorWills, Julian
dc.creatorXu, Xiaobing
dc.creatorYam, Kai Chi
dc.creatorYang, Xiaoyu
dc.creatorCunningham, William A.
dc.creatorSchweinsberg, Martin
dc.creatorUrwitz, Molly
dc.creatorUhlmann, Eric L.
dc.creatorThe Crowdsourcing Hypothesis Tests Collaboration
dc.creatorBaker, Bradley
dc.date.accessioned2021-11-09T15:48:48Z
dc.date.available2021-11-09T15:48:48Z
dc.date.issued2020-01-16
dc.identifier.citationCrowdsourcing Hypothesis Tests Collaboration, & Albers, C. (2020). Crowdsourcing hypothesis tests: Making transparent how design choices shape research results. Psychological Bulletin, 146(5), 451–479. https://doi.org/10.1037/bul0000220
dc.identifier.issn1939-1455
dc.identifier.doihttp://dx.doi.org/10.34944/dspace/7091
dc.identifier.urihttp://hdl.handle.net/20.500.12613/7111
dc.identifier.urihttp://dx.doi.org/10.34944/dspace/7091
dc.description.abstractTo what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = −0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.
dc.format.extent30 pages
dc.languageEnglish
dc.language.isoeng
dc.relation.ispartofFaculty/ Researcher Works
dc.relation.haspartPsychological Bulletin, Vol. 146, Iss. 5
dc.relation.isreferencedby© American Psychological Association, [2020]. This paper is not the copy of record and may not exactly replicate the authoritative document published in the APA journal. The final article is available, upon publication, at: https://doi.org/10.1037/bul0000220
dc.rightsAll Rights Reserved
dc.subjectConceptual replications
dc.subjectCrowdsourcing
dc.subjectForecasting
dc.subjectResearch robustness
dc.subjectScientific transparency
dc.titleCrowdsourcing hypothesis tests: Making transparent how design choices shape research results
dc.typeText
dc.type.genrePost-print
dc.description.departmentSport and Recreation Management
dc.relation.doihttps://doi.org/10.1037/bul0000220
dc.ada.noteFor Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu
dc.description.schoolcollegeTemple University. School of Sport, Tourism and Hospitality Management
dc.creator.orcidBaker|0000-0002-1697-4198
dc.temple.creatorBaker, Bradley J.
refterms.dateFOA2021-11-09T15:48:48Z


Files in this item

Thumbnail
Name:
Baker-PostPrint-2020-01.pdf
Size:
1.046Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record