Thumbnail Image
Item

D44. Artificial Intelligence In Plastic Surgery Writing: Can We Even Tell The Difference?

Habarth-Morales, Theodore E.
; 
Ewing, Jane
; 
Lemdani, Mehdi S.
; 
Rios-Diaz, Arturo J.
; 
Davis, Harrison D.
; 
Broach, Robyn B.
; 
Fosnot, Joshua A.
Citations
Altmetric:
Genre
Journal article
Date
2024-05-01
Advisor
Committee member
Group
Department
Subject
Permanent link to this record
https://scholarshare.temple.edu/handle/20.500.12613/11632
Collections
Faculty/Researcher Works
Show all metadata
Files
Thumbnail Image
Davis-JournalArticle-2024-05-01.pdf
Adobe PDF, 54.16 KB
Research Projects
Organizational Units
Journal Issue
DOI
https://doi.org/10.1097/01.gox.0001018668.18607.af
Abstract
PURPOSE: Artificial intelligence (AI) increasingly aids research writing. This study quantifies plastic surgeons’, trainees’, and laypeople’s ability to differentiate between AI and human-written research abstracts and evaluate the quality of writing comparatively METHODS: Ten abstracts were created using study outlines of previously accepted plastic surgery studies: 5 were generated using Pre-trained Transformer 4 (GPT-4) while 5 were written by plastic surgery trainees. Surveys were electronically distributed to participants including plastic surgeons, trainees, and laypeople who were blinded to authorship. Respondents were asked to rate 8 abstracts individually using a 12-point quality scale, and compared 4 abstracts side-by-side. Writing quality and the association between training levels and discrimination abilities were assessed using paired T-tests and regression models. RESULTS: Of 355 surveys, 174 (49%) were completed. Respondents averaged 29 years, 56% were females, 68.5% Caucasians, 51% hold/pursue doctorate degrees, and 27.3% had a bachelor’s degree or lower. Respondents in medicine constituted 63% medical students, 21% residents, and 17% attendings. Mean scores showed no significant difference between AI and human-written abstracts (9.9 vs. 10.2, P=0.11). Attendings favored AI abstracts, while other levels preferred human-written abstracts. In side-by-side comparisons, 71% selected human-written abstracts as superior. Risk-adjusted analysis showed medical students and residents more likely to prefer human-written abstracts over AI, compared to laypeople and attendings (P<0.001). CONCLUSION: GPT-4 can generate scientific abstracts comparable to human-written. Trainees exhibit better differentiation abilities and all readers discriminate better when compared side-by-side. The implications of AI in scientific writing warrant discussion due to its human-level content synthesis capability.
Description
Citation
Plastic & Reconstructive Surgery-Global Open 12(S5):p 89, May 2024. | DOI: 10.1097/01.GOX.0001018668.18607.af
Citation to related work
Ovid Technologies (Wolters Kluwer Health)
Has part
Plastic and Reconstructive Surgery - Global Open, Vol. 12, Iss. S5
ADA compliance
Embedded videos
License
Attribution-NonCommercial-NoDerivs CC BY-NC-ND
cbn