2026-08-06 16:08 UTC
An interesting evolution from Mike Thelwall. A year ago, he was cautiously exploring whether LLMs could approximate expert research assessment. Today, he reports that ChatGPT-5 mini reached correlations of up to 0.905 with expert REF2021 quality assessments across some disciplines, using a dataset of more than 107,000 UK research articles.
:doi: https://doi.org/10.1515/jdis-2026-0058
#ResearchEvaluation #Scientometrics #ResearchAssessment #ChatGPT #LLM #AI #SciencePolicy
Replies (2)
-
@vnikolov@ieji.de 2026-08-06 16:16
Have these articles been included in ChatGPT-5mini's training set? @serhii@mstdn.science
-
@kfort@social.sciences.re 2026-08-06 16:27
@serhii@mstdn.science Were these research papers in the pre-training dataset?