Fact Checked by 
AI large language models (LLMs) can predict the results of neuroscience studies more accurately than human experts, a new study found.
Researchers from the University College London (UCL) found that LLMs trained on vast datasets of text can distill patterns from scientific literature which allows them to forecast scientific outcomes with superhuman accuracy.
The team involved say the findings highlight LLM’s potential as a powerful tool for accelerating research, going far beyond just knowledge retrieval.
The study consisted of testing 15 different general-purpose language models and 171 human neuroscience experts, who had all passed a screening test to confirm their expertise.
The question was whether the AI or human could correctly determine which of the two paired abstracts was the real one with the actual study results.
AI beats neuroscience experts in study: ‘Won’t be long before scientists are using AI tools…’
In a press release by UCL, the researchers said: “All of the LLMs outperformed the neuroscientists, with the LLMs averaging 81% accuracy and the humans averaging 63% accuracy.
“Even when the study team restricted the human responses to only those with the highest degree of expertise for a given domain of neuroscience (based on self-reported expertise), the accuracy of the neuroscientists still fell short of the LLMs, at 66%.
“Additionally, the researchers found that when LLMs were more confident in their decisions, they were more likely to be correct.”
The researchers then took the study one step further as they adapted a version of Mistral (French generative AI) by training it specifically on neuroscience literature. This new version was even better at predicting study results, with an 86% accuracy score.
Senior author Professor Bradley Love (UCL Psychology & Language Sciences) said: “In light of our results, we suspect it won’t be long before scientists are using AI tools to design the most effective experiment for their question. While our study focused on neuroscience, our approach was universal and should successfully applied across all of science.
“What is remarkable is how well LLMs can predict the neuroscience literature. This success suggests that a great deal of science is not truly novel, but conforms to existing patterns of results in the literature. We wonder whether scientists are being sufficiently innovative and exploratory.”
