This Industry Insights: AI focuses on the use of AI in academic writing, ranging from writing summaries and providing feedback to conducting research and writing a paper.
Researchers from Stanford have created an automated pipeline using GPT-4 to provide comments on scientific papersthat were quantitatively compared with human peer-reviewed feedback. The overlap in points raised by GPT-4 and human reviewers was compared to the overlap between two human reviewers. The researchers also conducted a prospective study to understand how researchers perceived the feedback generated by the system. Overall, 57.4% found the GPT-4-generated feedback helpful or very helpful, and 82.4% found it more beneficial than feedback from at least some human reviewers. They stress that this would not be a method to remove human peer review; rather, this could have important use in improving drafts prior to submission and could fill a void for researchers who do not have access to experienced mentors who can provide feedback.
bioRxiv is a pre-print server that allows for the rapid dissemination of scientific information prior to formal peer review and publication. bioRxiv has partnered with ScienceCast, an AI start-up, to automatically generate multi-level summaries of manuscript preprints that target different reading levels. The aim is to make new research more understandable and accessible by generating summaries that analyse the full manuscript text. Although they have noted factual errors in summaries, they have found most to be accurate. It is expected that these large language models will continue to improve as they ingest more data and undergo further fine-tuning.
Two researchers from the Israel Institute of Technology have used freely available data on 250,000 people from the Centers for Disease Control. They wrote a programme to prompt ChatGPT in a certain way, and in under an hour, they were able to guide ChatGPT through a process that mirrors that of the scientific process to end up with a completed manuscript. The findings were not something that was novel in the understanding of diabetes, but rather highlighted how AI could be incorporated into research and publications.
However, there were several limitations, including the creation of false narratives, the generation of fake citations, and the regurgitation of existing ideas. It is unlikely that AI will be able to generate full scientific research without human checks in the near future; however, this research highlights how AI could potentially be used to help the scientific process, allowing researchers to do more insightful research.
