Enhancing Research Integrity through Artificial Intelligence Validation of Manual Scoring

Clinical research is the cornerstone of evidence-based medicine and the foundation of modern healthcare. While the design and quality of studies can be discussed, it has, until now, been taken at face value that studies are at least based on reliable data. In recent years, it has become apparent that this assumption is in-correct. This is worrisome since these studies, specifically randomised clinical trials (RCTs), continue to inform medical guidelines and clinical practice. Several assessment tools and methods have been developed to detect such problematic studies. Manual scoring methods, such as the TRACT checklist, recalculating statistics, or assessing the work of one author, are often subject to human bias, inconsistency, and fatigue, which can affect the reliability and transparency of results, and above all they are labour-intensive and time-consuming, therefore expensive. Given the rapid advancement of artificial intelligence (AI) and its potential to substantially reduce manual effort, we are collaborating with the School of IT to harness AI in developing innovative solutions that will streamline and accelerate this work. Our group has already developed an automated system that uses artificial intelligence to score potential problematic articles, The current challenge lies in validating the accuracy of AI-generated assessment results against human evaluations, and in designing and developing an automatically generated report that is both comprehensive and easy-to-interpret. In this project, we will compare the results of our new automation program to the results of manual scoring that we have applied in the past. We will compare the feasibility, time needed, and accuracy of our methods. Ultimately, this project aims to develop a reliable AI software program that enhances transparency, reduces bias, and supports fairer decision-making in research evaluation. By validating AI verification, the project contributes to a more robust and trustworthy research ecosystem. By participating in this project, you will gain an understanding of the structure of evidence-based research and what defines trustworthy research. You will collaborate with students and experts with backgrounds in AI, allowing you to develop knowledge not only in healthcare research, but also in the rapidly advancing field of AI technology.