The efficiency of machine learning-assisted platform for article screening in systematic reviews in orthopaedics

Abstract
Purpose With the development of machine learning and artifcial intelligence, various platforms were developed to aid in the
time-consuming process of article screening in systematic reviews. We aim to analyze the efciency of a machine learning-
assisted platform as an end-user to aid in the screening of the articles for selection into systematic review in orthopaedic
surgery. Methods We included three previously published systematic reviews in the feld of orthopaedics of increasing levels of difculty in the structure of the research question to assess the efciency of a platform with active-learning technology for article screening. We compared the efciency of the platform compared to the traditional screening and also across the vari-ous scenarios tested. We performed fve iterations for each review analyzed. The outcome parameters analyzed were the work saved at 95% recall (WSS-95), work saved at 100% recall (WSS-100), and relevant records found after screening the frst 30% of the total records (RRF-30).
Results The machine learning-assisted screening signifcantly improved the rate of identifying the relevant records compared
to the traditional screening method (p<0.001). The WSS-95 for the easy, intermediate, and advanced screening scenarios
were 78%, 59%, and 38%, respectively. The WSS-100 for the easy, intermediate, and advanced screening scenarios were
75%, 48%, and 7%, respectively. The RRF-30 for the easy, intermediate, and advanced screening scenarios were 97%, 86%,
and 64%, respectively. We noted a signifcant reduction (p<0.001) in the efciency with the increasing level of difculty of
the screening scenarios.
Conclusion The machine learning platform is signifcantly better than the traditional method as an assistive technology to
aid in article screening. However, the efciency of the platform signifcantly decreases as the complexity of the research
question increases.