Integration of AI Vision with Medical Imaging Systems: Enhancing Diagnostic Accuracy in Orthopedic Radiology

Abstract

This study compared the accuracy of ChatGPT Vision (GPT-4V, OpenAI; accessed April 2025) to four orthopedic specialists on 113 orthopedic X-ray images representing fractures, dislocations, and normal anatomical variants. To achieve reproducibility, a structured radiological prompt was used in a standardized manner for all images. The performance of the diagnosis was quantified by accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and balanced accuracy, The major metrics calculated are F1-score, and Cohen's Kappa coefficient, for which 95% Wilson confidence intervals (CIs) are shown. The model achieved an overall accuracy of 85.8% (95% CI: 78.2%–91.1%) and high sensitivity of 95.3% (95% CI: 88.5%–98.2%); however, specificity was moderate at 57.1% (95% CI: 39.1%–73.5%), Such systematic false-positive tendency was especially observed on the mobile-captured radiographs, which had a 2.4-fold higher rate of false-positive than standard radiographs. Note the balanced accuracy of 76.2% and F1-score of 91.0% to give an additional context about the class-imbalanced dataset (85 positive, 28 negative cases). Cohen's Kappa of 0.58 (95% CI: 0.389–0.771) indicates moderate human–AI agreement. The results validate ChatGPT Vision's potential as a valuable initial screening tool for orthopedic triage, highlighting the importance of physician supervision and the need for domain-specific model optimization.