Effect of clinician’s experience and expertise on the inter- and intra-observer reliability of the computed tomography-based classification systems in posterior malleolus fractures

Objective: Three computed tomography (CT)-based classifications, including Haraguchi, Bartoníček–Rammelt, and Mason–Molloy systems, have been developed to better determine the characteristics of posterior malleolus fractures (PMFs). The aims of this study were (1) to determine the intra- and inter-observer reliability of the computed tomography-based classification systems and (2) to investigate the clinical experience and expertise on their reliabilities.

Methods: Sixty-seven preoperative ankle CT scans of 67 adult patients with “ankle fracture with the involvement of the posterior malleolus”’’ were retrospectively identified. CT images were assessed by 10 observers with different levels of clinical experience from 2 different specialties, including orthopedics and radiology. The observers were asked to classify PMFs according to 3 CT-based classifications on 2 separate occasions with a 4-week interval. Cohen’s κ values were measured for 2 raters and Fleiss’ κ values were measured for 3 raters and more.

Results: Overall, each classification had moderate to very good intraobserver reliability (κ=0.41 to 0.95, P < .01) as well as moderate interobserver reliability for each of the 2 separate assessments (κ=0.41 to 0.60, P < .01 for the first occasion; κ=0.44 to 0.59, P < .01 for the second occasion). Interobserver agreement among the foot and ankle surgeons regarding the Haraguchi and Bartoníček classifications was substantial for both assessment periods, whereas there was a moderate agreement for the Mason classification. Orthopedic residents showed moderate interobserver agreement in each period for both Bartoníček and Mason classifications. Radiology experts illustrated slight and fair agreements in the 2 assessments for Mason classification, moderate agreement in both assessments for Haraguchi classification, and substantial to moderate agreement in the first and second assessments for Bartoníček classification, respectively.

Conclusion: Computed tomography-based classification system for PMFs demonstrated moderate interobserver reliability as well as moderate to very good intraobserver reliability. Moreover, foot and ankle specialists exhibit enhanced K values for both inter and intraobserver reliability for each classification system, consistency seems to increase as the interest in the field condenses.

Level of Evidence: Level III, diagnostic study.