Validation of a Novel Hip Arthroscopy Simulator: Establishing Construct Validity

Objective

The purpose of this study was to establish the construct validity of a novel VR simulation system, the VirtaMed ArthroS for HA by comparing various parameters between groups of novice, intermediate and experienced arthroscopists. We hypothesized that more experienced arthroscopists would outperform less experienced arthroscopists for all measured metrics, thus supporting the novel simulator’s construct validity.

Design

For the purpose of this study, three cohorts were identified a priori by level of experience with respect to prior arthroscopy volume. The three cohorts were stratified as novice. Subjects filled out a questionnaire prior to participating in the surgical tasks, including hand dominance, sub-specialty interest, total number of previous arthroscopic procedures performed and total number of previous HA procedures performed. In addition to categorizing trainees based on number of procedures performed, participants were also categorized level of training (PGY 1 through 5, fellow, staff) for Pearson correlation analysis.

Setting and Participants

Sixteen orthopaedic surgery residents, four senior medical students and one orthopaedic sports medicine fellowship-trained attending at a single academic tertiary institution participated in this prospective study in June 2018.

Results

On an average, the novice cohort performed 0.25 arthroscopic procedures (standard deviation [SD]: 0.66, range: 0–2), the intermediate cohort 36 (SD: 9.6, range: 28–50) and the experienced cohort performed 395 (SD: 465.64, range 80–1200) prior to participation in the study. On an average, the intermediate cohort performed four hip arthroscopies (SD: 5.68, range: 0–15) and the experienced cohort performed 52 hip arthroscopies (SD: 85.74, range: 1–200), whereas the novice cohort did not have any prior experience with HA.

Composite performance score was significantly lower in the novice cohort (114.5 ± 14.91) compared with the intermediate (146.4 ± 17.17) and experienced cohort (151.5 ± 17.18) (P < 0.002). Novice arthroscopists performed the module in an average time of 321 s (SD: 67.04) compared with 202 s (SD: 36.26) and 181 s (SD: 24.52) in the intermediate and experienced cohort, respectively (P < 0.002). Camera path length did not differ significantly between groups with an average of 202 ± 42.2 cm, 172 ± 74.5 cm, and 147 ± 59.4 cm in the novice, intermediate and experienced cohorts (P = 0.3804), respectively. Additionally, cartilage damage and safety score did not differ significantly between groups with a mean safety score of 78.75 ± 10.44, 82.11 ± 9.81 and 82 ± 6.52 in the novice, intermediate and experienced cohorts (P = 0.775), respectively

Conclusions

The ArthroS hip simulator shows good construct validity and performance correlates highly with total number of arthroscopic cases reported during training. Certain metrics, such as simulated iatrogenic cartilage injury, cannot distinguish between different levels of surgical experience. The ArthroS composite score may be a useful metric to assess progress for trainees learning HA.