Personalized assessment and training of neurosurgical skills in virtual reality: An interpretable machine learning approach

Surgical skill training is transforming with the use of virtual reality technology in surgical simulators, which offers new assessment and training methods. While virtual reality surgical simulators have gained popularity as a training tool and objective means of evaluating surgical skills, the current assessment of surgical skills tends to be subjective. During a virtual reality simulation task, the surgical simulator collects a large amount of surgery-related psychomotor data. These data are commonly used as performance indicators to assess the skill levels of trainees and play a crucial role in the simulator′s ability to evaluate and train them. Performance indicators have been employed in several studies to evaluate the effectiveness of simulator training, categorize user skill levels, and develop effective training programs for skill improvement. However, as most surgical procedures require diverse and complex psychomotor skills involving both hands, individual

metrics may ineffectively assess surgical expertise. Therefore, multiple metrics must be combined to analyze different datasets. Machine learning algorithms use a large dataset provided by a surgical simulator and can classify surgical expertise with greater accuracy. These algorithms offer new ideas for differentiating surgical skills at different levels of expertise. Machine learning algorithms have been applied to surgical simulators, for example, in assessing skills in vascular interventions, robot-assisted surgery, and spinal surgery. Siya et al. employed a linkage-weighting technique to determine the relative importance of each parameter from neural network data to discriminate between surgical performances in virtual reality. However, instead of using many hidden layer networks, most studies using connection-weighting techniques employ a single hidden-layer neural network. Therefore, understanding the performance of each participant feature is important, as well as developing an individualized plan for participant training. Shapley values are used to interpret the contribution of specific attributes to the prediction of a specific query point. This study selected five classifiers to distinguish skill levels: decision tree, linear discriminant analysis (LDA), naïve Bayes, support vector machine (SVM), and Knearest neighbor (KNN). This study aimed to investigate the potential of machine learning as a skill level predictor in virtual reality neurosurgery and explain the underlying mechanisms of machine learning models. To this end, this study aimed to address two questions: (1) Can machine learning be used to identify the skill levels of participants performing virtual reality neurosurgery? and (2) Can the machine learning model be interpreted to understand the performance of each metric for each participant?