AI-based surgical prediction models have limits

Prediction models generated by machine learning are being increasingly used in medicine to identify risk factors and possible outcomes, especially for total joint replacements of knees and hips—although researchers warn that machine-generated predictions are currently being drawn from a limited data pool.

“Machine learning has great potential for processing ‘big data’ and has proved its undeniable capability, although it is not free of issues,” warns Dr. Reza Hashemi from Flinders University’s College of Science and Engineering.

“The accuracy of predictive models is dependent on the quality of the data sources, and predictions may be significantly affected by the amount of data and the number of variables included.”

“At present, predictive models developed for total hip reconstruction and total knee reconstruction are based mainly on patient-reported factors and imaging variables. Therefore, the output of machine learning models in this area needs to be interpreted carefully.”

To study the application of supervised machine learning in predictive modeling for post-operative outcomes of total hip and knee replacements, Flinders researchers together with collaborators from the Australian Orthopedic Association National Joint Replacement Registry (AOANJRR), Royal Adelaide Hospital and UniSA assessed the most-widely used machine learning techniques, data sources, domains, limitations of predictive analytics and the quality of predictions.

The research, “Supervised machine learning for the prediction of post-operative clinical outcomes of hip and knee replacements: a review,” has been published in ANZ Journal of Surgery.

“The most widely used machine learning approach in medical sciences is ‘supervised learning,’ which estimates the mapping function for new input data in order to predict categorized, real values, or time-to-event outputs,” says research co-author, Flinders University’s Dr. Khashayar Ghadirinejad.

Conventional statistical methods of risk predictions rely on predetermined assumptions and mathematical equations to formalize relations between the variables, whereas machine learning techniques use large amounts of available data to recognize these relationships.

In assessing the effectiveness of machine learning to assist with total hip replacement and total knee replacement procedures, the researchers note that care should be taken by the medical profession when dealing with limited data on specific subjects.

Dr. Ghadirinejad suggests machine learning models should now be assessed and evaluated using a randomized cohort of studies and controlled trials in real-world settings, rather than just assessing data. “More improvements are needed in machine learning orthopedic applications to translate research aims into clinical practices,” he says.

Despite the current limitations of machine learning, the researchers recognize there is still a need for models that can predict various outcomes such as the early identification of prostheses outliers based on the available big data from the national joint registries around the world.

Joint registries aim to reduce the revision rates of arthroplasty surgeries by early detection of outlier joint arthroplasty devices. They provide population-based data on the comparative outcome of prostheses within the community. Joint registries, in particular the Australian Joint Registry, put efforts to significantly control the harm and cost of using poor-performing devices in hip and knee replacement surgeries.

The authors also suggest a future direction for machine learning in the domain of joint arthroplasty could be to develop decision-making support systems focused on pre-surgical predictions that enable surgeons to determine what is the best for their patients individually.