Predicting the Unpredictable: Using Machine Learning to Assess Preoperative Risk

Introduction

Surgery, though often a life-saving intervention, inherently carries risks. Accurate preoperative risk prediction is crucial for informed consent, patient safety, and efficient resource allocation. My recent college project explored the transformative potential of machine learning (ML) in revolutionizing preoperative risk assessment. This blog post delves into my findings, making complex technical details accessible even if you're not a tech expert.

The Limitations of Traditional Risk Assessment

Traditionally, clinicians have relied on scoring systems like POSSUM and ASA classifications to estimate surgical risk. While valuable, these systems often lack the granularity and personalization required for truly informed decision-making. For instance, the ASA classification, as shown in my project's data analysis, categorizes patients into risk groups based on their overall health status. However, it doesn't account for the nuances of individual patient profiles or specific surgical procedures.

The Power of Machine Learning

Imagine a system capable of analysing vast quantities of patient data – demographics, medical history, lab results, surgical details, and more – to predict the probability of complications such as mortality, ICU admission, or extended hospital stays. This is the promise of ML in healthcare.

My Project: A Deep Dive into Preoperative Mortality Prediction

My project focused on developing an ML model to predict postoperative mortality. I utilized the VitalDB dataset, a real-world collection of electronic health records, to train and evaluate different ML algorithms.

Methodology at a Glance:

1. Data Pre-processing: This crucial first step involved cleaning the data, handling missing values, and transforming categorical variables into a format suitable for ML algorithms. For example, I used one-hot encoding to represent categorical features like ASA classification.

2. Exploratory Data Analysis (EDA): Before diving into model building, I performed EDA to understand the underlying patterns and relationships within the data. This involved visualizing the distributions of key variables like age and BMI, examining correlations between features, and analysing mortality rates across different ASA classifications.

   

3. Model Training: I experimented with a range of ML algorithms, each with its strengths:

   • ExtraTreesClassifier: A robust algorithm that builds multiple decision trees and combines their predictions, known for its ability to handle high-dimensional data and prevent overfitting.

   • LGBMClassifier (LightGBM): A gradient boosting framework renowned for its efficiency in handling large datasets and its speed, making it suitable for complex prediction tasks.

   • RandomForestClassifier: Another ensemble learning method that constructs multiple decision trees, offering good performance with minimal hyperparameter tuning.

4. Hyperparameter Tuning: I fine-tuned each model's parameters using techniques like grid search and cross-validation to optimize their performance and ensure they generalize well to unseen data.

5. Model Evaluation: To assess the models' effectiveness, I used metrics such as ROC-AUC score, which measures the model's ability to distinguish between classes (in this case, patients who survived versus those who didn't), and accuracy.

Promising Results: A Glimpse into the Future

My ML models achieved impressive accuracy in predicting postoperative mortality, exceeding the performance of traditional risk assessment methods. Here's a snippet of the Python code I used to evaluate the ExtraTreesClassifier:

from sklearn.metrics import accuracy_score, roc_auc_score 
# # Create the ExtraTreeClassifier
extra_tree_model = ExtraTreeClassifier(random_state=42)
# Define hyperparameters for tuning
param_grid = {
'max_depth': [32],
'min_samples_split': [2, 5, 10],
'min_samples_leaf': [1, 2, 4]
}
# Create GridSearchCV instance
grid_search = GridSearchCV(extra_tree_model, param_grid, cv=5,␣
↪scoring='accuracy', n_jobs=-1)
# Fit the model on the training data
grid_search.fit(X_train, y_train)
# Get the best hyperparameters
best_params = grid_search.best_params_
print(best_params)
# Train the ExtraTreeClassifier with the best hyperparameters on the combined␣
↪train and validation sets
best_extra_tree_model = ExtraTreeClassifier(**best_params, random_state=42)
best_extra_tree_model.fit(np.concatenate((X_train, X_validation)), np.
↪concatenate((y_train, y_validation)))
# Make predictions on the test set
predictions_test = best_extra_tree_model.predict(X_test)
# Make predictions on the validation set
predictions_validation =best_extra_tree_model.predict(X_validation)
y_pred_prob_ml1 = best_extra_tree_model.predict_proba(X_test)[:, 1] # [:, 1]␣
↪for the positive class label
# y_pred_prob_ml1 now contains the predicted probabilities for the positive␣
↪class

# Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
roc_auc = roc_auc_score(y_test, y_prob)

print(f"Accuracy: {accuracy:.4f}")
print(f"ROC-AUC Score: {roc_auc:.4f}")

Beyond Accuracy: Interpretability and Real-World Application

Building accurate models is just the first step. Understanding why a model makes a specific prediction is critical for clinicians to trust and act upon the model's output. My project explored techniques like feature importance analysis to shed light on the factors driving risk predictions. For instance, I found that preoperative creatinine levels and ASA classification were among the most influential features in predicting postoperative mortality.

Shaping the Future of Preoperative Risk Assessment

Imagine a future where ML models are seamlessly integrated into hospital systems, providing doctors with real-time risk assessments and personalized treatment recommendations at their fingertips. This could lead to:

• Improved Patient Outcomes: By identifying high-risk patients early on, doctors can proactively optimize care, potentially preventing complications and improving surgical outcomes.

• Reduced Healthcare Costs: More accurate risk stratification can help avoid unnecessary procedures and allocate resources more efficiently.

• Enhanced Patient Engagement: ML-powered tools can empower patients with personalized risk information, fostering informed decision-making and active participation in their care.

Conclusion

My project showcased the immense potential of machine learning to transform preoperative risk assessment from a one-size-fits-all approach to a personalized and data-driven process. While further research and development are essential before widespread clinical implementation, the future of healthcare is bright, empowered by the fusion of data science and medical expertise.

Call to Action:

• For healthcare professionals: Embrace the potential of ML, advocate for its integration into clinical workflows, and actively participate in research and development efforts.

• For patients: Stay informed about advancements in healthcare technology, engage in open dialogues with your doctors about your risk assessment, and don't hesitate to ask questions.