Stranded Model Tutorial

This vignette details why the stranded_model dataset was created, how to load it, and gives examples of use with the caret Machine Learning library.

The dataset contains:

stranded.label: a character metric to indicate whether the patient is stranded, or not
age: Integer - the age of the patient on admission to hospital
care.home.referral: Integer - flag to indicate referred from care home
medicallysafe: Integer - flag to indicate whether the patient is medically safe for example safe to be discharged but hasn’t been
hcop: Integer - flag to indicate whether the patient is in a Health Care for Older People area
mental_health_care: Integer - flag to indicate mental health care provision
period_of_previous_care: Integer - flag to indicate previous periods of care
admit_date: Date - admit date
frailty_index: Character - specifying frailty type, if frail

First, load the data and inspect it

library(NHSRdatasets)
library(dplyr)
library(ggplot2)
library(caret)
library(rsample)
library(varhandle)

data("stranded_data")
glimpse(stranded_data)
#> Rows: 768
#> Columns: 9
#> $ stranded.label           <chr> "Not Stranded", "Not Stranded", "Not Stranded…
#> $ age                      <int> 50, 31, 32, 69, 33, 75, 26, 64, 53, 63, 30, 7…
#> $ care.home.referral       <int> 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, …
#> $ medicallysafe            <int> 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, …
#> $ hcop                     <int> 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, …
#> $ mental_health_care       <int> 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, …
#> $ periods_of_previous_care <int> 1, 1, 1, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 4, …
#> $ admit_date               <chr> "29/12/2020", "11/12/2020", "19/01/2021", "07…
#> $ frailty_index            <chr> "No index item", "No index item", "No index i…
prop.table(table(stranded_data$stranded.label))
#> 
#> Not Stranded     Stranded 
#>    0.5963542    0.4036458

This is good, it shows a relatively even split between the not stranded and stranded labels. Please refer to the webinar on Advanced Modelling to look at how you can deal with classification imbalance using techniques such as SMOTE (Synthetic Minority Oversampling Technique Estimation) and ROSE (Random Oversampling Estimation), to name a few.

Feature engineering

The next step will be to decide which features need to be engineered for our machine learning model. We will drop the admit_date and recode the frailty index, and perhaps allocate the age into age bands.

stranded_data <- stranded_data %>%
  dplyr::mutate(stranded.label = factor(stranded.label)) %>%
  dplyr::select(everything(), -c(admit_date))

Next, I will select the categorical variables and make these into dummy variables, that is a numerical encoding of a categorical variable:

cats <- select_if(stranded_data, is.character)
cat_dummy <- varhandle::to.dummy(cats$frailty_index, "frail_ind")
# Converts the frailty index column to dummy encoding and sets a column called "frail_ind" prefix
cat_dummy <- cat_dummy %>%
  as.data.frame() %>%
  dplyr::select(-frail_ind.No_index_item) # Drop the field of interest
# Drop the frailty index from the stranded data frame and bind on our new encoding categorical variables
stranded_data <- stranded_data %>%
  dplyr::select(-frailty_index) %>%
  bind_cols(cat_dummy) %>%
  na.omit(.)

The data is now ready for splitting into a simple train and validation split, to do the machine learning on the set.

Splitting the data

The next step is to create a simple hold out train/test split:

split <- rsample::initial_split(stranded_data, prop = 3 / 4)
train <- rsample::training(split)
test <- rsample::testing(split)

Create simple Logistic Regression Model to classify stranded patients

The next step will be to create a stranded classification model, in CARET:

set.seed(123)
glm_class_mod <- caret::train(factor(stranded.label) ~ .,
  data = train,
  method = "glm"
)
print(glm_class_mod)
#> Generalized Linear Model 
#> 
#> 524 samples
#>   9 predictor
#>   2 classes: 'Not Stranded', 'Stranded' 
#> 
#> No pre-processing
#> Resampling: Bootstrapped (25 reps) 
#> Summary of sample sizes: 524, 524, 524, 524, 524, 524, ... 
#> Resampling results:
#> 
#>   Accuracy  Kappa    
#>   0.758219  0.3771315

This is a very basic model and could be improved by model choice, hyperparameter selection, different resampling strategies, etc.

Predicting the test set to validate model

Next, we will use the test dataset to see how our model will perform in the wild:

preds <- predict(glm_class_mod, newdata = test) # Predict class
pred_prob <- predict(glm_class_mod, newdata = test, type = "prob") # Predict probs

# Join prediction on to actual test data frame and evaluate in confusion matrix

predicted <- data.frame(preds, pred_prob)
test <- test %>%
  bind_cols(predicted) %>%
  dplyr::rename(pred_class = preds)

glimpse(test)
#> Rows: 175
#> Columns: 13
#> $ stranded.label                 <fct> Not Stranded, Not Stranded, Not Strande…
#> $ age                            <int> 26, 64, 30, 80, 31, 68, 43, 69, 62, 22,…
#> $ care.home.referral             <int> 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, …
#> $ medicallysafe                  <int> 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, …
#> $ hcop                           <int> 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, …
#> $ mental_health_care             <int> 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, …
#> $ periods_of_previous_care       <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 1, 1, …
#> $ frail_ind.Activity_Limitation  <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, …
#> $ frail_ind.Fall_patient_history <dbl> 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, …
#> $ frail_ind.Mobility_problems    <dbl> 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, …
#> $ pred_class                     <fct> Not Stranded, Not Stranded, Not Strande…
#> $ Not.Stranded                   <dbl> 0.7795553767, 0.8270208028, 0.749439588…
#> $ Stranded                       <dbl> 0.2204446, 0.1729792, 0.2505604, 0.3148…

Evaluating with confusion matrix

The final step is to evaluate the model:

caret::confusionMatrix(test$stranded.label, test$pred_class, positive = "Stranded")
#> Confusion Matrix and Statistics
#> 
#>               Reference
#> Prediction     Not Stranded Stranded
#>   Not Stranded          111        3
#>   Stranded               28       33
#>                                           
#>                Accuracy : 0.8229          
#>                  95% CI : (0.7581, 0.8764)
#>     No Information Rate : 0.7943          
#>     P-Value [Acc > NIR] : 0.2015          
#>                                           
#>                   Kappa : 0.5689          
#>                                           
#>  Mcnemar's Test P-Value : 1.629e-05       
#>                                           
#>             Sensitivity : 0.9167          
#>             Specificity : 0.7986          
#>          Pos Pred Value : 0.5410          
#>          Neg Pred Value : 0.9737          
#>              Prevalence : 0.2057          
#>          Detection Rate : 0.1886          
#>    Detection Prevalence : 0.3486          
#>       Balanced Accuracy : 0.8576          
#>                                           
#>        'Positive' Class : Stranded        
#>

The model performs relatively well and could be improved by better predictors, a bigger sample and class imbalance techniques.

Conclusion

This dataset can be used for a number of classification problems and can be the NHS’s equivalent to the iris dataset for classification, albeit this only works for binary classification problems.