1. Visualization and prediction with Weibull model
After fitting a Weibull regression model, we could visualize it or use it to make inferences in many different ways.
2. .plot()
An alternative way to view the coefficients other than printing the model summary is to use the dot-plot method. This plot also shows the 95% confidence interval of each coefficient.
Keep in mind that we should use e to the power of the coefficients instead of the coefficients themselves for interpreting how much the survival function changes with a one-unit change in the covariate.
3. .plot_partial_effects_on_outcome()
To see how these coefficients manifest on the survival function, we could use a handy method called plot_partial_effects_on_outcome. This method returns a visual representation comparing the baseline survival curve of the model versus what happens when one or multiple covariates are varied over different values. This is useful to compare changes in the survival function as we vary specific covariate(s), all else being held equal.
4. How to plot partial effects?
Plot_partial_effects_on_outcomes takes at least two arguments - "covariates" which is the name(s) of covariate(s) to vary over, and "values" which is the value(s) we want the covariate(s) to take on. In this example, we want a toy covariate named "var" to take on values 0, 3, 6, 9, 12 and 15. The plot will automatically iterate over all the values specified to plot modified survival curves. The baseline survival curve is equal to the predicted survival curve at all average values in the original dataset.
5. How to plot partial effects?
There are many ways to specify the varying values. We could hard-code the values. We could also use a range function like numpy-dot-arange to return evenly spaced values within a given interval. If varying multiple covariates, we need to specify the values for each covariate using a multi-dimensional iterable or list of lists. If we used the formula parameter while fitting, luckily, all the necessary transformations will be made internally and automatically.
6. Mortgage example
Let's see an example using the mortgage DataFrame with covariates!
7. Mortgage example
Assume we have fitted a Weibull AFT model using all covariates and want to see how different credit scores affect the baseline survival curve. We run plot_partial_effects_on_outcome, specify "credit score" as the covariate, and a set of values ranging from 700 to 860 incrementing at 30. The plot shows that increasing credit score reduces the average time to event.
8. Predict survival functions
Often we obtain new data and need to ask questions about their future survival. From the partial effects plot, we see that individuals may have different survival curves based on their covariates' values. The predict method of WeibullAFTFitter uses the fitted model to predict new individuals' survival curves and durations.
9. Predict survival functions
The method predict_survival_function takes a covariate array or DataFrame as arguments and returns a DataFrame with predicted survival functions. The predict_median method predicts the median survival durations for the individuals and takes the same arguments as predict_survival_function.
10. Conditional after current durations
Both methods assume that subjects are new to the study, but we could pass the current durations of subjects to the conditional_after parameter to specify existing durations if known. Predictions made will be conditional after the existing durations. For example, a new subject's survival duration is predicted to be 4 time units. But we know that 2 time units have elapsed. Passing it to the conditional_after parameter will output 2 as the new prediction.
11. Let's practice!
Now let's practice visualizing and predicting with survival regression!