1. Interpreting model inference
In the previous lessons, you learned how to fit and interpret model coefficients. While we know that the model fit is the best fit given the data we still need information about the significance and size of the effects of the coefficients.
2. Estimation of beta coefficient
The regression coefficients are obtained by the maximum likelihood estimation, where the value of the parameters maximizes the probability of the observed data. Recall that the likelihood is the probability of data given coefficient estimates and that maximizing likelihood or loglikelihood is mathematically equivalent. The estimated beta coefficient is where the likelihood takes on its maximum value.
3. Estimation of beta coefficient
Unlike least squares estimation in linear regression, the maximization of the likelihood with respect to beta usually requires an iterative solution, which is sometimes called IRLS or iteratively reweighted least squares.
4. Significance testing
To test for the significance of the variables we use the information provided in the model summary, namely standard error, zvalue, its pvalue, and confidence intervals.
5. Standard error (SE)
Recall that the standard error is the standard deviation of a statistic, i.e. coefficient, and its value depends on the shape of the likelihood. Imagine the likelihood to be the mountain you are about to ascend. If the top is flatter it is hard to determine the summit, but if it sharp then the summit is easily found with less error. Similarly, with likelihood sharper at its peak, the location of the maximum is clearly defined with smaller standard error and vice versa.
6. Computation of the standard error
To compute the standard error we take the square root of the variance for the variable. The value of the variance we obtain from the diagonal entries of the model variance-covariance matrix, which is obtained using the cov_params function. The variance of weight is 0.142. Taking the square root we get 0.37 as in the model summary.
7. Significance testing
With significance testing we are concerned whether constraining the parameter values to zero would reduce the model fit. We use z statistic as the ratio of the estimated coefficient and its standard error, which follows the standard normal distribution. For zvalue greater than 2 we say that the variable is statistically significant. For our horseshoe crab model we see there is strong evidence of a positive effect of weight on the presence of satellite with zvalue of 4.819, concluding that weight is statistically significant.
8. Confidence intervals for beta
To report the results in a compelling way you should always report confidence intervals as they provide information on the uncertainty of the estimates. A large sample Wald confidence interval for the coefficient is computed as follows where beta is the estimate and SE its standard error.
9. Computing confidence intervals
In the horseshoe crab, the confidence interval for weight tells us that the change in the logodds can be as small as 1.07 or as much as 2.55 with 95 percent confidence.
10. Extract confidence intervals
Using the conf_int function we can extract the confidence intervals from the fitted model,
11. Extract confidence intervals
where zero denotes lower bound
12. Extract confidence intervals
and one gives upper bound for each variable.
13. Confidence intervals for odds
To interpret confidence intervals in terms of the odds, as we did with coefficients, recall that we used exponentiation to go from log odds to odds. Therefore, to obtain confidence intervals for the multiplicative effect on the odds of a unit increase in x we extract the confidence intervals for beta and exponentiate its endpoints. Using the numpy exp function this is computed directly. For the weight variable we can conclude that a unit increase in weight multiplies the odds by at least 2.93, from the lower bound, and at most by 12.85, from the upper bound, that a satellite is present.
14. Let's practice!
Time for practice problems.