1. Bivariate Gaussian Mixture Models with flexmix
In this lesson we will estimate the bivariate Gaussian mixture model using flexmix.
2. Bivariate Gaussian mixture model
Recall that each bivariate Gaussian distribution is explained by the mean and the covariance matrix.
The cross-terms in the covariance matrix correspond to the measure of joint variability between the variables, which gives the oval shape shown here.
To understand how the cross-term affects the results, we will fit the model with and without these quantities.
3. Fit bivariate Gaussian mixture model
Let's start by fitting the model without the cross-terms.
Observe that the code is pretty similar to the one exposed for the univariate case.
The differences stem from the formula, since we are now taking into account two variables instead of one, and from the model since we are now modeling the clusters as bivariate Gaussian distributions.
The argument `diag` equals true, specifying that the fitting should consider a diagonal covariance matrix, that is, without cross-terms.
4. The proportions: prior() function
As before, the function `prior()` gives the proportions of each cluster.
In this case, we find that 53% corresponds to cluster 1 and 47% to cluster 2.
5. parameters() function
Observe that the function `parameters()`, without specifying the components, gives back the means formed by the first two rows and the covariance matrices formed by the following four values.
So let's extract these parameters properly.
6. Extract the means
First, we extract each component estimation and save them into comp_1 and comp_2 for clusters 1 and 2.
This way, each mean corresponds to the first two values of this object.
The first cluster has a mean of 186, for the variable weight, and a value of 27, for the variable BMI.
The second cluster has a mean of 133, for weight, and 23, for BMI.
7. Extract the diagonal covariance matrices
To extract the covariance matrices, we specify that the last four values should be wrapped into a squared matrix of dimension 2.
As expected, the cross-terms of both matrices correspond to zero.
8. Ellipse curves
Now that we have extracted the means and the covariance matrices, we would like to visualize the clusters.
To do so, we can create an ellipse curve that encloses the 95% of each cluster with the function `ellipse()` from the `ellipse` package.
The `x` argument takes the covariance matrix, the `center` argument takes the mean and the `npoints` argument specifies the number of points used in the curve.
In this case, we are using the number of points as the number of rows.
9. Visualize the resulting distributions
To plot the observations together with the ellipses, we use the functions `geom_point()` and `geom_path()`, respectively.
Geom_path connects the observations in the order in which they appear in the data.
To distinguish both clusters, we give the colour red to the first cluster and blue to the second.
10. Visualize the resulting distributions
Looking at the figure, it doesn't seem that the model fits the data very well.
The reason is because we have imposed that there is no joint variability between these two variables, that is, we have considered that the cross-terms are zero.
Let's see how the model changes when considering the cross-terms different from zero.
11. Fit bivariate Gaussian mixture model
For doing that, we only need to specify that the covariance matrices are not diagonal through the argument `diag` equals false.
All the other arguments of the `flexmix` function remain the same.
If we proceed as before with the creation of the ellipses, what we find is two subpopulations that fit the data much better.
12. Plot the new GMM
Keep in mind that the ellipses do not incorporate the information of the proportions and are just a handy way to illustrate the clusters.
13. Let's practice!
Now let's try some examples.