Model development
1. Model development
SPEAKER: Let's advance to the second stage, model development, where you train the model and evaluate the result. Now that our data is ready, which, if you return to the cooking analogy, is the ingredients, it's time to train the model. This is like experimenting with recipes. This stage involves two steps, model training, which is like cooking the recipe, and model evaluation, which is like testing how good the meal is. This process might be iterative. To set up an ML model, you need to specify a few things. Please note that the user interface may not be exactly the same, as the product is evolving rapidly. However, similar features will remain. First of all is the training method, where you tell Vertex AI the data set you just uploaded from the preparation stage. Depending on the data type, whether it is tabular, image, text, or video, you specify the training objective. This is the goal of the model training and the task you want to solve. Then you choose the training method, AutoML, without code, or custom training, using code. The next step is to determine the training details. For example, if you are training the model to solve a supervised learning problem, such as regression and classification, you must choose the target column from your data set. In Training Options, you can choose certain features to participate in the training and transform the data type if needed. Finally, you specify the budget and pricing and then click Start Training. AutoML will train the model for you and choose the best performed models among thousands of others. Do you recall the powerful technologies behind AutoML? Right. The credit there goes to neural architecture search and transfer learning. While you were experimenting with a recipe, you need to keep tasting it to ensure that it meets expectations. This is the evaluation portion of the model development stage. Vertex AI provides extensive evaluation metrics to help determine a model's performance. Let's focus on the metrics of recall and precision when evaluating the performance of classification models. To do this, you'll use a confusion matrix. A confusion matrix is a specific performance measurement for machine learning classification problems. It's a table with combinations of predicted and actual values. To keep things simple, we assume the output includes only two classes. Let's explore an example. The first is true positive, which can be interpreted as the model predicted positive, and that's true. The model predicted that this is an image of a cat, and it actually is. The opposite of that is true negative, which can be interpreted as the model predicted negative, and that's true. The model predicted that the image is not a cat, and it actually isn't. Then there is false positive, otherwise known as a type I error, which can be interpreted as the model predicted positive, and that's false. The model predicted that the image is a cat, but it actually isn't. Finally, there is false negative, otherwise known as a type II error, which can be interpreted as the model predicted negative, and that's false. The model predicted that the image is not a cat, but it actually is. A confusion matrix is the foundation for many other metrics used to evaluate the performance of a machine learning model. Let's look at the two popular metrics, recall and precision, that you will encounter in the lab. Recall refers to all the positive cases and looks at how many were predicted correctly. This means that recall is equal to the true positives divided by the sum of true positives and false negatives. Precision refers to all the cases predicted as positive and how many are actually positive. This means that precision is equal to the true positives divided by the sum of the true positives and false positives. Precision and recall are often a trade off. Depending on your use case, you might need to optimize for one or the other. Consider a classification model where Gmail separates emails into two categories, spam and not spam. If the goal is to catch as many potential spam emails as possible, Gmail might want to prioritize recall. In contrast, if the goal is to only catch the messages that are definitely spam without blocking other emails, Gmail might want to prioritize precision. Vertex AI visualizes the precision recall curve, so it can be adjusted based on the problem that needs to be solved. You'll get the opportunity to practice adjusting precision and recall in the AutoML lab. In addition to the confusion matrix and the metrics generated to measure recall and precision, the other useful measurement is feature importance. In Vertex AI, feature importance is displayed through a bar chart to illustrate how each feature contributes to a prediction. The longer the bar or the larger the numerical value associated with the feature, the more important it is. This information helps decide which features are included in a machine learning model to predict the goal. You will observe the feature importance chart in the lab, as well. Feature importance is just one example of Vertex AI's comprehensive machine learning functionality called explainable AI. Explainable AI is a set of tools and frameworks to help understand and interpret predictions made by machine learning models. Please check the reading list if you want to know about explainable AI on Google Cloud.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.