Introduction to Vertex AI Feature Store - Demo
1. Introduction to Vertex AI Feature Store - Demo
Did you know that most of the time spent by data scientists goes into wrangling data? More specifically in feature engineering, which is transforming raw data into high quality input signals for ML models. But this process is often inefficient and brittle. Well, I'm Priyanka and in this video we will identify the key challenges with feature engineering. How vertex feature store help solve them and see a quick demo now? What are the key challenges with ML features? The first is that they are hard to share and reuse across your different steps of the ML workflow and across projects which results in duplicate efforts. Second is that it is hard to serve and production reliably with lower latency, and the third is that there is an inadvertent skew in feature values between training and serving, usually, which causes your model quality to degrade over time. That is exactly where vertex feature store comes in. It's a fully managed and unified solution to share, discover and serve machine learning features at scale across different teams within your organization. And it also helps reduce the time to build and deploy your ML applications by making it easy to manage and organize your ML features in one place. It makes the features reusable, easy to serve, and avoids skew. Now let's see how to set it up in the console. In what exhibit we see the feature tab. To get started, let's click on this documentation and explore using feature store section. Now the first thing you need is a feature store. At the time of this recording feature stored is in preview. So just know that depending on when you're watching this, there might be more options and updates that you would see. You cannot create a feature store in the console, so let's use this sample notebook to learn how to created using the SDK. This sample uses a movie recommendations dataset and the task is to train a model to predict if a user is going to watch a movie and serve this model online, we will learn to import our features into features. Store Search Online Prediction Request Using the imported features and then access important features in offline jobs such as training jobs to set up. We install some additional packages set up our project and authenticate a Google Cloud account. Step one is to create dataset for output. We are creating BigQuery dataset to host the output data input, the name of the dataset and the table. We want to store the output later. Then we are defining constants and features store related imports. Here's how the Vertex Feature Store actually works. It organizes the data with the three hierarchical Concepts Features Store, which is the place to store your features entity type under feature store describes an object to be modeled real or virtual and feature itself under entity type describes an attribute of that entity type. Now in our movies prediction example, we will create a feature store called Movie Prediction. This store has two entity types users and movies. The user's entity type has age, gender and like genre features. The movie two entity type has the genre and average rating features. The first thing we do is to create the feature store. The method to create a feature store returns a long running operation that starts and asynchronous job. This may take about 3 minutes or so, and once the feature store is created, we can see it in the console and we can create our entities in this store. I'm creating two entities here, user and movies. We can also create features within these entities. Here. I've created age, gender and like genres under user and title genres and average rating under our movies. If we want, we can search through and filter on these features. Now we need to import feature values before we can use them for online, offline serving. Let's head back into our notebook to see how to import features in bulk using the python SDK, we defined the data source, the Big Quote table or cloud storage bucket and the destination feature store entity and the features to be imported. You do this for both users and the movies entity. Now for a latency sensitive service such as online model prediction, we would need to serve our feature values online. For example, for a movie service, we might want to quickly show movies that the current user would most likely watch by using online predictions. You can read one entity per request or even read multiple entities per request. Now, if you need feature values for high throughput, typically for training a model or batch prediction, then serving feature values in batch is a better idea than serving online. Consider this example if the task is to prepare a training dataset to train a model which predicts if a given user will watch a given movie, then to achieve this, we need two sets of inputs features that we have already imported and labels, which is the ground truth data recorded that User X has watched movie y. It also includes the timestamp, which indicates when the ground truth was actually observed as labels and feature values are collected over time, those feature values change. The feature store can perform a point in time lock up so that you can fetch the feature values at a particular time. It's literally the data version of going back to a previous version of your source code in GitHub. Imagine freezing the state of the feature values at two different timestamps and that was a quick summary of what takes feature store. Just note that at the time of this recording, the feature store is in preview. So chances are there are more options and features available when you're watching this video. To explore more up to date information, check out the documentation that I've linked below. Any questions? Let me know in the comments or on Twitter, @pvergadia.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.