Starting the fine-tuning job
1. Starting the fine-tuning job
Welcome back. Let's talk about what we're going to do in this video. In this video, we will be starting our first fine-tuning job. We will talk about cost considerations, access permissions, and controls. We will look at examples of how to run your fine-tune job from your Snowflake notebook. In the last video, we finished preparing our data, and we are now ready to start our fine-tune job. But before we get started with fine-tuning our LLM, we will take a moment to talk about the costs that the fine-tune jobs incur and what factors affect this. The Cortex fine-tuning function incurs costs based upon the number of tokens that are consumed during the training. On average, each token is roughly equivalent to four characters of text, which we mentioned before, so you could calculate a rough cost that way. However, that is not the extent of what the training is going to cost you. Let's go through this step-by-step. The way to calculate fine-tuning training tokens is to take the number of input tokens and multiply it by the number of epochs that you are planning to run. The number of times that the model is trained on the training data and evaluated on evaluation data during the fine-tuning job is called an epoch. Think of epochs as the series of actions that your model executes on the LLM during the fine-tuning job. Just like when a human is learning something, the model will rarely gain complete understanding of the new information after one pass. So much like a human learner that is exposed to an information set over and over again and who develops a greater and greater mastery of the new data, our fine-tuning job benefits from running the training more than once. Also, just like a student that reads a textbook over and over again, there are initially big improvements over the first couple of epochs, but diminishing returns over time. By default, Cortex will select an optimal number of epochs between 1 and 10. Or, you can choose the number of epochs which controls the number of epochs the model trains for. Deciding on the number of epochs to assign to your fine-tuning job depends on several variables. In cases where the training data is small or accuracy is paramount, you may also want to add additional epochs to your fine-tuning job. But in cases where the training data is larger, the number of epochs can be reduced. Ultimately, the goal is to tune the model so that it accurately returns based upon the data it was trained on, while not overspending time and money. It is worth noting that your costs do not end with calculating input tokens multiplied by epochs. You will have costs incurred for the storage and warehouse costs for storing any of your customized adapters and any SQL queries you run. Fine-tuning jobs are often long-running and are not attached to a Snowflake notebook session. This means they run in the background. This is also part of your compute costs. Next up, let us look at the permissions you need to run the fine-tuned job in Snowflake. Any user who wants to invoke the fine-tune function needs to be assigned Snowflake Cortex user role. Ask your account admin to grant this role to you if you want to run a fine-tuned job. For the purpose of this course, you will be using a Snowflake free trial account, so you already have the account admin privileges. So don't worry about these access controls and permissions. However, in a real-world setting, most of you would not be the account admins of the Snowflake accounts in your organization. So reach out to your organization's account admin to grant you Snowflake Cortex user role. And next, you will need the usage privilege on the database where the training and evaluation data set is stored. This is so you have access to the training data to run the fine-tuned job. In addition to that, in order to create models within a specific schema in a database, you must have either the create model privilege along with usage access or full ownership privilege on that schema, allowing you to create and manage all objects within it, including models. To sum it up, you need to review three access permissions. First, do you have permission to run the Cortex fine-tune function? Second, do you have usage access on the database that stores fine-tuning data? And lastly, do you have permission to store the fine-tuned model in the desired schema? All right. With all that information, let's get our fine-tuned model up and running. If you're not logged into your Snowflake account, please pause the video now to log into your account. Navigate to the projects tab in the left panel and select notebooks and click on fine-tuning Mistral 7B notebook and select the start button at the top to initiate the notebook session. We had run the notebook until preparing the fine-tuning data set already. The next cell has everything we need to run fine-tuning job in our notebook. Look at the code snippet in the fine-tuned Mistral 7B model using Cortex cell. When we invoke the fine-tune function, we need to pass a few parameters. First is the create string. This is to indicate that we want to create a new fine-tuning job. Sometimes we use describe in place of create to check the status of the fine-tuning job. In this case, we use create. The next parameter is the name of the fine-tuned model. We are calling it support messages fine-tuned Mistral 7B. The third parameter is the foundation model we are fine-tuning, Mistral 7B in our case. And then we pass on the training and evaluation data set as well. Once we run the cell, we can see the job ID for the fine-tuned job as the cell output. That is it. In a single line of code, we are now fine-tuning the Mistral 7B foundation model with our own proprietary customer support ticket data to generate responses in a particular style. In just a single line of code. How cool is that? The fine-tuning job can take a couple of minutes. As I mentioned before, this job runs in the background. So even if the notebook session stopped, the fine-tuned job continues to run in the background. How do we know when it is done? How do we know if it runs successfully? Let's take a look at what's happening under the hood when the model is running. Fine-tuned jobs can be long running and depending on the input table you use and the model you choose, this job could be running for quite some time. We can check the status of our jobs by calling the Snowflake Cortex fine-tune with show as the first argument to see how the job is progressing. Here, if we pass the show argument to Cortex fine-tune, we should see a list of all the fine-tuned jobs that we have running, which we have access to. Depending on the job and where it is in its run, we may see pending, in progress, success, error, or even canceled status. Another argument that we can pass to the fine-tuned function is describe. In the next cell, we invoke the fine-tuned function, but this time with the describe string and pass the job ID from the previous cell output as the second parameter. That is, we want the fine-tuned function to describe the job we kicked off in the previous cell. We get back a lot of information about the fine-tuned job when we run this. First, we are given the base model our job used, the Unix date stamp it was created on, and if the job has finished, the finished on time stamp as well. Below this, we get the job ID, followed by the database, schema, and model name. Next, we get the progress and status, followed by the training data information. Under this, we get the number of tokens that the fine-tuned job consumed and a training result that tells us the training and validation loss. Lastly is the validation data information. Describe gives us much more information than show, but only for one job per call. Well done. We have now run our first fine-tuning job in Snowflake. We have more than one option to run our fine-tuned job. We can either run it from our notebooks like we just did, or we can use the Snowflake Studio interface to set up and run our job. Let's look at running our fine-tuned job using the AIML Studio interface. The AIML Studio is a no-code interface that streamlines the LLM customization and AI app development in Snowflake. Since we have set up the training and evaluation datasets, we can now jump over to the AI and ML Studio. Open a new tab in your browser by clicking on the snowflake icon in the upper left side of your browser. From here, select AI and ML, and then select Studio. On the right pane, you will see Create Custom LLM, and on the right side of that, you will see a button that reads Fine-Tune. Select that. It opens up the fine-tune wizard. In the first page, you will be prompted to select the base model from the list of supported models. For this, select Mistral 7b. Note that when you select different models that are available to you, a brief description of model's capabilities pops up on the right side. Here, we see that Mistral 7b is a model that is fast to deploy and customize, and that it is a small model, and that it is good for many use cases. We're also told the size of the context window that it has. Select the role. It will be Account Admin by default if you're using Snowflake free trial account, and select Compute Warehouse as a warehouse as well. For the database, select Telco Support DB and Support Data for the schema as well, because that is where our fine-tuning data is stored. And then enter a name for the fine-tuned model. Use a different model name than the one we created earlier through Snowflake Notebook. I'm going to call mine as support underscore messages underscore fine-tuned underscore Mistral underscore 7b underscore v2 underscore studio. I'm going to leave the number of epochs as auto, but you can experiment with that as well. When you're done, select the blue Let's Go button at the bottom. The next page prompts us to select our training data. Look for Support Tickets fine-tuned message style train table, and then select Next. The page will stay the same, but now we're asked to identify the prompt column for the fine-tuned job to use. Thankfully, we labeled this prompt, so identifying it is pretty easy. Next, we are asked to identify the completion column that we want our job to use as reference when it is producing its own completions. Select Mistral Large Response. Lastly, we are asked to identify the validation data set we want to use. It is important that the format of our validation data matches our training data. Select the validation data set as Support Tickets fine-tuned message style eval table, and select Next. We are given a chance to review our choices before starting the job. If for any reason something looks wrong, we can select Back to go back and change it. And that's it. We can start fine-tuning the model by clicking on Start Training. This will return a job ID, the base model we used, the name we chose for the fine-tuned model, where the training data is, and the location that the fine-tuned model will go to when complete. You can now click on Done. Like I said before, this fine-tuning job takes a few minutes and it runs in the background, so we just wait. We accomplished a lot in this video. We explored the cost considerations that we need to think about when we are setting up our fine-tuned jobs, and how epochs affect the number of tokens we will spend. We learned about added costs that are incurred by the warehouse and compute during the fine-tune process. We looked at access control requirements and discussed the LLM models that are available to us to use within fine-tune. From there, we created our fine-tuned job, named an identifier, selected our base LLM model, and added training and validation data. We did this both through our notebooks and the Snowflake AI ML Studio. We started our fine-tuned job and got the ID number, which we used to monitor and manage our fine-tuned job while it is running. We looked at how we can manage and check the status of our fine-tuned jobs using show and describe functions. We saw how this returns identifying information on our running jobs, as well as the status information, such as the percent completed, and whether our jobs are pending, in progress, have succeeded, have returned an error, or have been cancelled. In the next video, we will be exploring how we can use our fine-tuned models for inference. See you then!2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.