Using Helper Functions
1. Using Helper Functions
Welcome back! In this video, we are going to learn how to use the third type of LLM functions in Snowflake Cortex, the helper functions, and get some practical experience using them. Helper functions are purpose-built managed functions that can help us estimate inference costs, also reduce cases of failures when running other LLM functions. As noted before, there are two helper functions, count tokens and try complete. As the name implies, count tokens returns the number of tokens in a given text string or prompt. In addition to the text string or prompt, it takes the name of the model in the case of the complete or the name of the function in the case of task-specific LLM functions. This is required because each model or function may use different tokenizers and therefore results in a different number of tokens for the same text string. Count tokens can be very useful when we are not sure if the context passed to the LLM will exceed its context window limit. And for big projects, it can be helpful for estimating token costs by counting the number of tokens for small but representative samples of the queries, then extrapolating to the number of tokens that will be used for the entire project. It's time to see it in action. This is the time to pause the video and log into your Snowflake account if you haven't already. Navigate to projects on the left panel and select workspaces. Click on notebook titled using LLM functions and select the start button at the top to start the notebook session. We had already run the notebook until the helper function section. Look at the code below helper functions. First, we call the count tokens functions. For this, we need to pass the name of the model to the function as well as pass the input text we want to assess. In this example, we pass the Llama 3.1 model. As we learned before, different foundation models use different tokenizers and will output different numbers for the same input text. This model is cutting edge today, but always check Snowflake's documentation for the latest model for your use case. Let's move on to the second helper function, try complete. How can the try complete function help? This helps us test if complete will work or not before committing to the cost of running it. This function works in the same manner as complete, but returns null instead of an error when the operation runs into difficulty. The best part of this is that the operation returns null, then you're not billed for the token use. Test away. Here we're passing Llama 3.1 as the model to complete function. It errors out since we mistyped the model name. Now, let's try the same on try complete instead. It fails gracefully and returns none. This might look unintuitive at first because when the user inputs the wrong values, the function should error out, right? Imagine you built an AI application that has multiple functions. It is being used by your business teams across the org. Because of the invalid input from the user, the complete function errors out and breaks the entire application. It is a bummer your business users cannot use other parts of the application that still works. Try complete is particularly useful when there are concerns that the input may exceed the context window. We need a way to gracefully fail to avoid these scenarios. This is a common practice in software engineering. Isn't it great that we're able to follow these best practices in AI app development as well? Thanks, try complete. All right, that's the helper functions. When you're ready to use them in your applications, be sure to check the docs for boundary conditions and other details we didn't have time to cover here. In this video, we learned about helper functions and how to use them. We looked at count tokens and how it can be useful to get the total number of tokens that is specific to the model when processing a given input text. We looked at how to use try complete, which allows us to run the operation and not pay for the execution if the execution fails. Coming up, we will start pulling all of these pieces together and create a Streamlit app that will allow us to switch between these functions. See you soon.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.