Using Helper Functions
1. Using Helper Functions
Welcome back! In this video, we are going to learn how to use the third type of LLM functions in Snowflake Cortex, the helper functions, and get some practical experience using them. Helper functions are purpose-built and managed functions that can help us estimate inference costs and also reduce cases of failures when running other LLM functions. As we noted before, there are two helper functions, COUNTTOKENS and TRICOMPLETE. As the name implies, COUNTTOKENS returns the number of tokens in a given text string or prompt. In addition to the text string or prompt, it takes the name of the model in the case of COMPLETE or the name of the function in the case of task-specific LLM functions. This is required because each model or function may use a different tokenizer and therefore result in a different number of tokens for the same text string. COUNTTOKENS can be very useful when we are not sure if the context passed to the LLM will exceed its context window limit. And for big projects, it can be helpful for estimating token costs by counting the number of tokens for a small but representative sample of queries and then extrapolating to the number of tokens that will be used in the entire project. It's time to see it in action. This is the time to pause the video and log into your Snowflake account if you haven't already. Navigate to Projects on the left panel and select Notebooks. Click on the notebook titled Using LLM Functions and select the Start button at the top to start the notebook session. It takes a couple of seconds. We had already run the notebook until the helper function section. Look at the code below helper functions. First, we call the COUNTTOKENS function. For this, we need to pass the name of the model to the function as well as pass the input text we want to assess. In this example, we pass the LLM 3.1 70 billion model. As we learned before, different foundation models use different tokenizers and will output different numbers for the same input text. Let's move on to the second helper function, TRICOMPLETE. How can the TRICOMPLETE function help? This helps us to test if COMPLETE will work or not before committing to the cost of running it. This function works in the same manner as COMPLETE, but returns NULL instead of an error when the operation runs into difficulty. The best part of this is that if the operation returns NULL, then you're not billed for the token use. Test away! Here, we're passing LLM 2 70 billion chatty as the model to COMPLETE function. It errors out since we mistyped the model name. Now, let's try the same on TRICOMPLETE instead. It fails gracefully and returns NULL. This might look unintuitive at first, because when the user inputs the wrong values, the function should error out, right? Imagine you built an AI application that has multiple functions. It is being used by your business teams across the org. Because of an invalid input from the user, the COMPLETE function errors out and breaks the entire application. It is a bummer your business user cannot use other part of the application that still work. TRICOMPLETE is particularly useful when there are concerns the input may exceed the context window. We need a way to gracefully fail to avoid these scenarios. This is a common practice in software engineering. Isn't it great that we are able to follow these best practices in AI app development as well? Thanks to TRICOMPLETE! Alright, that's the helper functions. When you're ready to use them in your applications, be sure to check the docs for boundary conditions and other details we didn't have time to cover here. In this video, we learned about helper functions and how to use them. We looked at COUNT tokens and how it can be used to get the total number of tokens that a specific model will use when processing a given input text. We looked at how to use TRICOMPLETE, which allows us to run the operation and not pay for the execution if the execution fails. Coming up, we will start pulling all of these pieces together and create a streamlined app that allows us to switch between these functions. See you soon!2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.