Analyzing Customer Support Tickets
1. Analyzing Customer Support Tickets
Welcome back. In this video, you will explore the support tickets data set and use Cortex LLM functions to analyze them. Sign in to your Snowflake account if you're not already signed in. Navigate to the projects tab on the left panel and select notebooks and click on the fine-tuning Mistral 7B notebook. Select the start button at the top right to initiate the notebook session. Right after we start the notebook session, click on packages button and type Snowflake to install the Snowflake core package. This allows us to use Snowflake features such as Snowpark and Cortex. Let's run the first cell in the notebook to import Snowpark context and Snowpark functions for exploratory analysis. Next, let's select the database and schema where the data set is stored for the context. In the next cell, we're loading the support tickets from the support tickets table into a Snowpark data frame. Looking at the data, we can see that the requests often cover multiple product lines with text and formats, making it difficult to quickly identify the issue. Interesting. But worry not, as I mentioned before, analyzing and making sense of long text data is exactly what LLMs are a perfect fit for. Before we use LLMs to review these tickets, we have a quick note from our business teams. They want us to categorize each support ticket into one of five categories, meaning the main issue from each support ticket could be about roaming fees or slow data speed or lost phone or adding a new line or just closing the account. In a real-world setting, the customer complaints might not be limited to these five categories only, but for the purposes of this project, let us stick to these categories. So in the next cell, we have defined a variable called ticket categories with a list of five categories we received from our business teams. As I mentioned earlier, we have two main tasks in front of us. One is to analyze the support tickets and categorize customer issues into one of five buckets. Second is to automatically generate a custom response for every support ticket to resolve the issue. For the first task, we will use classified text LLM function we learned about in the earlier part of the course. For the second task, we will prompt a large language model to generate a custom response for every support ticket. In the next cell, we've defined our prompt to guide the LLM on how to generate this custom response. If the customer has set their contact preference to text message, the LLM should generate a text response under 25 words. If the customer has their contact preference set to email, the LLM should generate an email response under less than 100 words. Look at the next course snippet. We have used classified text LLM function that takes the support ticket request as input and generates the ticket category as output. With a prompt defined, we will call the Mistral 7b model using the Cortex complete function that we already learned. The complete function takes the model name and prompt, the support ticket request, ticket category, which is the output of the classified text function, and contact preference of the customer are concatenated into the prompt as input. The model then generates the text or email responses output. How cool! This is a perfect example to show how LLM functions can be used as building blocks to develop sophisticated AI apps and use cases. Let's run this cell and look at the email and text response the Mistral 7b generated. Oh, it really is that simple. Hmm, except look at this example. The customer's contact preference is a text message, but the model generated the text response and email response in the same output. Not good Mistral 7b. Well, what if we use the larger, more powerful model? Would it be able to generate accurate responses for our support tickets? Let's try the Mistral Large now. Using the same prompt and parameters with the Cortex complete function, we will simply switch out the model name to Mistral Large. It is that simple to play around with different foundation models using the Cortex complete function. Let's investigate the response from Mistral Large. Clearly, it did a better job than Mistral 7b. Fewer errors than Mistral 7b, but we still have some incorrect responses. We could certainly engineer the prompt in different ways and experiment to see if the model is able to generate the right responses. This prompt engineering approach can be effective in some cases, but in many cases it can feel like a whack-a-mole, where the engineered prompt can work sometimes but be flaky the other times, not to mention the time it requires to manually craft your prompt. For specific tasks like this, where we want to generate responses in a certain style to every support ticket, fine-tuning the foundation models would be an effective approach. So coming up, we will learn how to fine-tune a smaller model, such as the Mistral 7b, and teach it to generate custom responses to support tickets in a desired style based on customers' contact preferences. See you in the next video.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.