Monitoring Your Agent in Snowflake
1. Monitoring Your Agent in Snowflake
Let's look inside your agent. We'll explore the monitoring interface and learn how to read traces. By the end of this video, you'll know how to diagnose any agent interaction. I'm in Snowflake. Let's navigate to AI and ML, then Agents. Click on your Sales Intelligence agent. You should see the agent overview. On the top, there's a navigation pane. Click on Monitoring. This opens the monitoring view for your agent. The monitoring pane shows all interactions with your agent. Each row is a conversation. You see the timestamp, the user who initiated it, and a preview of the conversation. Recent interactions appear at the top. Let's generate some data to look at. Let's open our agent in Snowflake Intelligence. At the top, click the button that says Preview in Snowflake Intelligence. Let's ask a few questions so we have traces to examine. Let's type, what is our win rate by product line? Let's wait for the response. Now type, what concerns did TechCorp raise? Then we wait for the answer. Now finally type, which deals should we focus on next quarter and why? This last question requires both tools. Let's go back to our agent. Let's navigate to AI and ML, then Agents. Click on your Sales Intelligence agent and go back to the Monitoring tab. Now let's refresh the page. You should see the new entry. Let's click on it. You should see the new entry. Click on the most recent conversation. The Conversation Detail view opens. On the left, you should see the full conversation history, each message from the user, each response from the agent. This is useful for context, but the real power is in the traces. Right next to that, you should see the execution trace. This is a detailed breakdown of how the agent processed each question. Let's examine the first question about win rates. Click on the interaction. The trace expands. You should see multiple spans. The first one is LLM Planning. Click on it. The Planning span shows how the agent interpreted the question. You can see the input, which is the user's question, plus your orchestration instructions. You can see the output, which is the agent's plan. In this case, it decided to use Cortex Analyst because win rate is a quantitative metric. This is where you verify tool selection. If the agent chose the wrong tool, you would see here. For this question, Cortex Analyst is correct. The agent understood this is a quantitative question. Let's click and look at the next span. This is tool execution for Cortex Analyst. Expand it. You should see the actual query sent to Cortex Analyst. You see the SQL that was generated, and you see the results returned. Look at the SQL. It's selecting product line and calculating win rate using the formula from your semantic view. It's grouping by product line. This matches what we expect for this question. Let's look at the result. For premium security, our win rate was 100%. Analytics Pro, again, 100%. And Enterprise Suite, again, 100%. Our basic package was only at 50%. These numbers flow into the next step. Click the final span. This is the LLM response generation. The agent takes the tool's results and formats a response. You see the inputs include the Cortex Analyst results. The output is the formatted answer the user receives. That is a complete trace for a single tool question. Planning, tool execution, response generation. Now let's look at the last question, the complex one about the deals to focus on. Click on the interaction in the conversation. This trace is longer. Multiple tool executions are happening. The agents plan to use both Cortex Analyst and Cortex Search. Let's walk through it. The first span is planning. As you can see, it starts off by thinking it should use a sales metrics view. It uses Cortex Analyst to do that. We can see what it's thinking through, how it's getting the idea. We can see which tool it selected. Next, we'll go to Cortex Analyst and see the SQL generated as well as what it executed. As you can see here, there's the SQL query. It found the deals that are coming up and what's in the pipeline itself. Then we can see the SQL execution and see what kind of deals it found. Then it went back to thinking to understand and see if it needs to find more information. Then it decided to use Cortex Analyst again to see if it can do a different query to find more information for us. Let's go ahead and look at that Cortex Analyst span. You can see in the SQL query that it decided to gather more data. Here, we can see the results from the SQL query. It found a couple more deals that are coming up. Then again, it went through a planning phase to see if there's anything else that it can gather in terms of information. As you can see here, it decided to use other tools. It wanted to look at our qualitative data as well. It's going to use the Cortex search service that we set up. We can take a look at that and see the results. I found some different conversations that have happened that may apply to our pipeline for deals. The final span is response generation. The agent synthesized both data sources. It ranked the deals by likelihood based on the structured metrics and added context from the conversation insights. This is how you verify complex interactions. Each tool call is logged. You can confirm the agents use the right tools in the right order with the right inputs. Let me show you how to spot a problem. I'll ask a question that might cause issues. Let's go back to stuff like intelligence. We're going to type, what was the sentiment in our lost deals? Now let's go back to the monitoring tab. We'll go ahead and refresh. Let's click the very top trace. Look at the planning span. The agent decided to use Cortex search. But wait, to find lost deals, it should first query Cortex analyst to identify which deals were lost. Then search those conversations. Look at the search results. It searched for lost deals, negative sentiment, concerns, objections, reason why lost, but didn't specifically target the conversations from deals marked as lost in the metrics table. The response might include sentiment from deals that weren't actually lost. This is exactly the kind of issue monitoring reveals. The agent answered the question, but it used a self-optimal approach. Without traces, you might not notice. With traces, the gap between expected and actual behavior is clear. You can also filter and search through your monitoring data. Use the filters at the top to narrow down by date range or which user used it. This helps when you're looking for specific patterns across many interactions. In the next video, we'll use what we've learned here to improve our agents. We'll take a real issue found through monitoring, update our instructions, and verify the fix.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.