Context Engineering & Conversation Management
1. Context Engineering & Conversation Management
Let's talk about context management. Managing context is one of the core responsibilities of an agent harness. Every tool call, tool result, correction, summary, user message, and model response consumes tokens and takes up space in the context window. And even before you hit the hard context limit, oversized context increase latency, cost, and often reduce reasoning quality. As agents become longer-running and more capable, context engineering starts becoming one of the main architectural problems that you need to solve. There are a few common strategies you'll see repeatedly. You can externalize context into files, databases, or memory systems instead of keeping everything in the prompt. You can dynamically select relevant information for the current task using tools. You can compress context through summarization or compaction. And you can isolate context between agents so each one only sees what's relevant to its role. Strands ships with a few different out-of-the-box components that make context engineering a lot easier to implement so you can save tokens. I'll show you the basics first, then at the end of the video, I'll show you the one line that gives you everything you need out of the box. This way, you'll understand what you're getting when you use built-in context management features and have the ability to select what makes sense for you. For compaction, Strands ships with built-in conversation managers to help manage this automatically. The first is the sliding window conversation manager. This one is straightforward. It keeps only the N most recent messages and removes older history as the conversation grows. Let's add it to our customer service agent. Here we create a sliding window conversation manager with a window size of 20 and pass it into the agent during creation. The conversation manager handles trimming automatically during the agent loop. And importantly, it trims intelligently. It won't split tool calls from the results. And it trims along conversation boundaries, so history remains structurally valid. There's also an option called should truncate results. When enabled, large tool outputs get compacted automatically. Long results are shortened while preserving the beginning and end of the output. And large image blocks can be replaced with lightweight placeholders. This helps control context growth from especially large tool responses. Sliding window is simple, fast, predictable, and honestly, where most people start. But it's also naive because eventually you lose older information entirely, which might actually be desired depending on your use case. But for a lot of longer-running workflows, it can become a problem to suddenly lose important context from the beginning of the conversation. That's where the second built-in option comes in, the summarizing conversation manager. Instead of dropping older messages completely, this manager compresses them into summaries. This is that example. This setup looks very similar, but now we're using the summarizing conversation manager instead of the sliding window. When the context window starts approaching its limit, the manager takes older portions of the conversation, summarizes them, replaces them with a compressed summary message, and retries the request with the reduced context. There are a few important configurations here. Summary ratio controls how much of the older conversation gets summarized during compaction. Preserve recent messages ensures the most recent messages remain untouched, so the active working context stays intact. You can also You how summarization happens. For example, for a customer support agent, you might provide a summarization prompt that tells the summarizer to focus on customer identity, issue history, actions already taken, and unresolved problems. And importantly, the summarization step itself can use a separate summarization agent or cheaper model provider entirely. Simple summarization tasks usually don't need your most expensive reasoning model. That's another example of a harness engineering design decision that shapes system cost and performance. Now, summarization is useful, but it's important to understand that summarization is lossy compression. Sometimes important details disappear, especially after repeated compression cycles in long-running agent systems. You might need more sophisticated strategies depending on your use case. Some systems extract structured state as a list of facts instead of summarizing. Others retrieve information dynamically from external memory stores like short-term and long-term memory. Or you could split workflows across multiple isolated agents with different context to work on different parts of the problem, each one having its own conversation management strategy. The conversation managers we just saw are reactive. They kick in when you hit the limit. But Strands also supports proactive compression. You can set a compression threshold, and the manager will start compressing before you overflow the context window. This is how you add it to your agent. Just include the proactive compression parameter and set your threshold, then you're all set. There are many different ways you can tackle context engineering, but there usually is no single perfect strategy. You experiment, evaluate behavior, tune compaction approaches, and evolve the harness over time based on the workloads that you actually observe. The good news is that Strands gives you extension points for this. You can build custom conversation managers when the built-in approaches stop being sufficient. Now, there's a separate problem that you might run into with context that we haven't talked about yet, but is super common. For agents that use many different tools, individual tool results can be enormous and consume a ton of tokens. A single verbose tool response or file read could be tens of thousands of tokens, and that's sent to the model on every single turn of the conversation, driving up costs and reducing performance. You're going to want a way to solve for that. The context offloader plugin solves this. It intercepts large tool results, stores the full context externally, and keeps a compact preview in context for the agent. The agent still knows what the result contained, it just doesn't bloat the window with the full thing. This is all driven using hooks and tools, so concepts you already understand. In this example, we're bringing in the context offloader plugin, which stacks on top of the other plugins we're already using. We then set the storage for the offloaded tool results to go to a local directory, set the max result tokens, and set preview tokens to be the first 2,000 tokens. This reduces the amount of tokens in the context window for tool calls, but still gives a fair amount in the preview. You can tune these numbers for your specific use case to see what works best. These two mechanisms, conversation management for history and context offloading for individual tool results, work together. Now, let's see the built-in context management configurations you can give to your agent so you can get context offloading, summarization, and proactive compression all with one line of code. You can drop this into your agent. context_manager = auto This gives you the default behavior where large tool results are offloaded to external storage and replaced with a truncated preview, old messages are automatically compressed into structured summaries rather than dropped, and proactive compression fires at 85% context usage to stay ahead of overflow. In Strand's benchmarks on real code investigation tasks, cost dropped by 55% while accuracy went from 68 to 98. You end up using about half the tokens, but you get way better results. There's also context manager equals agentic for when the model is better positioned to decide what stays in context. The model gets tools to summarize, truncate, or pin messages. It trades tokens for judgment. We recommend you start with auto by default and use agentic when your agent needs to protect specific context across long conversations. Now, we've spent some time here talking about memory systems, but everything we've done so far hasn't used memory. If the agent restarts at any point, the conversation history, the summaries, and the agent state all go away. And there is no way to bring back up conversation history for a given session across different runs. In the next video, we'll solve that by giving our agent persistent memory.2. Final slide
Next, persist conversation history across sessions with session managers.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.