1. Selecting a foundation model
Hello! Welcome to this video on interacting with foundation models in Amazon Bedrock.
2. Enabling model access in Amazon Bedrock
To set up model access, we navigate to Bedrock in the AWS console and go to access settings.
For this course, we'll use Amazon's Nova Micro and Anthropic's Claude Sonnet. We can enable these, and save our changes in the console. For this course, we won't need to enable any models as access is already granted.
3. Lightweight vs. advanced foundation models
Bedrock offers different types of foundation models. Lightweight models are optimized for basic operations like Q&A and summarization, offering quick responses at lower costs. Examples include Nova Micro and Claude Haiku - perfect for straightforward tasks.
Advanced models like Nova Plus and Claude Sonnet are designed for complex operations requiring deeper analysis and reasoning. While they have higher costs, they excel at sophisticated tasks needing comprehensive understanding. Choosing the right model depends on our project-specific needs.
4. Other foundation models in Amazon Bedrock
Beyond Nova and Claude, Bedrock includes Meta's Llama for general text tasks, Stability AI's Stable Diffusion for image generation, and AI21's Jurassic for text generation and analysis. Though we won't use these in this course, they're available for projects requiring different capabilities.
5. Invoking Bedrock models
Now, let's see how we interact with these models. First, we initialize the Bedrock client with 'bedrock-runtime'. The .invoke_model() method is how we communicate with Bedrock models. To call it, we specify which model we want with modelId, and provide a dictionary of model inputs to the body parameter. Since the Bedrock API expects JSON inputs, we import the json library and use json.dumps() to convert our input dictionary into a JSON string. The model then sends back its response in JSON format as requested.
6. Invoking Claude
Let's see how to use different models in Bedrock. With Claude, we have a couple of extra keys to include in the dictionary. We include the anthropic version, which we can find along with model version we're using, and a max_tokens parameter to specify the maximum number of tokens, which correspond to chunks of words or characters. Finally, we add the message as a dictionary in a list.
7. Invoking Nova
Nova also uses the 'messages' list to hold user interactions with the model, without other required parameters. Each message is a dictionary with a 'role' key, indicating whether the prompt is from the user or the AI. In this case, it is from the user, so this is specified in the dictionary. It also has a 'content' field to hold the actual message as a list, containing the prompt as a dictionary where the key is 'text', and the value is the prompt itself.
8. Extracting model response
After invoking a Claude model, we process the response in three steps.
We read and decode the API response body using response['body'].read(), and convert the JSON response to a Python dictionary with json.loads(). Then, we extract the generated text from the 'text' key in the first element of 'content'.
9. Extracting model response
For Nova models, the response processing follows these steps:
Reading and decode the API response using response.get('body').read().decode(), and converting the JSON to a Python dictionary with json.loads(); and finally, extracting the content by navigating a nested path, going from output to message, to the first element of content, and then text.
Nova's structure is more nested, requiring deeper navigation to reach the AI-generated content.
10. Let's practice!
Let's practice invoking models in some exercises!