Prompt engineering
1. Prompt engineering
SPEAKER: This lesson covers the first half of the prompt-to-production lifecycle, prompt engineering from design to evaluation and refinement. A good prompt, as we learned previously, considers both content-- instructions, context, and examples-- and structure-- order, labels, delimiters. So how do you engineer a good prompt? It begins with prompt design, supported by a rich toolkit provided by Vertex AI Studio. This is your primary playground for crafting prompts. On the left, specify the context in system instructions, then pose your tasks or questions in the Prompt section. Need help? Gemini, the built-in AI assistant, can help you create your prompt. Powered by multimodal foundation models like Gemini, you can incorporate multimedia data such as documents, images, and videos from diverse sources, including Google Cloud Storage, Google Drive, your local computer, or a URL. You can even embed YouTube video links into a prompt. Guide the AI's output by adding examples using the default input and output features. Or customize them to question and answer. Enterprise users can also import example files of their company's data. Ann is a developer looking for a way to code prompts, perhaps using a function or method with variables to streamline repetitive actions. Vertex AI Studio's new prompt template feature is the perfect solution. It uses replaceable variables, allowing you to reuse a prompt by simply changing values. Imagine a function in coding but using natural language. The beauty is, you only need to tell GenAI what to do without worrying about how to do it with specific programming languages. Consider this example. By clicking Add Variables, you can assign values, just like passing arguments to a function. For instance, you could ask AI to research Los Angeles tenant vacancy rate and generate a report on real-estate market analysis. You can also instruct AI to add variables to study annual crime rate and conduct an insurance risk assessment by using the same prompt template with different values. When your draft is complete, navigate to the right side of the user interface to experiment with various model parameters. Begin with model selection. Vertex AI Studio offers a wide selection of Google and third-party models, including Anthropic Claude, Meta Llama, and OpenAI GPT. A key advantage of Vertex AI Studio, though, is its access to Google's cutting-edge GenAI models, like Gemini. Choosing the right Google model depends on your task. In the previous lesson, you were introduced to Google's different foundation models. To refresh your memory, Gemini family-- example, Gemini Flash and Gemini Pro-- ideal for general purposes and multimodal data use cases, specialty models designed for specific tasks. For instance, here are a few options when you are in media studio, where you create multimedia with Vertex AI Studio-- Imagen for image creation, Chirp for voice generation, Veo for video processing, Lyria for music composition. After model selection, the next step is parameter specification, like temperature, Top P, and Top K. You might find some of these options in the advanced settings. These parameters control the randomness of the model's responses by adjusting how output tokens are selected. But how do they actually work? Let's look at an example. The garden was full of beautiful dot, dot, dot. When prompted with this incomplete sentence, language models predict the probability of potential words, like flowers, trees, herbs, and bugs. The selection strategy impacts the outcome. Always choosing the most probable word can lead to repetitive and potentially biased text, while random sampling might yield unlikely responses, such as bugs. Adjusting model parameters to control randomness allows you to balance predictability and variety, finding the ideal strategy for a specific task. Let's explore these parameters in depth. First, temperature-- this number controls the degree of randomness in generated output. A low temperature setting narrows the range of possible output to high-probability, more typical words. This is ideal for tasks like question answering and summarization, where a more typical answer with less variability is expected. A high-temperature setting expands the range to include lower-probability, more unusual words, useful for generating creative or unexpected content. Another parameter is Top K. Top K allows the model to randomly select a word from the Top K most probable words, where K equals a number. For example, top two means the model will randomly select either of the two most probable words, such as flowers or trees. This approach gives high-scoring words an equal chance. However, if the probability distribution is highly skewed-- example, flowers at 80% and books at 10%-- it can result in strange responses, like the garden was full of beautiful books. The challenge of selecting the optimal top K value led to Top P, where P stands for probability. Top P allows the model to return a word from the smallest subset with a sum of likelihoods that exceeds or equals P. For example, a P of 75% means sampling from a set of words with a cumulative probability greater than 75%-- in this case, flowers, trees, and herbs. This dynamically adjusts the size of the word set based on the probability distribution of the next word. And that is an overview of the model parameters-- model type, temperature, Top K, and Top P. Note that you are not required to adjust them constantly, especially Top K and Top P. After crafting the prompt and specifying parameters, how can you ensure you've selected the optimal model and parameters for the task? This is where evaluation and refinement come in. Vertex AI Studio allows you to compare prompts side by side to see which produces the best results. This helps you understand how different prompts, models, and/or parameter settings influence the output. You can even generate your own evaluation metrics by adding ground truth from your field knowledge, your preferred answer to the prompt against which all other model responses are evaluated. Ready to take your prompt to the next level? Optimize it in a Colab Enterprise notebook by adding labeled examples to refine the results. You can perform these tasks' comparison, optimization, and evaluation under the Prompt management menu. Imagine Prompt management as storage to save and share prompts for future use and collaboration, complete with tools like version control and security. Beyond general purpose prompts, you can apply these prompt engineering techniques and tools on Vertex AI Studio to specific tasks, such as generating real-time streaming, creating multimedia content, translating content, and converting speech and text. Ann has learned so much about what she can do with prompts and is eager to leverage these tools to create custom prompts using her own data to solve business problems. She's excited to learn how to deploy the prompt to code, which will be revealed in the next lesson.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.