Deployment and model tuning
1. Deployment and model tuning
SPEAKER: In the previous lesson, Ann experimented with the Vertex AI Studio toolkit to design, evaluate, and manage prompts. Now she's ready to deploy the prompt to an application. She teams up with Ian, the ML engineer who is responsible for building a pipeline and monitoring performance. This brings us to the second half of the prompt-to-production lifecycle, integration and deployment, covering stages from build and test to monitor and optimize. Recall that, early in their journey, Bea and Ann quickly tested their ideas by simply clicking Build with Code and then Deploy as App to build their own application. However, what if they wanted to customize the application or integrate its features into other applications? Vertex AI Studio provides this flexibility by automatically generating the code for you. Besides the User Interface, UI, which requires no code to explore and test prompts, Vertex AI Studio offers other low-code approaches to access AI models. These are Software Development Kits, or SDKs, in Python, and APIs with cURL. Simply click Build with Code to find the code describing the prompt and its parameters. Automated code generation simplifies application development. Additionally, the integrated development environment with Cloud Run and Cloud Shell streamlines production and removes the need to worry about the underlying cloud architecture that supports application deployment. After building your application, continuous monitoring and optimization are key to maintaining high performance. But how do you ensure your Gen AI models deliver accurate, up-to-date results? One way is through grounding and Retrieval Augmented Generation, or RAG. Gen AI models are often pretrained, meaning their responses rely on potentially outdated or inaccurate training data. Grounding connects these models to trusted external data sources, ensuring their answers are verified against the latest information. RAG is a method for implementing this idea. Think of grounding as the what and RAG as the how. When constructing your prompt with Vertex AI Studio, you can choose to ground the results either through Google real-time search for the most current information or your own data to instruct the AI with field-specific knowledge. To further your knowledge of these advanced technologies, we recommend our courses Create Embeddings-- vector search and RAG with BigQuery and Vector Search and Embeddings with Vertex AI. These courses introduce how to implement RAG pipelines with Google's two widely used platforms, BigQuery and Vertex AI, respectively. You can scan the QR code on the screen or find the link in your reading list. Other than fact-checking using grounding and RAG, what if you want to improve the quality of content generation itself? That's where model tuning comes in. It's another way to enhance Gen AI accuracy, providing the model with a training data set of specific downstream task examples. While fine tuning refines the model's internal knowledge and abilities, grounding augments its knowledge with external, real-time and reliable information. Remember the earlier analogy about foundation models versus fine-tuned models? If K-12 education represents the foundation model, then fine-tuning is specialized professional training like medical school that embeds domain-specific expertise. Grounding, then, is the ongoing practice of checking the latest research, drug treatments, and medical policies to stay current. Now let's look at how to tune and customize a Gen AI model with Vertex AI Studio. You have different options, ranging from less technical methods like prompt design, which require fewer computational resources, to more technical methods that require more computational resources, like full fine-tuning. You are already familiar with prompt design, which lets you tune a generative AI model with examples and instructions in natural language. Remember that prompt design does not alter the parameters of the AI model. Instead, it improves the model's ability to respond appropriately by guiding it on how to react. One benefit of prompt design is that it enables rapid experimentation and customization of generative AI results. Another benefit is that it doesn't require specialized machine learning knowledge or coding skills, making it accessible to a wider range of users. However, for more complex tasks that require tailored results, consider customizing an AI model with either parameter-efficient tuning or full fine-tuning. Parameter-efficient tuning, also called adapter tuning, enables efficient adaptation of large models to your specific task or domain. This method also updates a relatively small subset of the model's parameters during the tuning process. Full fine-tuning is ideal for highly complex tasks, as it can achieve higher quality results. However, this method requires more computational resources for both tuning and serving, as it updates all the model's parameters. Given these techniques and even some variations between them, Vertex AI currently supports supervised fine-tuning to customize foundational models. Supervised fine-tuning improves model performance by teaching it a new skill. It uses data containing hundreds of labeled examples to teach the model to mimic a desired behavior or task. Each labeled example demonstrates the desired model output. The output of the tuning job is a new model that combines newly learned parameters with the original model. Supervised fine-tuning is a good option for well-defined tasks with available labeled data. For example, it can improve model performance for classification, summarization, extraction, and chat tasks. Supervised fine tuning trains a model with labeled data. It can be implemented with different techniques like parameter-efficient tuning or full fine-tuning, depending on task complexity and available computational resources. Now let's move to Vertex AI Studio and see how to start a tuning job. From the Vertex AI Studio menu, select Tuning, then Create a Tuned Model. Specify the model details and the tuning data set. Note that the UI may change as the product progresses. The tuning data set should be structured as supervised training data in a JSONL file. Each record or row contains a pair of text data, the input text, which is the prompt, and the output text, which is the expected response from the model. For example, if the prompts are "This commercial building is architecturally interesting and has a great interior layout." and "The room was terrible. It needs major rework." the expected sentiment labels would be positive and negative, respectively. This structure allows the model to learn and adapt to your desired behavior. You can then start the tuning job and monitor the status in the Google Cloud console. When the tuning job completes, you'll see the tuned model in the Vertex AI Model registry. And you can deploy it to an endpoint for serving or further test it in Vertex AI Studio. That was a lot of information. Ann and Ian can't wait to explore and start experimenting with Vertex AI Studio themselves. It's time for you to have some hands-on practice. This lab will give you the opportunity to create an application directly from a prompt, design effective prompts by applying best practices, engineer and manage prompts by using features like prompt evaluation, and use multimodal prompts and generate media. By the end of the lab, you'll be well equipped to use Gemini Multimodal with Vertex AI Studio.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.