Case study: Google Flights
1. Case study: Google Flights
Google Flights. Imagine the scene. You were busy at work and suddenly realized you forgot to book the flight for your upcoming trip. Purchasing flight tickets can be difficult. Identifying patterns in flight prices is challenging due to changes in flight prices, inconsistent pricing across sites, and sometimes even pricing breakdowns are hard to understand. Google Flights believes that by putting some of the data and smart AI in the hands of users, it would help them demystify what they need to pay for a certain flight at a certain time. This would potentially save users time, stress, and money. One of the challenges with predictions and a common theme across AI-driven products, is that machine learning predictions can't be 100% right all the time for two reasons. First, the predictions are specific to certain flights to certain places at specific times. And second, we don't have enough pricing data to provide an accurate prediction of whether the price is fair or not. As they thought through how to build a tool, they felt it was important to help users make informed and better decisions by explaining where this data was coming from and what it relates to. Therefore, users were allowed to; assess price goodness today and in the future, track the model's predictions and check them, make confident decisions about when to book while at the same time ensuring that users understand where our data is coming from, view the general trends in flight pricing, have reasonable expectations for the correctness of our predictions. Google Flights started designing a new tool to help users understand whether the prices for a given flight are currently high, low, or typical, and to help users learn market trends for similar trips. At one point, they considered a pivot to a more direct approach, hiding the complicated calculations in the background and giving users a simple conclusion such as, today is a good day to book. But during testing, users expressed that this felt salesy, was upsetting and didn't feel trustworthy. That was not an option because they wanted a tool that only worked if people trusted it. To start setting some guide rails for further iterations, the team created three design principles for price intelligence. Any Google flight price insights surfaced to users would have to be; honest, actionable, and concise, yet explorable. How do you explain the machine learning model output to users in a way that is actionable and compelling, but also accurate? Through the following ways. A price goodness indicator with corresponding descriptions of high, typical, or low. A single-line explanation of the usual price for a trip like the one the user is planning. Prediction text stating whether prices are likely to go up or down. An info icon that opens an explanation bubble with text explaining which data sources were used to compute the insight. At first, a tool showed the likelihood that a price would go up or down in a very specific way. For example, a medium confidence prediction could say, "Prices are unlikely to drop and there's a 75% chance they'll increase by $17 in the next five days. " However, this was too much information for a user to process in decision-making. Because medium confidence predictions were confusing and not actionable, the decision was made to use a confidence rating of 90% or higher. Wording was also changed to say likely to go up or not likely to go down. As a result, three strategies were most helpful in the design of price insights in flights. Articulate data sources. Telling the user what data was being used in the AI's prediction helped the product team avoid contextual surprises and privacy suspicion and helped the user know when to apply their own judgment. Experiment with different confidence indicators. Showing model confidence in categorical buckets and visual graphs helped give users relevant information about flight prices in a way that was easy for them to understand, and account for unexpected user behaviors. Conducting user research early and frequently helped anticipate any unintended consequences of detailed explanations. This helped the product team change its communications approach and therefore bolster user trust. In summary, building machine learning products can be challenging. As you build, applying responsible AI principles throughout your project can profoundly influence a user's trust in both your system and the system's machine learning's usefulness in decision-making. We all have a role in how responsible AI is applied. As you saw in this case study, whatever stage in the AI process you are involved with, from design to deployment or application, your decisions have an impact. Remember, it's important that you too have a defined and repeatable process for using AI responsibly.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.