Data and Model Transparency
1. Data and Model Transparency
Let's explore transparency in detail, specifically data and model transparency. Transparency is a clear, easily understandable, and plain language explanation of what something is, what it does, and why it does that. Machine learning transparency involves sharing information about system behaviors and organizational processes. This might include documenting and sharing how models and datasets were created, trained, and evaluated. Transparency artifacts are a form of structured information reporting that focuses on transparency during product creation and performance to encourage responsible AI adoption and application. Think of this as being similar to nutritional labels for food products. Data cards and model cards are two types of transparency artifacts. Let's first talk about data cards. Data cards are structured summaries of essential facts about various aspects of ML datasets that are needed by stakeholders across a project's life cycle for responsible AI development. You can take advantage of these templates to simply and reliably document your datasets. Data cards can include any of the following that are appropriate for your use case. One, upstream sources. Two, data collection and annotation methods. Three, training and evaluation methods. Four, intended use and five, decisions affecting model performance. When creating documentation for the data that you use, be sure to involve the whole team. This will ensure you ask all the questions that you need to understand about your data and that you include clear and actionable answers to these questions in the data card. You should also involve the following roles when interacting with the data card. Producers, the individuals or teams who will be creating the data card. Consumers, the individuals or teams who will be using the data card and end users, the individuals who might be taking actions based on the system that is or will be built on the dataset. At Google, we provide a data card template that captures 15 themes besides a summary. These are the themes we frequently look for when making decisions, many of which are not traditionally captured in technical dataset documentation. Each of the themes is a section on the data card template like shown in the screenshot. You can find an example of a full data card template in the resources. The data card playbook is a toolkit for transparency in AI dataset documentation. Google provides a data card playbook to help AI ML practitioners achieve and maintain proactive data transparency. The playbook contains four modules designed with participatory activities that define long term transparency for datasets and in context. The transparency patterns capture practical ways to create data cards that are people centric, purposeful and actionable. Now that we've learned about the data card, let's look at another tool kit that provides transparency. The model card, model cards explain what the model is supposed to be used for, how its performance was tested and other important details. Model cards offer benchmark assessments for a range of conditions including diverse, cultural, demographic or phenotypic groups, as well as intersectional groups pertinent to the models intended application domains. They also reveal the context for which the model is designed, elaborate on the performance evaluation procedures and provide additional relevant information. Model developers are primarily responsible for creating a model card. However, anyone involved in idea generation, development, or testing should also be involved in the process, since the requisite knowledge is often distributed across the team. The model card tool kit or MCT library streamlines and automates generation of model cards. Jinja templates form the underlying structure of a model card document. The model card toolkit provides a few ready made templates, but you have the freedom to modify these templates or even create your own for users of tensor flow extended or T effects that MCT can automatically fill these fields using ML metadata or MOND. Additionally, you can manually populate model card fields through a Python API. Here's an example. To get started, import the model card toolkin module in Python. Then initialize the model card tool kit with a path for storing generated assets. Next, initialize the Model Card Toolkit Model card with a path for storing generated assets. Then, initialize the Model Card Toolkit model card, which can be flexibly customized. Finally, we write the model card data to a JSON file and generate the model card document as an XML page. If you're using Tensor Flow Extended or TFX, you can integrate model card generation into your TFX pipeline by using the model card generator component. However, as the model card generator component is being migrated to the TFX avons library, you'll need to install the TFX add ons package beforehand. The model card for the census income classifier serves as a prime example. In the model detail section, you'll find an overall description of the model including its purpose, the data said it was trained on and other relevant information. Additionally, you can access information about the models, version owners and references. The consideration section delves into the models, use cases, limitations and ethical considerations. The lower portion of the model card presents graphs for the train set and eval set, which provide insights into the main training data elements and performance evaluation metrics. Transparency in AI is a fundamental principle that should be accessible to all. This is why model cards are not intended to be a proprietary Google product, but a shared evolving framework that draws from diverse perspectives and contributions.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.