Data sanitization and masking

1. Data sanitization and masking

Welcome back! In this video, you'll learn data sanitization techniques, implement masking for different data types, and ensure compliance with GDPR and HIPAA requirements. Let's begin.

2. The log file that exposed everything

A company logged user activity for debugging. Their logs contained full national ID numbers, credit cards, and passwords in plain text, sent to CloudWatch where the entire dev team had access. A compliance audit discovered this GDPR and HIPAA violation. The fine was $2 million dollars. How can this be avoided? Sanitize data before logging. Let's see what data sanitization is.

3. What is data sanitization?

Data sanitization removes or obscures sensitive information before it leaves your secure environment. Sanitize before logging to CloudWatch, before displaying data in user interfaces, and before sending to third-party services. The goal is maintaining data utility for debugging and analytics while protecting privacy. You can still troubleshoot issues without exposing actual national IDs or credit cards.

4. Full masking technique

Full masking replaces every character with asterisks. Use this for passwords, API keys, and authentication tokens that should never be visible. The implementation is simple: replace the entire string with asterisks of the same length, or use a fixed number like ten asterisks. Never log passwords, even masked: just log that authentication occurred.

5. Partial masking technique

Partial masking shows the last few characters for identification while hiding the rest. Perfect for national IDs, credit cards, and account numbers. Show the last 4 digits so users can verify which card or account, but mask everything else. Preserve formatting like dashes to maintain readability. This balances security with usability.

6. Hashing and tokenization

Hashing converts data to a fixed-length string using SHA-256 or similar: it's one-way, meaning you can't reverse it back to the original. Use it for analytics where you need to count unique users without storing actual emails. Tokenization replaces sensitive data with random tokens and stores the mapping securely: unlike hashing, this is reversible. Use tokenization for testing environments or when you need to retrieve the original data later.

7. Redaction technique

Redaction completely removes sensitive information from logs and documents. Use redaction for medical diagnoses, legal information, or anything that shouldn't appear in logs at all. It replaces the data with [REDACTED] or [REMOVED] to show something was there. This is the most secure option when you don't need the data for debugging or analytics.

8. Implementing masking functions

Create a utility library with masking functions for each data type: mask_national_id, mask_credit_card, mask_email. Apply them consistently before logging, displaying, or transmitting data. Critical: never store masked data in your database. Store the original encrypted data and mask only when outputting. You might need the entire data for customer support or fraud investigation.

9. Compliance and testing

GDPR requires data minimization: only collect and display minimum necessary data. HIPAA has a minimum necessary standard meaning showing only what's needed for the job. Test your masking functions thoroughly with real patterns: national IDs with and without dashes, international phone numbers, various email formats. Edge cases will break your masking if you don't test.

10. Let's practice!

Time to practice your data sanitization skills. Complete the exercises to master masking techniques and compliance requirements.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.