Logging

1. Logging

Monitoring is the basis of Google Cloud Observability, but the service also provides logging, error reporting, and tracing. Let's learn about logging. Cloud Logging allows you to store, search, analyze, monitor, and alert on log data and events from Google Cloud and AWS. It is a fully managed service that performs at scale and can ingest application and system log data from thousands of VMs. Logging includes storage for logs, a user interface called Logs Explorer, and an API to manage logs programmatically. The service lets you read and write log entries, search and filter your logs, and create log-based metrics. Logs are only retained for 30 days, but you can export your logs to Cloud Storage buckets, BigQuery datasets, and Pub/Sub topics. Exporting logs to Cloud Storage makes sense for storing logs for more than 30 days, but why should you export to BigQuery or Pub/Sub? Exporting logs to BigQuery allows you to analyze logs and even visualize them in Looker Studio. BigQuery runs extremely fast SQL queries on gigabytes to petabytes of data. This allows you to analyze logs, such as your network traffic, so that you can better understand traffic growth to forecast capacity, network usage to optimize network traffic expenses, or network forensics to analyze incidents. For example, in this screenshot I queried my logs to identify the top IP addresses that have exchanged traffic with my web server. Depending on where these IP addresses are, and who they belong to, I could relocate part of my infrastructure to save on networking costs or deny some of these IP addresses if I don't want them to access my web server. If you want to visualize your logs, I recommend connecting your BigQuery tables to Looker Studio. Looker Studio transforms your raw data into the metrics and dimensions that you can use to create easy-to-understand reports and dashboards. I mentioned that you can also export logs to Pub/Sub. This enables you to stream logs to applications or endpoints.

2. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.