CloudWatch metrics

1. CloudWatch metrics

Welcome back! In this video, we'll look in more detail at CloudWatch metrics.

2. Anatomy of a CloudWatch Metric

A CloudWatch metric is a time-ordered set of data points. Metrics are separated into namespaces and can have dimensions for filtering, such as by instance name. The core data in each is the unit of measure, value, and timestamp.

3. Standard vs. custom metrics

Standard metrics are published automatically and typically included in service pricing. Finer granularity often costs extra, as do custom metrics. Custom metrics let you measure almost anything: page load times, orders processed, cache hit rates, with full control over namespace, dimensions, unit, and resolution down to one second. Remember that every unique combination of namespace, metric name, and dimensions is a separate billable custom metric, so be deliberate about what you track.

4. Metric resolutions

AWS tracks many core metrics by default, found in each service's monitoring pane and the CloudWatch console. These are typically coarser, and you alter the resolution for more detail. OS-level metrics for things like VMs aren't included by default and can require extra configuration. Custom metrics come into their own for instrumenting applications, giving deeper insight and control.

5. Publishing custom metrics

There are four ways to publish custom metrics. First, the AWS CLI and its put-metric-data command, handy for quick testing, which we'll see shortly. Second, the REST API over HTTP POST, for direct integration. Third, an Amazon SDK, the best fit for building metric publishing into application code, with libraries for all major languages. Fourth, the CloudWatch Unified Agent, which collects OS-level metrics like memory and disk that AWS doesn't capture.

6. Publishing with CLI

Here we use the CloudWatch module in the AWS CLI with the put-metric-data command, publishing how long a step took in a CI/CD pipeline and adding the pipeline name as a dimension. Anything that can call a CLI can send a custom metric to CloudWatch; just remember to use dimensions to aid searching later.

7. Publishing with SDK

Several SDKs are available. In the C# shown here, the using statements pull in the Amazon CloudWatch libraries; there are also SDKs for Java, Go, Rust, and more. As with the CLI, we use dimensions to segment the same metric across parts of your application for clearer analysis.

8. Unified agent

The Unified Agent collects OS-level metrics and logs that CloudWatch doesn't capture by default, on EC2 machines and containers. Install the agent and apply a JSON (JavaScript Object Notation) configuration file defining what to collect and how often. Here a 30-second default is overridden to 5 seconds for CPU, giving the granularity needed for troubleshooting.

9. Embedded Metric Format (EMF)

There's one more way to publish custom metrics, and the exam expects you to know it: the embedded metric format, or EMF. Instead of calling an API, you write a specially structured JSON object to your logs. The underscore-aws block declares the namespace, the dimensions, and which fields are metrics, here latency in milliseconds, while the plain fields below carry the values. CloudWatch reads those logs and turns them into metrics for you. It's the go-to for high-cardinality metrics from Lambda and containers, because you get your metrics and your logs from a single write, with no separate API call.

10. Metric streams

Metric streams continuously export your metrics outside CloudWatch, to an S3 bucket or partner tools like New Relic or Datadog, giving you one place for analytics across systems.

11. Streaming to Amazon S3

To reach S3, metrics flow through Amazon Kinesis Data Firehose, which lands them as JSON in your bucket, ready for other tools to query.

12. Streaming to partner solution

For partners like New Relic, Datadog, or Splunk it's the same pattern: Firehose delivers to the partner's endpoint, with an S3 bucket catching any failed deliveries. The console's Quick Start templates handle the setup.

13. Summary

To recap: CloudWatch metrics measure attributes of your workloads, and custom metrics extend that to your own applications, with dimensions to slice the data. Resolution matters too, a trade-off between free standard metrics and paid high-resolution, with the default service dependent. Publish custom metrics by instrumenting processes and applications, adding dimensions for categorization, and using the SDK or CLI over the raw API. Finally, metric streams export data to S3 or partner solutions for custom analytics, a single pane of glass across hybrid environments.

14. Let's practice!

Let's now put what we've learned about metrics into practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.