Resource Monitors
1. Resource Monitors
Some things in the data world are like brushing your teeth – You know you should do it, you’re happy after you’ve done it, so you do it, but you don’t usually wake up going: “You know what I’m most looking forward to today? A good toothbrushing.” That’s kind of how I feel about resource monitors. Resource monitors are Snowflake objects you can create to put limits on credit usage for an account or a specific warehouse. Very flexible, really important, and you’re happy once you’ve created them, but the topic doesn’t seem inherently awe-inspiring. The good news is that Snowflake makes this kind of data hygiene very easy – So if creating resource monitors is a little like brushing your teeth, at least it’s like brushing your teeth with a delightful toothpaste and a really, really great toothbrush. Let’s start by using the UI to create a resource monitor. If you go to the left hand side of the screen and click on Admin -> Cost Management -> Resource Monitors, you’ll see that there’s a “+ Resource Monitor” button at the top right of the screen. Click on that. And if you’re not seeing an “Admin” tab on the left at all, you might have to go to the bottom left of the screen and switch your role to Account Admin. Let’s name our Resource Monitor “first_resource_monitor.” Before we specify a credit quota, let’s talk about the two different kinds of resource monitors – account-level monitors, and warehouse-level monitors. If you create an account-level monitor, it means you’re putting guardrails on total credit consumption across all warehouses in the account. You can only have one account-level monitor. If you create a warehouse-level monitor, that’s specific to a particular warehouse. Here, let’s start by making an account-level one. Let’s set a daily credit maximum of 30 credits – that’s a lot of credits. If an x-small warehouse runs for an hour, that’s 1 credit, so you could have an x-small warehouse running continuously for 24 hours and still not hit this quota. Yes, we want to start monitoring immediately. No, we don’t want to end the monitoring. No, we don’t want to reset monthly – let’s click “customize” and go to the bottom to change that to daily. Then let’s have it start immediately. Great! Now we can move on to what we want to have happen as we hit certain percentages of our quota during a given day. Let’s make sure we get a notification once we’ve used 80% of our credit quota for the day. Then let’s suspend all of the warehouses in the account once we hit 100% of our credit quota, but we’ll let the running queries finish. And if those running queries go through enough credits to bring us to the 110% mark of our credit quota, let’s immediately suspend even those queries. This will be the hard upper limit. Awesome! Now we click “Create Resource Monitor” to make our monitor. Cool, so now that we’ve made an account-level monitor using the UI, let’s make a warehouse-level monitor through code to see how that experience differs. So let’s go to Projects -> Worksheets, and open up a worksheet I made called “Resource Monitors” Here, we use a CREATE RESOURCE MONITOR command, followed by the name of our monitor, “tasty_test_rm.” Then we have a “WITH” clause followed by whatever it is we want to specify about our monitor. CREATE RESOURCE MONITOR tasty_test_rm WITH CREDIT_QUOTA \= 20 -- 20 credits FREQUENCY \= daily -- reset the monitor monthly START_TIMESTAMP \= immediately -- begin tracking immediately TRIGGERS ON 80 PERCENT DO NOTIFY -- notify accountadmins at 80% ON 100 PERCENT DO SUSPEND -- suspend warehouse at 100 percent, let queries finish ON 110 PERCENT DO SUSPEND_IMMEDIATE; -- suspend warehouse and cancel all queries at 110 percent In our case, let’s set a daily limit of 20 credits for a particular warehouse. (Note that we haven’t yet specified which warehouse we want to apply this to – that’s a separate step we’ll do later. Right now, we’re just creating the monitor before we activate it.) We’ll set the frequency to daily. The other options are [Weekly](https://docs.snowflake.com/en/user-guide/resource-monitors), Monthly, Yearly, or Never. We’ll have it start immediately. We could also have set a future timestamp as the start time. We’ll specify similar triggers to those we set at the account level: notify us at 80% of the daily credit quota, prevent the warehouse from starting new queries at 100% of the credit quota, and shut down even running queries at 110% of the quota. These [three](https://docs.snowflake.com/en/user-guide/resource-monitors) – notify, notify and suspend, and notify and suspend immediately – are the only three action types a Resource Monitor can trigger, so we’re availing ourselves of all the options here. Given the number of other times we’ve seen this pattern, it shouldn’t surprise you to hear that there’s a SHOW RESOURCE MONITORS command. Let’s run that and make sure our Monitor was created successfully. SHOW RESOURCE MONITORS; We can see the account-level monitor we made before, which has a 30-credits-per-day quota, and the new monitor, which has a 20-credits-per-day quota. All of that looks good. Great, so now let’s apply this monitor to a warehouse to get it up and running. To do this, we run an “ALTER WAREHOUSE” command. ALTER WAREHOUSE tasty_de_wh SET RESOURCE_MONITOR \= tasty_test_rm; I could have imagined a version of the world where we specify inside the Resource Monitor object which warehouse we want it to apply to, but instead we alter the warehouse to associate it with the Resource Monitor. Now if we run SHOW RESOURCE MONITORS again, we should see that it’s at the level of a warehouse: SHOW RESOURCE MONITORS; Great! That worked. If you want to alter something about a resource monitor, you can go to the list of resource monitors in Admin -> Cost Management -> Resource Monitors and edit the resource monitor there. Or you can use an ALTER RESOURCE MONITOR command and specify the parameter you want to change. For example, let’s say we wanted to change our resource monitor to have a 30-credit daily quota. We’d just run: ALTER RESOURCE MONITOR tasty_test_rm SET CREDIT_QUOTA=30; You can also drop a resource monitor with the DROP RESOURCE MONITOR command, followed by the name of the resource monitor. I’ve grown a bit attached to this tasty_test_rm, so I’m sad to do this, but let’s drop it. We do that by running: DROP RESOURCE MONITOR tasty_test_rm; And if we show resource monitors, we’ll see it’s gone: SHOW RESOURCE MONITORS; Goodbye, tasty_test_rm. Your stay here was far too short. So that’s all we’re going to cover for resource monitors! Here’s a quick recap of what we did: We learned about the difference between account-level and warehouse-level resource monitors, we create an account-level resource monitor using the UI, we learned about the three different actions Resource Monitors can trigger, we created a warehouse-level resource monitor using the CREATE RESOURCE MONITOR command, we applied the resource monitor to a warehouse with the ALTER WAREHOUSE command, we adjusted a parameter on a resource monitor with the ALTER RESOURCE MONITOR command, we showed all resource monitors with the SHOW RESOURCE MONITORS command, and we dropped a resource monitor with DROP RESOURCE MONITOR. Hygiene metaphors are inherently uncomfortable, but I really do think that Snowflake makes this kind of “brushing your teeth” experience pretty darn pleasant, all things considered. And once you get used to setting up resource monitors and adjusting them quickly, you’ll feel so proficient that you might even find yourself having fun from time to time!2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.