Introduction to query tools
1. Introduction to query tools
Being able to sort through large amounts of information is a highly useful skill in cybersecurity. In this video, you'll learn how to sift through aggregated data to find important information. That way, you can support data analysis during the investigation of a potential security threat. Let’s begin by learning how to query effectively. When working with logging systems in a cloud environment, you'll likely encounter large amounts of log data from multiple sources. Knowing how to effectively query log data can help you filter, sort, and manipulate log data to find specific information. There are lots of powerful query tools that can be used to do this. One common method is regular expression, or RegEx, which is a sequence of characters that forms a pattern. Regular expressions use these characters to search text and match specific patterns. Patterns define the parameters of a search. If a piece of text matches a given pattern, RegEx retrieves it for you. When analyzing logs in cloud environments, you can use RegEx to search through log data and extract any relevant information you need. For example, imagine you’re given an authentication log with thousands of lines of events, and need to find out if there were any failed SSH login attempts. You can use RegEx to specifically search only for entries containing the SSH protocol and port number, along with any failed login attempts. Let’s check out an example from a Google Cloud Logs Explorer search. Here's a regular expression query that matches SSH log entries. Let's examine each element. The first field, logName, identifies the log file the log entry belongs to. The equal sign and tilde is an operator that tells the application to query using regular expressions. For this example, it's Logs Explorer. Here is where the RegEx begins. The quotations are used in RegEx syntax to match a specific string. In this example, the quotations enclose the string ssh. The caret symbol is used in RegEx to match the beginning of a string. This means it will only find log entries that start with the string ssh. There are lots of different implementations of RegEx available in programming languages, applications, and tools. For example, with Google Cloud's Logs Explorer, you can use regular expressions to build queries to search through log entries. Searching with regular expressions has lots of advantages, like increased search performance compared to a regular text query. Because of its wide usage across different applications and programming languages, RegEx syntax can vary depending on the implementation. But, what's important to know is the core set of features used to match patterns in text. Let's examine some RegEx fundamentals. Square brackets are used to match a range of characters. For example, the brackets here enclose the characters abc. This means, RegEx will match any of the characters defined in the brackets. Similar to a wildcard, periods are used to match any single character. In this example, d.g will substitute the period for any other character. It will return match results for text like dig, dog, and dug. \s matches any whitespace character like a space or tab. In this example, \s is between small and coffee. This means RegEx will only match for text that has a space or tab between the words small and coffee. \d matches any digit from 0 to 9. This example begins with a caret symbol, then \d. This will match a string that starts with a digit. The curly brackets contain the number of digits you want to match, which in this case, is 5 digits. As you learn more about regular expressions, you'll be able to use these fundamentals in more complex ways to help you find the information you need. In Google Cloud's Chronicle, YARA-L is a computer language used to create detection rules for searching through ingested log data. The first field rule provides the rule name DifferentCityLogin. The meta section provides additional information about the rule. Under the meta section, there's events, which specify the UDM events that the rule needs to match on. In this case, the rule is looking for three things: user logins, users, and cities. The next section, match, specifies what must be met for the rule to match. In this case, it's user over 5m, which means the rule tries to identify a user who has logged in within a five minute period and ignores all login events that exceed 5 minutes. Finally, the condition specifies the conditions that must be met for the rule to generate an alert. In this case, the rule is looking for UDM events that match what's defined in the match section, and that have more than one city value. So, this rule will match if a user logs in from more than one city in a five minute period. If all conditions defined in this YARA-L rule are met, an alert will be generated. In the YARA-L rule you just examined, you might remember the use of UDM event types. Chronicle formats aggregated data using unified data model, or UDM. UDM is a data model used to process and store data. UDM is kind of like a filing cabinet that's used to organize files or, in this case, aggregated log data. Just like a filing cabinet uses clear labeling to identify the contents of a file, UDM organizes data and attaches UDM fields to it to represent important data points. This makes data easier to search through, and helps security professionals identify potential threats. With Chronicle, you can perform a UDM search. Through a UDM search, Chronicle searches security data that’s been ingested and formatted in UDM. With time and practice, you can learn how to effectively use these querying tools throughout your career. And as an added bonus, you'll be able save lots of time in log analysis by crafting effective queries.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.