Log Analytics
Log Analytics
Scenario
Azure Monitor collects log data and stores it in tables. As an Administrator, you
configure the input data sources and then conduct queries. Queries provide insights
into your infrastructure. For example, assessing system updates and troubleshooting
operational incidents. To retrieve and consolidate data in the repository you'll create
Kusto Query Language (KQL) queries.
As part of a larger team, you must know what the capabilities are to query and evaluate
the log data that's fed into the service.
3 minutes
Log Analytics is a service in that helps you collect and analyze data generated by
resources in your cloud and on-premises environments.
Log queries help you to use the data collected in Azure Monitor Logs. A powerful query
language allows you to join data from multiple tables, aggregate large sets of data, and
perform complex operations with minimal code. Virtually any question can be answered
and analysis performed as long as the supporting data has been collected, and you
understand how to construct the right query.
Some features in Azure Monitor such as insights and solutions process log data without
exposing you to the underlying queries. To use other features of Azure Monitor, you
should understand how queries are constructed and how you can use them to
interactively analyze data in Azure Monitor Logs.
Example 1 - Assessing updates
An important part of the daily routine for any IT administrator is assessing systems
update requirements and planning patches. Accurate scheduling is critical, as it directly
relates to SLAs to the business and can seriously impact business functions. In the past,
you had to schedule an update with only limited knowledge of how long the patching
would take. Operations Management Suite collects data from all customers performing
patches and uses that data to provide an average patching time for specific missing
updates. This use of “crowd-sourced” data is unique to cloud systems, and is a great
example of how Log Analytics can help meet strict SLAs.
Create a workspace
To get started with Log Analytics you need to add a workspace.
Provide a name for the new Log Analytics workspace.
Select a Subscription from the drop-down list.
For Resource Group, select an existing resource group that contains one or more
Azure virtual machines.
Select the Location your VMs are deployed to.
The workspace will automatically use the Per GB pricing plan.
3 minutes
When you build a query, you start by determining which tables have the data that you're
looking for. Each data source and solution stores its data in dedicated tables in the Log
Analytics workspace. Documentation for each data source and solution includes the
name of the data type that it creates and a description of each of its properties. Many
queries will only require data from a single table, but others may use a variety of options
to include data from multiple tables.
Some common query tables are: Event, Syslog, Heartbeat, and Alert.
Copy
Event
| where (EventLevelName == "Error")
| where (TimeGenerated > ago(1days))
| summarize ErrorCount = count() by Computer
| top 10 by ErrorCount desc
Some common operators are:
Copy
StormEvents | count
Copy
T | limit 5
summarize - Produces a table that aggregates the content of the input table.
Copy
T | summarize count(), avg(price) by fruit, supplier
Copy
T | top 5 by Name desc nulls last
Copy
T | where fruit=="apple"
Tables
Event queues
Text files
An online retailer has a large web farm with more than 100 virtual machines. They would
like to use Log Analytics to ensure these machines are responding to requests. To
automate the process, the team will create a search query. Which source table should be
used?
Event
SysLog
Heartbeat
Microsoft Azure provides a robust alerting and monitoring solution called Azure
Monitor. You can use Azure Monitor to configure notifications and alerts for your key
systems and applications. These alerts will ensure that the correct team knows when a
problem arises.
You work for a large shipping company that recently deployed several web applications
to the Azure platform. Due to a configuration error, the customer-facing order tracker
was offline. The issue wasn't identified until customers started complaining that they
couldn't track their orders. As a consequence, customer satisfaction with your service
dropped.
As your company's Azure solution architect, you need to find a solution that detects
problems in your environments in real time. The correct team will be notified so it can
resolve any problems before your customers notice.
Learning objectives
In this module, you'll:
Prerequisites
Knowledge of Azure Monitor
Training
Products
Roles
Learning Paths
Courses
Educator Center
Student Hub
FAQ & Help
LEVEL 712400 /16199 XP
1. Learn
2. Training
3. Browse
Previous
Unit 2 of 8
Next
Explore the different alert types that Azure Monitor supports
Completed200 XP
6 minutes
Azure Monitor is a powerful reporting and analytics tool. Use it for insights into the
behavior and running of your environment and applications. You can then respond
proactively to faults in your system.
After the downtime that your customers faced, you set up monitoring on your key
resources in Azure. With the monitoring in place, you want to make sure the right
people are being alerted at the right level.
In this unit, you'll see how Azure Monitor receives resource data, what makes up an
alert, and how and when to use an alert. Finally, you'll learn how to create and manage
your own alerts.
The focus for metric-based data types is the numerical time-sensitive values that represent
some aspect of the target resource.
The focus for log-based data types is the querying of content data held in structured,
record-based log files that are relevant to the target resource.
You'll learn about the three signal types that you can use to monitor your environment:
Metric alerts provide an alert trigger when a specified threshold is exceeded. For example,
a metric alert can notify you when CPU usage is greater than 95 percent.
Activity log alerts notify you when Azure resources change state. For example, an activity
log alert can notify you when a resource is deleted.
Log alerts are based on things written to log files. For example, a log alert can notify you
when a web server has returned a number of 404 or 500 responses.
RESOURCE
o The target resource to be used for the alert rule. It's possible to assign multiple target
resources to a single alert rule. The type of resource will define the available signal
types.
CONDITION
o The signal type to be used to assess the rule. The signal type can be a metric, an activity
log, or logs. There are others, but this module doesn't cover them.
o The alert logic applied to the data that's supplied via the signal type. The structure of
the alert logic will change depending on the signal type.
ACTIONS
o The action, like sending an email, sending an SMS message, or using a webhook.
o An action group, which typically contains a unique set of recipients for the action.
ALERT DETAILS
o An alert name and an alert description that should specify the alert's purpose.
o The severity of the alert if the criteria or logic test evaluates true. The five severity levels
are:
o 0: Critical
o 1: Error
o 2: Warning
o 3: Informational
o 4: Verbose
Scope of alert rules
You can get monitoring data from across most of the Azure services, and report on it by
using the Azure Monitor pipeline. In the Azure Monitor pipeline, you can create alert
rules for these items and more:
Metric values
Log search queries
Activity log events
Health of the underlying Azure platform
Tests for website availability
The following alert capabilities aren't yet available for the generation of monitoring
data:
As an Azure solution architect, you'd use Azure Monitor to enable tightly focused and
specific alerts before any application change. You'd then disable the alerts after a
successful deployment.
You can control the alert state to manage and specify where you are in the alert
resolution process. Currently, there are three states:
Every new alert has an alert state of New. This state means that the issue has been
detected, but not yet reviewed.
After an admin has reviewed the alert and is working on it, the alert state changes
to Acknowledged.
When the issue is resolved, the alert state is set to Closed.
1.
Database tables
Learn
Documentation
Training
Certifications
Q&A
Code Samples
Shows
Events
Training
Products
Roles
Learning Paths
Courses
Educator Center
Student Hub
FAQ & Help
LEVEL 712500 /16199 XP
1. Learn
2. Training
3. Browse
Previous
Unit 3 of 8
Next
Use metric alerts for alerts about performance issues in your
Azure environment
Completed100 XP
4 minutes
Azure Monitor can use thresholds to monitor specific resources. In an organization, it's
far more useful to be notified when the free disk space on a server is less than 5 percent,
instead of being alerted every time a file is saved.
As a solution architect, you want to implement regular threshold monitoring for many of
your target resources and instances. Monitoring will help to head off potential issues
before they can affect your customers.
In this unit, you'll investigate the different kinds of metric alerts that Azure Monitor
supports.
Metric alerts can be useful if, for instance, you need to know when your server CPU
utilization is reaching a critical threshold of 90 percent. You can be alerted when your
database storage is getting too low, or when network latency is about to reach
unacceptable levels.
You must define the type of statistical analysis to be used with either static or dynamic
metric alerts. Example types are minimum, maximum, average, and total. In this example,
you define the period of data to be assessed: the last 10 minutes. Finally, you set the
frequency by which the alert conditions are checked: every 2 minutes.
Static metric alerts are based on simple static conditions and thresholds that you define.
With static metrics, you specify the threshold that will be used to trigger the alert or
notification.
In the previously defined scenario, a static alert with a threshold of 85 percent CPU
utilization checks the rule every 2 minutes. It evaluates the last 10 minutes of CPU
utilization data to assess if it rises above the threshold. If the evaluation is true, the alert
triggers the actions associated with the action group.
Dynamic metric alerts use machine-learning tools that Azure provides to automatically
improve the accuracy of the thresholds defined by the initial rule.
There's no hard threshold in dynamic metrics. However, you'll need to define two more
parameters:
Understand dimensions
Until now, the assessed metric alerts have focused on a single target instance. Azure
Monitor supports dimensions, which enable monitoring data to be supplied from
multiple target instances.
You can use dimensions to define one metric alert rule and have it applied to multiple
related instances. For example, you can monitor CPU utilization across all the servers
running your app. You can then receive an individual notification for each server
instance when the rule conditions are triggered.
You can define the dimensions by naming each target instance specifically, or you can
define the dimensions by using the asterisk (*) wildcard, which will use all available
instances.
Creating scaling metric alert rules to monitor multiple resources is no different than
creating any other metric alert rule. You just select all the resources that you want to
monitor.
Like dimensions, a scaling metric alert is individual to the resource that triggered it.
10 minutes
Microsoft Learn needs your permission to create Azure resources.
Review permissions
The shipping company you work for wants to avoid any future issues with updates to its
applications on the Azure platform. To improve the alert capabilities in Azure, you've chosen to
use Azure metric alerts.
In this exercise, you'll create a Linux virtual machine (VM). This VM will run an app that runs
the CPU at 100 percent utilization. You'll create monitoring rules in the Azure portal and in the
Azure CLI to alert you about high CPU usage.
Create the VM
This VM will run a specific configuration that stresses the CPU, and generates the metric
monitoring data needed to trigger an alert.
BashCopy
cat <<EOF > cloud-init.txt
#cloud-config
package_upgrade: true
packages:
- stress
runcmd:
- sudo stress --cpu 1
EOF
2. To set up an Ubuntu Linux VM, run the following az vm create command. This command
uses the cloud-init.txt file that you created in the previous step to configure the VM
after it's created.
Azure CLICopy
az vm create \
--resource-group [sandbox resource group name] \
--name vm1 \
--location eastUS \
--image UbuntuLTS \
--custom-data cloud-init.txt \
--generate-ssh-keys
Wait until the VM is successfully created before proceeding with the exercise. The VM creation
process is complete when you see the completed JSON output in the Azure Cloud Shell window.
You can use either the Azure portal or the CLI to create a metric alert. In this exercise we'll cover
both, starting with the Azure portal.
1. Sign in to the Azure portal by using the same account that you used to activate the
sandbox.
2. On the Azure portal menu, search for and select Monitor. The Monitor | Overview pane
appears.
3. First, you'll set the scope of the resources to be monitored. In the left menu pane,
select Metrics. The Monitor | Metrics pane appears with the Select a scope pane open on
the right.
4. On the Select a scope pane, Enter vm in the Search to filter items... field. You should
see vm1 in the Scope list. Select the box for vm1, then select Apply at the bottom of the
pane.
5. On the Monitor | Metrics pane, select Percentage CPU in the Metric dropdown list, and
select Max in the Aggregation dropdown list.
6. On the command bar, select New alert rule. The Condition tab of the Create an alert
rule pane appears.
7. Enter (or confirm) the following values for each setting.
Setting Value
Alert logic
Threshold Static
Threshold value 90
When to evaluate
Setting Value
10. Open the Advanced options section, and confirm the following values for each setting.
Setting Value
11.
You've successfully created a metric alert rule that will trigger an alert when the CPU percentage
on the VM exceeds 90 percent. The rule will check every minute and review one minute of data.
It can take up to 10 minutes for a metric alert rule to become active.
Let's create a new metric alert similar to the one you set up in the Azure portal.
1. Run the following command in Cloud Shell to obtain the resource ID of the virtual
machine previously created.
BashCopy
VMID=$(az vm show \
--resource-group [sandbox resource group name] \
--name vm1 \
--query id \
--output tsv)
2. Run the following command to create a new metric alert that will be triggered when the
VM CPU is greater than 80 percent.
Azure CLICopy
az monitor metrics alert create \
-n "Cpu80PercentAlert" \
--resource-group [sandbox resource group name] \
--scopes $VMID \
--condition "max percentage CPU > 80" \
--description "Virtual machine is running at or greater than 80% CPU
utilization" \
--evaluation-frequency 1m \
--window-size 1m \
--severity 3
Note
It might take 10 minutes before you see the alerts show up in your portal.
This step presents the alert summary pane, where you can see the count of the number of
alerts. If you don't see your alerts listed, wait a few minutes and select Refresh.
3. You configured your metric alerts with severities of 2 and 3. Select one of the alerts to
view the severity level.
Learn
Documentation
Training
Certifications
Q&A
Code Samples
Shows
Events
Training
Products
Roles
Learning Paths
Courses
Educator Center
Student Hub
FAQ & Help
LEVEL 712700 /16199 XP
1. Learn
2. Training
3. Browse
Previous
Unit 5 of 8
Next
Use log alerts to alert on events in your application
Completed100 XP
4 minutes
You can use Azure Monitor to capture important information from log files. These log
files can be created by applications, operating systems, other hardware, or Azure
services.
As a solution architect, you want to explore ways that monitoring log data can detect
issues before they become issues for your customers. You know that Azure Monitor
supports the use of log data.
In this unit, you'll understand how using log data can improve resilience in your system.
By its nature, log data is historical. So usage is focused on analytics and trends.
You use these types of logs to assess if any of your servers have exceeded their CPU
utilization by a given threshold during the last 30 minutes, or you can evaluate response
codes issued on your web application server in the last hour.
When a log search evaluates as positive, an alert record is created, and any associated
actions are triggered.
Log query: Query that runs every time the alert rule fires.
Time period: Time range for the query.
Frequency: How often the query should run.
Threshold: Trigger point for an alert to be created.
Log search results are one of two types: number of records or metric measurement.
Number of records
Consider using the number-of-records type of log search when you're working with an
event or event-driven data. Examples are syslog and web-app responses.
This type of log search returns a single alert when the number of records in a search
result reaches or exceeds the value for the number of records (threshold). For example,
when the threshold for the search rule is greater or equal to five, the query results have
to return five or more rows of data before the alert is triggered.
Metric measurement
Metric measurement logs offer the same basic functionality as metric alert logs.
Aggregate function: The calculation that will be made against the result data. An
example is count or average. The result of the function is called AggregatedValue.
Group field: A field by which the result will be grouped. This criterion is used with the
aggregated value. For example, you might specify that you want the average grouped by
computer.
Interval: The time interval by which data is aggregated. For example, if you specify 10
minutes, an alert record is created for each aggregated block of 10 minutes.
Threshold: A point defined by an aggregated value and the total number of breaches.
Consider using this type of alert when you need to add a level of tolerance to the results
found. One use for this type of alert is to respond if a particular trend or pattern is
found. For example, if the number of breaches is five, and any server in your group
exceeds 85 percent CPU utilization more than five times within the given time period, an
alert is fired.
As you can see, metric measurements greatly reduce the volume of alerts that are
produced. Still, give careful consideration when you're setting the threshold parameters
to avoid missing critical alerts.
Learn
Documentation
Training
Certifications
Q&A
Code Samples
Shows
Events
Training
Products
Roles
Learning Paths
Courses
Educator Center
Student Hub
FAQ & Help
LEVEL 712800 /16199 XP
1. Learn
2. Training
3. Browse
Previous
Unit 6 of 8
Next
Use activity log alerts to alert on events within your Azure
infrastructure
Completed100 XP
10 minutes
Activity log alerts allow you to be notified when a specific event happens on some Azure
resource. For example, you can be notified when someone creates a new VM in a
subscription.
An activity log can also include alerts for Azure service health. A company can be
notified when service issues or planned maintenance happens on the Azure platform.
As an Azure solution architect, you want to explore the capability to monitor selected
Azure resources within your subscription. You'll understand how the resources can be
used to improve your team's responsiveness and the stability of your systems.
In this unit, you'll explore the two different kinds of activity log alerts. Now that you've
seen all the different kinds of alerts you can use in Azure Monitor, you'll see how you
can trigger actions for your alerts. Actions might include sending an email, or creating
an IT Service Management (ITSM) support ticket.
Activity log alerts are designed to work with Azure resources. Typically, you'd create this
type of log to receive notifications when specific changes occur on a resource within
your Azure subscription.
There are two types of activity log alerts:
Specific operations: Applies to resources within your Azure subscription, and often has a
scope with specific resources or a resource group. You'll use this type when you need to
receive an alert that reports a change to an aspect of your subscription. For example, you
can receive an alert if a VM is deleted or new roles are assigned to a user.
Service health events: Include notice of incidents and maintenance of target resources.
Activity log alerts are based on events. The best approach for defining them is to use
Azure Monitor to filter all the events in your subscription until you find the one that you
want. To begin the creation process, you'll then select Add activity log alert.
Like the previous alerts, activity log alerts have their own attributes:
When you create your activity log alert, you'll select Activity Log for the signal type.
You'll then see all the available alerts for the resource you select. The following image
shows all the administrative alerts for Azure VMs. In this example, an alert is triggered
when a VM is powered off.
Changing the monitor service will enable you to reduce the list of options.
Selecting Administrative filters all the signals to show only admin-related signals.
Create a service health alert
Service health alerts aren't like all the other alert types you've seen so far in this module.
To create a new alert, on the Azure portal, search for and select Service Health. Next,
select Health alerts. After you select Create service health alert, the steps to create the
alert are similar to the steps you've seen to create other alerts.
The only difference is that you no longer need to select a resource, because the alert is
for a whole region in Azure. What you can select is the kind of health event on which
you want to be alerted. It's possible to select service issues, planned maintenance, or
health advisories, or to choose all of the events. The remaining steps of performing
actions and naming the alerts are the same.
When any event is triggered, you can create an associated action in an action group.
Action groups allow you to define actions that will be run. You can run one or more
actions for each triggered alert.
Send an email
Send an SMS message
Create an Azure app push notification
Make a voice call to a number
Call an Azure function
Trigger a logic app
Send a notification to a webhook
Create an ITSM ticket
Use a runbook (to restart a VM, or scale a VM up or down)
You can also reuse action groups on multiple alerts after you've created them. For
example, after you've created an action to email your company's operations team, you
can add that action group to all the service health events.
You can add or create action groups at the same time that you create your alert. You
can also edit an existing alert to add an action group after you've created it.
Exercise - Use activity log alerts to alert on events within your
Azure infrastructure
Completed100 XP
10 minutes
This module requires a sandbox to complete.
A sandbox gives you access to free resources. Your personal subscription will not be
charged. The sandbox may only be used to complete training on Microsoft Learn. Use
for any other reason is prohibited, and may result in permanent loss of access to the
sandbox.
Microsoft provides this lab experience and related content for educational purposes. All
presented information is owned by Microsoft and intended solely for learning about the
covered products and services in this Microsoft Learn module.
Activate sandbox
The shipping company you work for wants to avoid any future issues with updates to its
applications on the Azure platform. To improve the alerting capabilities within Azure,
you'll use activity log alerts.
Your goal is to set up a Linux VM and create an activity log monitoring rule to detect
when a VM is deleted. You'll then delete the VM to trigger this alert.
6. You want an alert when any virtual machine in your resource group is deleted.
Select the box for the [sandbox resource group name] resource group, and then
select Done.
9. The Create an alert rule pane reappears. You want to receive alerts of all types, so
leave Alert logic settings at their default of All selected. Leave the Create an alert
rule pane open for the next section.
Setting Value
Project details
Resource group From the dropdown list, name of your sandbox resource group
Instance details
3. Select Next : Notifications, and enter the following values for each setting.
Setting Value
7. Select Create.
Setting Value
9. Open the Advanced options section, and confirm that the following value is set.
Setting Value
12. Select Create.
Recipients added to the configured action group (operations team) will receive a
notification:
4. To confirm the deletion, check the box at the bottom of the pane to confirm, then
select Delete.
1. You should have received a notification email that reads, Important notice: Azure
Monitor alert VM was deleted was activated... If not, open your email program
and look for an email from [email protected].
2. On the Azure portal resource menu, select Monitor, and then in the Monitor |
Alerts menu select Alerts.
3. You should see three verbose alerts that were generated by deleting vm1.
4. Select the name of one of the alerts (For example, VM was deleted). An Alert
details pane appears that shows more details about the event.