Microsoft Cloud What IT architects need to know about

networking in Microsoft cloud services

Networking for and platforms

Enterprise Architects
This topic is 1 of 6 in a series 1 2 3 4 5 6

Evolving your network for cloud connectivity

Cloud migration changes the volume and nature of traffic flows within and outside a corporate
network. It also affects approaches to mitigating security risk.

Before the cloud After the cloud

Most networking infrastructure investments were spent on ensuring With new and migrated productivity and IT workloads running in the
available, reliable, and performant connectivity to on-premises cloud, infrastructure investments shift from on-premises datacenters
datacenters. For many organizations, Internet connectivity was not to Internet connectivity, which is now critical for internal business
critical for internal business operations. Network boundaries were operations. Federated connectivity shifts security strategy to
primary defenses against security breaches. protecting identities and data as they flow through the network and
points of connectivity to Microsoft cloud services.
Network infrastructure investments begin with connectivity. Additional
investments depend on the category of cloud service.

SaaS Azure PaaS Azure IaaS

Software as a Service Platform as a Service Infrastructure as a Service

Microsoft SaaS services include Office 365, In addition to the investments for Microsoft In addition to the investments for Microsoft
Microsoft Intune, and Microsoft Dynamics 365. SaaS services, multi-site or geographically SaaS and PaaS services, running IT workloads
Successful adoption of SaaS services by users distributed PaaS applications might require in IaaS requires the design and configuration
depends on highly-available and performant architecting Azure Application Gateway or of Azure virtual networks that host virtual
connectivity to the Internet, or directly to Azure Traffic Manager to distribute client machines, secure connectivity to applications
Microsoft cloud services. traffic. Ongoing investments include running on them, routing, IP addressing,
performance and traffic distribution DNS, and load balancing. Ongoing
Network architecture focuses on reliable,
monitoring and failover testing. investments include performance and
redundant connectivity and ample bandwidth.
security monitoring and troubleshooting.
Ongoing investments include performance
monitoring and tuning.

Microsoft 365 is a combination of Office 365, Enterprise Management + Security (EMS), and
Windows 10. Microsoft 365 combines multiple SaaS and Azure services for a complete, intelligent
solution that empowers everyone to be creative and work together securely.

Areas of networking investment for success in the cloud

Enterprise organizations benefit from taking a methodical approach to
optimizing network throughput across your intranet and to the The scope of network investments depend on the category of cloud
Internet. You might also benefit from an ExpressRoute connection. service. Investing across Microsoft’s cloud maximizes the investments of
networking teams. For example, investments for SaaS services apply to
all categories.
Optimize intranet For a high SLA to
connectivity to your edge Microsoft cloud services,
Investment area SaaS PaaS IaaS
network use ExpressRoute
Over the years, many organizations have Although you can utilize your current Architect reliable, redundant Internet
optimized intranet connectivity and Internet connection from your edge connectivity with ample bandwidth
performance to applications running in network, traffic to and from Microsoft
Monitor and tune Internet throughput for
on-premises datacenters. With cloud services must share the pipe with
performance
productivity and IT workloads running in other intranet traffic going to the Internet.
the Microsoft cloud, additional investment Additionally, your traffic to Microsoft Troubleshoot Internet connectivity and
must ensure high connectivity availability cloud services is subject to Internet traffic throughput issues
and that traffic performance between congestion.
Design Azure Traffic Manager to load balance
your edge network and your intranet
For a high SLA and the best performance, traffic to different endpoints
users is optimal.
use ExpressRoute, a dedicated WAN Architect reliable, redundant, and performant
connection between your network and connectivity to Azure virtual networks
Azure, Office 365, Dynamics 365, or all
Optimize throughput at three. Design secure connectivity to Azure virtual
your edge network ExpressRoute can leverage your existing
machines

network provider for a dedicated Design and implement routing between on-
As more of your day-to-day productivity premises locations and virtual networks
traffic travels to the cloud, you should connection. Resources connected by
closely examine the set of systems at your ExpressRoute appear as if they are on your Architect and implement load balancing for
edge network to ensure that they are WAN, even for geographically-distributed internal and Internet-facing IT workloads
current, provide high availability, and have organizations.
Troubleshoot virtual machine connectivity and
sufficient capacity to meet peak loads. ExpressRoute for Office 365 throughput issues
ExpressRoute for Azure

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].
 Microsoft Cloud What IT architects need to know about
networking in Microsoft cloud services
Networking for and platforms

Enterprise Architects
This topic is 2 of 6 in a series 1 2 3 4 5 6

Common elements of Microsoft cloud connectivity

Integrating your networking with the Microsoft cloud provides optimal access to a
broad range of services.

Steps to prepare your network for Microsoft cloud services

On-premises network Internet

1 Analyze your 2 Analyze your 3 Analyze the 1 Analyze the latency between 2 Analyze the capacity and
client computers on-premises capacity and your Internet edge device utilization of your current
and optimize for network for performance of (such as your external Internet connection and add
network hardware, traffic latency your Internet firewall) and the regional capacity if needed.
software drivers, and optimal edge device locations of the Microsoft Alternately, add an
protocol settings, routing to the and optimize cloud service to which you ExpressRoute connection.
and Internet Internet edge for higher levels are connecting.
browsers. device. of traffic.

Microsoft cloud connectivity options

Use your existing Internet pipe or an ExpressRoute connection to
Office 365, Azure, and Dynamics 365.
Components of a typical DMZ

On-premises
Internet Internal firewall: Barrier between your trusted network
network and an untrusted one. Performs traffic filtering (based
on rules) and monitoring.
Microsoft cloud

External workload: Web sites or other workloads made

Users ExpressRoute available to external users on the Internet

Office 365 Azure Dynamics 365

Internet Proxy server: Services requests for web content on
pipe behalf of intranet users. A reverse proxy allows
unsolicited inbound requests.
Microsoft Intune

External firewall: Allows outbound traffic and specified

inbound traffic. Can perform address translation, packet
DMZ inspection, SSL Break and Inspect, or data loss
prevention.

ISP WAN connection to ISP: A carrier-based connection to

an ISP, who peers with the Internet for connectivity and
Internal firewall External Proxy External firewall
workload server
routing.

Areas of networking common to all Microsoft cloud services

Intranet performance Edge devices Internet connection Internet DNS
Performance to Internet-based resources Devices at the edge of your network are Your WAN connection to your ISP and Use A, AAAA, CNAME, MX, PTR and other
will suffer if your intranet, including client egress points and can include Network the Internet should have enough records to locate Microsoft cloud or your
computers, is not optimized. Address Translators (NATs), proxy servers capacity to handle peak loads. services hosted in the cloud. For example,
(including reverse proxies), firewalls, you might need a CNAME record for your
intrusion detection devices, or a You can also use an ExpressRoute app hosted in Azure PaaS.
combination. connection.

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].
 Microsoft Cloud What IT architects need to know about
networking in Microsoft cloud services
Networking for and platforms

Enterprise Architects
This topic is 3 of 6 in a series 1 2 3 4 5 6

ExpressRoute for Microsoft cloud connectivity

ExpressRoute provides a private, dedicated, high-throughput network connection to
Microsoft's cloud.

ExpressRoute to the Microsoft cloud

Without ExpressRoute
With an Internet connection, the only part of the
On-premises traffic path to the Microsoft cloud that you can
Internet Microsoft cloud
network control (and have a relationship with the service
provider) is the link between your on-premises
network edge and your ISP (shown in green).
Users
E
I E The path between your ISP and the Microsoft
d d Office 365 Microsoft Intune cloud edge is a best-effort delivery system
S
Users g g subject to outages, traffic congestion, and
P
e e
monitoring by malicious users (shown in yellow).

Azure Dynamics 365 Users on the Internet, such as roaming or remote

users, send their traffic to the Microsoft cloud
over the Internet.

With ExpressRoute
With an ExpressRoute connection, you now have
On-premises control, through a relationship with your service
Internet Microsoft cloud
network provider, over the entire traffic path from your
edge to the Microsoft cloud edge. This
connection can offer predictable performance
and a 99.9% uptime SLA.
Users
I You can now count on predictable throughput
S Microsoft Intune
and latency, based on your service provider's
P connection, to Office 365, Azure, and Dynamics
E E 365 services. ExpressRoute connections to
d d Microsoft Intune are not supported at this time.
Users
g g
e e
Traffic sent over the ExpressRoute connection is
Office 365 Azure no longer subject to Internet outages, traffic
congestion, and monitoring.

ExpressRoute Users on the Internet, such as roaming or remote

Dynamics 365 users, still send their traffic to the Microsoft cloud
over the Internet. One exception is traffic to an
intranet line of business application hosted in
Azure IaaS, which is sent over the ExpressRoute
Even with an ExpressRoute connection, some traffic is still sent over the Internet, such as DNS connection via a remote access connection to the
queries, certificate revocation list checking, and content delivery network (CDN) requests. on-premises network.

See these additional resources for more information: ExpressRoute for Office 365 ExpressRoute for Azure

Advantages of ExpressRoute for Azure

Predictable performance Data privacy for your traffic High throughput connections Lower cost for some configurations
With a dedicated path to the edge of the Traffic sent over your dedicated With wide support for ExpressRoute Although ExpressRoute connections are
Microsoft cloud, your performance is not ExpressRoute connection is not subject connections by exchange providers and an additional cost, in some cases a single
subject to Internet provider outages and to Internet monitoring or packet capture network service providers, you can obtain ExpressRoute connection can cost less
spikes in Internet traffic. You can and analysis by malicious users. It is as up to a 10 Gbps link to the Microsoft than increasing your Internet capacity at
determine and hold your providers secure as using Multiprotocol Label cloud. multiple locations of your organization to
accountable to a throughput and latency Switching (MPLS)-based WAN links. provide adequate throughput to
SLA to the Microsoft cloud. Microsoft cloud services.

An ExpressRoute connection is not a guarantee of higher performance in every configuration. It is For the latest recommendations for using
possible to have lower performance over a low-bandwidth ExpressRoute connection than a high- ExpressRoute with Office 365, see ExpressRoute for
bandwidth Internet connection that is only a few hops away from a regional Microsoft datacenter. Office 365.

Continued on next page

ExpressRoute connectivity models
Co-located at a cloud exchange Point-to-point Ethernet Any-to-any (IP VPN) connection
Microsoft If your datacenter is co- Microsoft If your datacenter is If you are already using
located in a facility with a located on your Microsoft an IP VPN (MPLS)
cloud exchange, you can premises, you can use Your provider to connect the
order a virtual cross- a point-to-point location 3 sites of your organization,
connection to the Ethernet link to Your an ExpressRoute
Microsoft cloud through connect to the location 2 connection to the
the co-location Microsoft cloud. Microsoft cloud acts like
Your co- provider's Ethernet
Your another location on your
location location private WAN.
exchange. WAN
Your
location 1

ExpressRoute peering relationships to Microsoft cloud services

A single ExpressRoute connection supports up to two different Border Gateway Protocol (BGP) peering relationships to
different parts of the Microsoft cloud. BPG uses peering relationships to establish trust and exchange routing information.

Microsoft SaaS

Office 365 Dynamics 365 Microsoft peering

Microsoft peering
• Is from a router in your DMZ to the public
addresses of Office 365, Dynamics 365,
and Azure services.
Azure PaaS • Supports bidirectional-initiated
Application types: communication.
On-premises
network • Compute • Analytics
• Web and mobile • IoT
ExpressRoute • Data • Media and CDN
• Hybrid integration

Users
Azure IaaS Private peering
• Is from a router on the edge of your
Private peering organization network to the private IP
addresses assigned to your Azure VNets.
• Supports bidirectional-initiated
Gateway Virtual machines communication.
• Is an extension of your organization
Virtual network network to the Microsoft cloud, complete
with internally-consistent addressing and
routing.

Example of application deployment and traffic flow with ExpressRoute

How traffic travels across ExpressRoute connections and within the Microsoft cloud is a function of the routes at the hops of
the path between the source and the destination and application behavior. Here is an example of an application running on
an Azure virtual machine that accesses an on-premises SharePoint farm over a site-to-site VPN connection.

The application locates the IP address of the SharePoint

Azure IaaS farm using the on-premises DNS and all traffic goes over
On-premises network
the site-to-site VPN connection.

Virtual network
Traffic flow

SharePoint farm Site-to-site VPN

Application server
Internet
Gateway
pipe

This organization migrated their on-premises SharePoint farm to

With the Microsoft and private peering relationships:
SharePoint Online in Office 365 and deployed an ExpressRoute
connection.
• From the Azure gateway, on-premises locations are
Microsoft SaaS available across the ExpressRoute connection.
• From the Office 365 subscription, public IP addresses
of edge devices, such as proxy servers, are available
across the ExpressRoute connection.
Office 365 • From the on-premises network edge, the private IP
addresses of the Azure VNet and the public IP
addresses of Office 365 are available across the
On-premises network Azure IaaS ExpressRoute connection.

When the application accesses the URLs of SharePoint

Virtual network Online, it forwards its traffic across the ExpressRoute
connection to a proxy server in the edge.
E Traffic flow
d
g When the proxy server locates the IP address of
e
Application server
SharePoint Online, it forwards the traffic back over the
Gateway ExpressRoute connection. Response traffic travels the
ExpressRoute
reverse path. The result is hair pinning, a consequence of
the routing and application behavior.
Continued on next page
ExpressRoute and Microsoft’s cloud network
With ExpressRoute With ExpressRoute Premium
How traffic travels between your organization network and a Microsoft For organizations that are globally distributed across continents, you can
datacenter is a combination of: use ExpressRoute Premium.

• Your locations. With ExpressRoute Premium, you can reach any Microsoft datacenter on
• Microsoft cloud peering locations (the physical locations to connect to the any continent from any Microsoft peering location on any continent. The
Microsoft edge). traffic between continents is carried over the Microsoft cloud network.
• Microsoft datacenter locations.
With multiple ExpressRoute Premium connections, you can have:
Microsoft datacenter and cloud peering locations are all connected to the
Microsoft cloud network. • Better performance to continentally local Microsoft datacenters.
• Higher availability to the global Microsoft cloud when a local
When you create an ExpressRoute connection to a Microsoft cloud peering ExpressRoute connection becomes unavailable.
location, you are connected to the Microsoft cloud network and all the
Microsoft datacenter locations in the same continent. The traffic between
the cloud peering location and the destination Microsoft datacenter is
carried over the Microsoft cloud network.

This can result in non-optimal delivery to local Microsoft datacenters for the
any-to-any connectivity model. Example of ExpressRoute Premium connections for a
global enterprise using Office 365
In this example, traffic Location 1 Location 2
from the east coast
branch office has to go
WAN
across the country to a
west coast Microsoft Peering
cloud peering location location
and then back across to Microsoft cloud
network Microsoft cloud
the East US Azure Datacenter network
datacenter. Microsoft cloud
network Microsoft cloud
network

For optimal delivery, use

multiple ExpressRoute Location 1 Location 2
Microsoft cloud
connections to regional
network Microsoft cloud
Microsoft cloud peering
locations. network
Peering Peering location
This can provide: location
Microsoft cloud
• Better performance to network
Datacenter With a portion of the Microsoft cloud network in each continent, a global
regionally local Microsoft
enterprise creates ExpressRoute Premium connections from its regional
datacenter locations.
hub offices to local Microsoft peering locations.
• Higher availability to the
Microsoft cloud when a
For a regional office, appropriate Office 365 traffic to:
local ExpressRoute
connection becomes
• Continental Office 365 datacenters travels over the Microsoft cloud
unavailable.
network within the continent.
• Office 365 datacenters in another continent travels over the
intercontinental Microsoft cloud network.
This works well for organizations in the same continent. However, traffic to
Microsoft datacenters outside the organization’s continent travels over the
Internet.

For intercontinental traffic over the Microsoft cloud network, you must use
Network planning and performance tuning for Office 365
ExpressRoute Premium connections.

ExpressRoute options
Security at your edge Internet traffic for VMs WAN optimizers Quality of service
To provide advanced security for the To prevent Azure VMs from initiating You can deploy WAN optimizers on both Use Differentiated Services Code Point
traffic sent and received over the traffic directly with Internet locations, sides of a private peering connection for a (DSCP) values in the IPv4 header of your
ExpressRoute connection, such as traffic advertise the default route to Microsoft. cross-premises Azure virtual network traffic to mark it for voice, video/
inspection or intrusion/malware Traffic to the Internet is routed across (VNet). Inside the Azure VNet, use a WAN interactive, or best-effort delivery. This is
detection, place your security appliances the ExpressRoute connection and optimizer network appliance from the especially important for the Microsoft
in the traffic path within your DMZ or at through your on-premises proxy Azure marketplace and user-defined peering relationship and Skype for
the border of your intranet. servers. Traffic from Azure VMs to Azure routing to route the traffic through the Business Online traffic.
PaaS services or Office 365 is routed appliance.
back across the ExpressRoute
connection.

More ExpressRoute for Office 365 ExpressRoute for Azure

information http://aka.ms/expressrouteoffice365 https://azure.microsoft.com/services/expressroute/

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].
 Microsoft Cloud What IT architects need to know about
networking in Microsoft cloud services
Networking for and platforms

Enterprise Architects
This topic is 4 of 6 in a series 1 2 3 4 5 6

Designing networking for Microsoft SaaS (Office 365,

Microsoft Intune, and Dynamics 365)
Optimizing your network for Microsoft SaaS services requires the configuration of internal and
edge devices to route the different categories of traffic to Microsoft SaaS services.

Steps to prepare your network for Microsoft SaaS services

1 Go through the Steps to 2 Add an Internet connection 3 Ensure that the ISPs for 4 Examine your network 5 Configure your edge devices
prepare your network for to each of your offices. all Internet connections hairpins, intermediate to bypass processing for the
Microsoft cloud services use a DNS server with a destinations such as cloud- Optimize and Allow
in topic 2 of this model. local IP address. based security services, and categories of Microsoft SaaS
eliminate them if possible. traffic.

Optimizing traffic to Microsoft’s SaaS services

Optimize Allow Default
Three categories Required for connectivity to every Required for connectivity to specific Represent Microsoft SaaS services and
of Microsoft SaaS Microsoft SaaS service and represent Microsoft SaaS services and features but dependencies that do not require any
over 75% of Microsoft SaaS bandwidth, are not as sensitive to network optimization. You can treat Default
traffic: connections, and volume of data. performance and latency as those in the category traffic like normal Internet traffic.
Optimize category.

This is the recommended configuration of all offices, including

branch offices and regional or central ones, in which:

• Default category and general Internet traffic is routed to

offices that have proxy servers and other edge devices to
Central office Internet provide protection against Internet-based security risks.
• Optimize and Allow category traffic is forwarded directly
to the edge of the Microsoft network front end nearest to
the office containing the connecting user, bypassing proxy
servers and other edge devices.

Proxy External ISP

Web server
Users server firewall Nearest Microsoft front end
SD-WAN
device

E
Office 365 Microsoft Intune
d
g
Branch offices e

Azure Dynamics 365

Software-defined wide area network (SD-WAN) devices

in branch offices separate traffic so that:

• Default category and general Internet traffic goes

SD-WAN to a central or regional office over the WAN
Users device
backbone.
ISP • Optimize and Allow category traffic goes to the ISP
providing the local Internet connection.

Default category and general Internet traffic

Optimize and Allow category traffic

Network connectivity principles Network planning and

More performance tuning for Office 365
http://aka.ms/pnc
information http://aka.ms/tune

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].
 Microsoft Cloud What IT architects need to know about
networking in Microsoft cloud services
Networking for and platforms

Enterprise Architects
This topic is 5 of 6 in a series 1 2 3 4 5 6

Designing networking for Azure PaaS

Optimizing networking for Azure PaaS apps requires adequate Internet bandwidth
and can require the distribution of network traffic across multiple sites or apps.

Planning steps for hosting organization PaaS applications in Azure

1 Go through the Steps to 2 Optimize your Internet 3 Determine whether you 4 For web-based workloads, 5 For distribution of traffic
prepare your network bandwidth using steps need an ExpressRoute determine whether you to different endpoints in
for Microsoft cloud 2 – 4 of the Steps to connection to Azure. need the Azure different data centers,
services in topic 2 of this prepare your network for Application Gateway. determine whether you
model. Microsoft SaaS services need Azure Traffic
in topic 4 of this model. Manager.

Internet bandwidth for organization PaaS applications

Organization applications hosted in Azure PaaS require Internet bandwidth for intranet users.

Option 1 Use your existing pipe, optimized

for Internet traffic with the capacity to On-premises
Azure PaaS
handle peak loads. See page 4 of this model network
for Internet edge, client usage, and IT Application types:
operations considerations. • Analytics
Internet • Compute • IoT
Option 2 For high-bandwidth or low
Users pipe • Web and mobile • Media and CDN
latency needs, use an ExpressRoute ExpressRoute • Data • Hybrid integration
connection to Azure.

Azure Application Gateway Microsoft Azure

Application-level routing and load balancing services that let you build a
scalable and highly-available web front end in Azure for web apps, cloud
services, and virtual machines. Application Gateway currently supports layer
7 application delivery for the following: Web app
• HTTP load balancing
• Cookie based session affinity
Application
• SSL offload Users Gateway
Cloud Service

Application Gateway
Virtual machine

Azure Traffic Manager Example for three geographically-

distributed web apps Microsoft Azure
Distribution of traffic to different endpoints, which can include cloud services
or Azure web apps located in different data centers or external endpoints.
1 Traffic Manager

Traffic Manager routing methods Web app East US

Failover The endpoints are in the same or different Azure datacenters Web app West Europe
2
and you want to use a primary endpoint for all traffic, but provide
Users
backups in case the primary or the backup endpoints are unavailable.
Web app East Asia
Round robin You want to distribute load across a set of endpoints in
the same datacenter or across different datacenters.
1. A user DNS query for a web site URL gets directed to Azure Traffic
Performance You have endpoints in different geographic locations and Manager, which returns the name of a regional web app, based on the
you want requesting clients to use the "closest" endpoint in terms of the performance routing method.
lowest latency.
2. User initiates traffic with the regional web app. Traffic Manager

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].
 Microsoft Cloud What IT architects need to know about
networking in Microsoft cloud services
Networking for and platforms

Enterprise Architects
This topic is 6 of 6 in a series 1 2 3 4 5 6

Designing networking for Azure IaaS

Optimizing networking for IT workloads hosted in Azure IaaS requires an understanding
of Azure virtual networks (VNets), address spaces, routing, DNS, and load balancing.

Planning steps for hosting an IT workload in an Azure VNet

Planning for any VNet

Prepare your Optimize your Determine the Determine the Determine the Determine the
1 2 3 4 5 6
intranet for Internet type of VNet address space of subnets within DNS server
Microsoft cloud bandwidth. (cloud-only or the VNet. the VNet and the configuration and
services. cross-premises). address spaces the addresses of
assigned to each. the DNS servers to
assign to VMs in
the VNet.

Determine the 8 Determine the Determine how 10 For multiple

7 load balancing use of virtual 9 computers from VNets, determine
configuration appliances and the Internet will the VNet-to-
(Internet-facing user-defined connect to virtual VNet connection
or internal). routes. machines. topology.

Planning for cross-premises VNets

Determine the Determine the Add routes to For Determine the Configure on- Determine the
1 2 3 4 5 6 premises DNS 7 use of forced
on-premises on-premises make the ExpressRoute, Local Network
connection to VPN device or address space plan for the address space servers for DNS tunneling and
the VNet (S2S router. of the VNet new for the Azure replication with user-defined
VPN or reachable. connection gateway. DNS servers routes.
ExpressRoute). with your hosted in
provider. Azure.

Planning steps for any Azure VNet

Step 1: Prepare your intranet for Microsoft cloud services.
Go through the Steps to prepare your network for Microsoft cloud services in
topic 2 of this model.

Step 2: Optimize your Internet bandwidth.

Go through steps 2 – 4 of the Steps to prepare your network for Microsoft SaaS
services in topic 4 of this model.

Step 3: Determine the type of VNet (cloud-only or cross-premises).

Cloud only Cross-premises
A VNet with no connection to an A VNet with a Site-to-Site (S2S) VPN or ExpressRoute connection to an on-premises
on-premises network. network through an Azure gateway.

Virtual network On-premises network Virtual network

S2S VPN

Virtual machines Users Gateway Virtual machines

ExpressRoute

Continued on next page See the additional Planning steps for a cross-premises Azure VNet in this topic.
Step 4: Determine the address space of the VNet.
Addressing for virtual networks Addressing for virtual machines

Virtual machines are assigned an address configuration from the address

Type of VNet Virtual network address space space of the subnet by DHCP:
• Address/subnet mask
Cloud only Arbitrary private address space • Default gateway
• DNS server IP addresses
Interconnected Arbitrary private, but not overlapping with
cloud-only other connected VNets You can also reserve a static IP address.
Virtual machines can also be assigned a public IP address, either individually
Cross-premises Private, but not overlapping with on-premises or from the containing cloud service (for classic deployment machines only).

Interconnected Private, but not overlapping with on-premises

cross-premises and other connected VNets

Step 5: Determine the subnets within the VNet and the address spaces assigned to each.

Azure gateway subnet

Virtual network
Needed by Azure to host the two virtual machines
of your Azure gateway. Specify an address space
with at least a 29-bit prefix length (example: Gateway subnet Subnet Subnet Subnet
192.168.15.248/29). A 27-bit prefix length is
recommended, especially if you are planning to
use ExpressRoute.
Gateway Virtual Virtual Virtual
machines machines machines

Best practice for determining the address space

of the Azure gateway subnet: Example of defining the address prefix for the gateway subnet
1. Decide on the size of the gateway subnet. The address space of the VNet is 10.119.0.0/16. The organization will initially use a site-to-site
2. In the variable bits in the address space of VPN connection, but will eventually get ExpressRoute.
the VNet, set the bits used for the gateway
subnet to 0 and set the remaining bits to 1.
Step Results
3. Convert to decimal and express as an address
space with the prefix length set to the size of
the gateway subnet. 1. Decide on the size of the gateway subnet. /28

With this method, the address space for the 10.119. bbbbbbbb . bbbbbbbb
gateway subnet is always at the farthest end of 2. Set the bits in the variable portion of the
the VNet address space. VNet address space: 0 for the gateway
10.119. VVVVVVVV . VVVVGGGG
subnet bits (G), otherwise 1 (V). 10.119. 11111111 . 11110000
Address space calculator for Azure gateway subnets
3. Convert result from step 2 to decimal and
10.119.255.240/28
express as an address space.

Virtual machine-hosting subnets

Virtual machines Host bits Subnet size
Place Azure virtual machines in subnets according to typical on-premises
guidelines, such as a common role or tier of an application or for subnet
isolation. 1-3 3 /29

Azure uses the first 3 addresses on each subnet. Therefore, the number of 4-11 4 /28
possible addresses on an Azure subnet is 2 n – 5, where n is the number of
host bits. 12-27 5 /27

Networking limits 28-59 6 /26

60-123 7 /25

Step 6: Determine the DNS server configuration and the addresses of the DNS servers to assign to
VMs in the VNet.
Azure assigns virtual machines the addresses of DNS servers by DHCP. DNS
Type of VNet DNS server
servers can be:
• Supplied by Azure: Provides local name registration and local and Internet Azure-supplied for local and Internet name
name resolution resolution
• Provided by you: Provides local or intranet name registration and either Cloud only
intranet or Internet name resolution Azure virtual machine for local and Internet name
resolution (DNS forwarding)
Name Resolution for resources in Azure virtual networks
On-premises for local and intranet name
resolution
Cross-premises
Azure virtual machine for local and intranet name
resolution (DNS replication and forwarding)

Continued on next page

Step 7: Determine the load balancing configuration (Internet-facing or internal).

Internet-facing load Virtual network Internal load Virtual network

balancing balancing
Randomly distribute Randomly distribute
Load balanced set Load balanced set
unsolicited incoming unsolicited incoming
traffic from the Internet traffic from other
to the members of a Azure VMs or from
load-balanced set. intranet computers Rule for
Virtual Virtual Inbound Virtual
(not shown) to the
machine machine traffic machine
members of a load-
Azure Load Balancer
Inbound balanced set.
NAT rule or Azure load Azure load
endpoint balancer balancer
Virtual Virtual Virtual
machine machine machine

Step 8: Determine the use of virtual appliances and user-defined routes.

User-defined routing On-premises
Virtual network
You may need to add one or more user-defined network
routes to a subnet to forward traffic to virtual
appliances in your Azure virtual network. Subnet Subnet

User-defined
S2S VPN route
Virtual network traffic routing
Virtual Virtual
VPN device Gateway appliance machines
ExpressRoute

Step 9: Determine how computers from the Internet will connect to virtual machines.
Includes access from your organization network through your proxy server or other edge device.
Virtual network
Methods for filtering or inspecting unsolicited incoming traffic

Method Deployment model Cloud Service

Endpoint
1. Endpoints and ACLs configured on cloud services Classic 1
Virtual machine
2. Network security groups Resource Manager and classic
Network security group
3. Internet-facing load balancer with inbound NAT rules Resource Manager
2
4. Network security appliances in the Azure
Resource Manager and classic Virtual machine
Marketplace (not shown)

Additional security: Load balanced set

• Remote Desktop and SSH connections are authenticated and encrypted
• Remote PowerShell sessions are authenticated and encrypted 3
• You can use IPsec transport mode for end-to-end encryption Inbound
Azure load Virtual machine
NAT rules
• Azure DDOS protection helps prevent external and internal attacks balancer

Microsoft Cloud Security for Enterprise Architects

Step 10: For multiple VNets, determine the VNet-to-VNet connection topology.
Azure VNets can be connected to each other using topologies similar to those used for connecting
the sites of an organization using VNet peering or VNet-to-VNet (V2V) connections.
Virtual network peering
Daisy chain

Virtual Network Virtual Network Virtual Network Virtual Network Virtual Network

Spoke and hub Full mesh

Virtual Network Virtual Network Virtual Network Virtual Network

Virtual Network Virtual Network Virtual Network Virtual Network Virtual Network Virtual Network

Continued on next page

 Planning steps for a cross-premises Azure VNet Simulated cross-premises virtual network in Azure

Step 1: Determine the cross-premises connection to the VNet (S2S VPN or ExpressRoute).

Site-to-Site (S2S) VPN Connect 1–10 sites (including other VNets) to a single Azure VNet.

ExpressRoute A private, secure link to Azure via an Internet Exchange Provider (IXP) or a Network Service Provider (NSP).

Other types of connections:

Point-to-Site (P2S) VPN Connects a single computer to an Azure VNet.

VNet peering or VNet-to-

Connects an Azure VNet to another Azure VNet.
VNet (V2V) VPN

Networking limits VPN devices and IPsec/IKE parameters for site-to-site VPN gateway connections Virtual network peering

Connecting to VMs in the VNet:

On-premises network Virtual network
• Administration of VNet VMs from your on-
premises network or the Internet
P2S
• IT workload access from your on-premises
network
• Extension of your network through additional S2S or
Administrator Gateway Virtual machines
Azure VNets

Security for connections: V2V VNet peering

Virtual Network
• P2S uses the Secure Socket Tunneling Protocol
(SSTP) Virtual Network
• S2S and V2V VPN connections use IPsec tunnel
mode with AES256 Microsoft Cloud Security for Enterprise Architects
• ExpressRoute is a private WAN connection

Step 2: Determine the on-premises VPN device or router.

Your on-premises VPN device or router:
On-premises network Virtual network
• Acts as an IPsec peer, terminating the S2S
VPN connection from the Azure gateway.
• Acts as the BPG peer and termination S2S VPN
point for the private peering ExpressRoute
connection. VPN device Gateway Virtual machines
ExpressRoute
VPN gateways

Step 3: Add routes to your intranet to make the address space of the VNet reachable.

Routing to VNets from on-premises

On-premises network Virtual network
1. Route for the virtual network address space that
Virtual network
points toward your VPN device address space
Virtual network 2
2. Route for the virtual network address space on
address space
your VPN device 1
VPN device S2S or ExpressRoute
Gateway

Step 4: For ExpressRoute, plan for the new connection with your provider.

You can create an ExpressRoute connection with

On-premises
private peering between your on-premises Microsoft Azure
network
network and the Microsoft cloud in three different
ways:
ExpressRoute Virtual network
• Co-located at a cloud exchange
Router
• Point-to-point Ethernet connections Virtual machines

• Any-to-any (IP VPN) networks

See topic 3, ExpressRoute.

Step 5: Determine the Local Network address space for the Azure gateway.

Routing to on-premises or other VNets

from VNets On-premises network Virtual network
Local Network
Azure forwards traffic across an Azure gateway address space
that matches the Local Network address space
assigned to the gateway.
S2S VPN or
VPN device ExpressRoute Gateway
Continued on next page
Defining the Local Network address space:
Option 1: The list of prefixes for the address Example of defining the prefixes for the Local Network around the address space
space currently needed or in use (updates “hole” created by the virtual network for S2S VPN connections
might be needed when you add new subnets).
An organization uses portions of the private address space (10.0.0.0/8, 172.16.0.0/12, and
Option 2: Your entire on-premises address 192.168.0.0/16) across their on-premises network. They chose option 2 and 10.100.100.0/24 as
space (updates only needed when you add their virtual network address space.
new address space).

Step Prefixes
Because the Azure gateway does not allow
summarized routes for S2S VPN connections, you
1. List the prefixes that are not the root space
must define the Local Network address space for 172.16.0.0/12 and 192.168.0.0/16
for the virtual network address space.
option 2 so that it does not include the virtual
network address space. 10.0.0.0/16, 10.1.0.0/16…
2. List the non-overlapping prefixes for variable
10.99.0.0/16, 10.101.0.0/16…
octets up to but not including the last used
10.254.0.0/16, 10.255.0.0/16
octet in the virtual network address space. (255 prefixes, skipping 10.100.0.0/16)

10.100.0.0/24, 10.100.1.0/24…
3. List the non-overlapping prefixes within the
10.100.99.0/24, 10.100.101.0/24…
last used octet of the virtual network address
10.100.254.0/24, 10.100.0.255.0/24
space. (255 prefixes, skipping 10.100.100.0/24)
The virtual network address space
The root space

Step 6: Configure on-premises DNS servers for replication with DNS servers hosted in Azure.

To ensure that on-premises computers On-premises

can resolve the names of Azure-based Virtual network
network
servers and Azure-based servers can
resolve the names of on-premises
Subnet
computers, configure:
DNS replication and forwarding
• The DNS servers in your virtual
DNS server Subnet DNS server
network to forward to on-premises
DNS servers.
• DNS replication of the appropriate S2S VPN or
zones between DNS servers on-
Virtual
premises and in the Azure VNet. VPN device Gateway machines
ExpressRoute

Step 7: Determine the use of forced tunneling.

The default system route for Azure subnets points

On-premises
to the Internet. To ensure that all traffic from Virtual network
virtual machines travels across the cross-premises
network
connection, create a routing table with the
default route that uses the Azure gateway as its Subnet
next-hop address. You then associate the route
table with the subnet. S2S VPN or Default route
This is known as forced tunneling.
VPN device Virtual
Gateway
ExpressRoute machines
Configure forced tunneling

SharePoint Server 2016 farm in Azure

A highly-available, multi-tier SharePoint Server 2016 farm is
an example of an intranet IT workload hosted in Azure IaaS. Virtual network

On-premises
network
WEB1 APP1 SQL1 DC1
TCP 443
MN1
AD
WEB2 APP2 SQL2 DC2
Site-to-site
VPN device Gateway
VPN or
Subnet Subnet Subnet Subnet
ExpressRoute

SharePoint Server 2016 in Microsoft Azure Intranet SharePoint Server 2016 in Azure dev/test environment

More Microsoft Security Hybrid

cloud IT resources aka.ms/cloudarchsecurity aka.ms/cloudarchhybrid

November 2019 © 2016 Microsoft Corporation. All rights reserved. To send feedback about this documentation, please write to us at CloudAdop [email protected].