P21 WWUG Virtual Connect 2020 8/20/2020

HIGH AVAILABILITY & DISASTER

RECOVERY FUNDAMENTALS
Presented by: David Klee

Prophet 21 World Wide User

Group Officers
President
Sam Snow Vice President of Operations
T. J. Snow Company, Inc. Ted Hoffman
Utility Supply and Construction
Vice President of Finance
Tim Edmunds
TSI Solutions Vice President of Member Relations
Eric Lunsford
Vice President of Marketing and Education Pye-Barker Engineered Solutions
Mike Chadwick
T. J. Snow Company, Inc.

CONNECT 2020 2

Do not distribute as your own outside of your 
organization 1
P21 WWUG Virtual Connect 2020 8/20/2020

DISCLAIMER
• This conference is an attempt by P21WWUG members to assist each other
by demonstrating ways that we utilize the Prophet21 system and other related
products.
• The P21WWUG and the individuals conducting the classes and round tables
take no responsibility for potential issues that arise as a result of taking the
advice given during the conference.
• The P21WWUG does not recommend using any SQL statements to update
your database without having those statements first reviewed by Epicor or
other experienced SQL professionals. Test any code in your Play Database!
• Using SQL statements to update your database may result in corrupting your
database. Test any code in your Play Database!

CONNECT 2020 3

David Klee
Founder, Heraflux Technologies
[email protected]
kleegeek
davidaklee
hfxte.ch/youtube

CONNECT 2020 4

Do not distribute as your own outside of your 
organization 2
P21 WWUG Virtual Connect 2020 8/20/2020

1. Introduction/Bio
2. Start thinking about BC

Agenda
3. Service Level Agreements
4. System Dependencies
5. High Availability
6. Disaster Recovery
7. Cloud
8. Q&A

CONNECT 2020 5

 David Klee
 Founder & Chief Architect
Heraflux Technologies
 SQL Server & Infrastructure Consultant
 Performance
 High Availability / Disaster Recovery
 Cloud / Virtualization / Enterprise Architecture

CONNECT 2020 6

Do not distribute as your own outside of your 
organization 3
P21 WWUG Virtual Connect 2020 8/20/2020

May 2019

…from my driveway…

CONNECT 2020 7

“Stuff” Happens

 Business depends on biz continuity planning

 Planning - failure scope:
 Individual component
 Entire server
 Datacenter rack
 Entire datacenter
 Entire geo-location
 Location might be up, but you can’t physically get there
 2020: COVID-19, protests
 2019: train derailment carrying nuclear waste 0.1 miles from main warehouse

Do not distribute as your own outside of your 
organization 4
P21 WWUG Virtual Connect 2020 8/20/2020

Definitions

 High Availability (HA)

 Smaller in scale
 Usually isolated to single component failure
 Disaster Recovery (DR)
 Larger in scale
 Usually site failure, primary platform component failure
 What are your criteria for declaring a disaster?
 WHO declares a disaster?
 HA != DR

what are your

AVAILABILITY
requirements

10

10

Do not distribute as your own outside of your 
organization 5
P21 WWUG Virtual Connect 2020 8/20/2020

Realistic SLAs

 Granularity Matters
 Need planned outage windows
 Two separate metrics (planned versus unplanned)
 Unwritten expectations
 What constitutes an issue or disaster?

11

11

skip the tech

Recovery Point Objective
How much data can we lose?
RPO
Recovery Time Objective
How long can this be down?
RTO
12

12

Do not distribute as your own outside of your 
organization 6
P21 WWUG Virtual Connect 2020 8/20/2020

Service Level Agreements

12:45 PM 1:12 PM 3:55 PM

RPO = 15 min Recovery Goals

RTO = 1 hour

Tranlog backups
every 30 mins RPO
RTO (1h)
on the :15/45 (15m)
of hour
SLA SLA
Breach Breach

Time

Observed Data Loss Recovery Window

13

13

SLAs – Be Careful

Nines Downtime Per Year

99% 3.65 days
99.9% 8.77 hours
99.99% 52.56 minutes
99.999% 5.28 minutes
99.9999% 36.53 seconds

14

14

Do not distribute as your own outside of your 
organization 7
P21 WWUG Virtual Connect 2020 8/20/2020

Signatures

 Present budget based on business expectations

 Business must sign off on – physically
 Without it, expectations != budget
 Therefore, expectations != architecture
 Set expectations!

15

15

Technology Components

16

16

Do not distribute as your own outside of your 
organization 8
P21 WWUG Virtual Connect 2020 8/20/2020

Group Components

 Dependencies matter
 Example:
 SQL Server database
 P21 RDS / Citrix server
 P21 web server
 File server
 Active Directory server

17

17

P21 Logical Components

Active Remote Prophet

Active
Directory
Remote
Desktop 2017 Web File SQL
Remote Prophet
Directory
Controllers Desktop
Servers / App Server Server
Desktop Web App
Controllers Servers
Citrix / Server(s)
Servers Server(s)
Citrix /
Citrix

Backup Virtualization
Virtualization Desktops
App Central Email
Cluster / Laptops
Server Controller

18

18

Do not distribute as your own outside of your 
organization 9
P21 WWUG Virtual Connect 2020 8/20/2020

P21 Physical Components

Backup
VM Host Data Network
VM Host Target Network
Servers
VM Host Storage Switches
Network
Servers Storage Switches
Servers Switches

Core
Internet Phone
Routers /
Connectivity System
Firewalls
19

19

High Availability
Create HA when no options existed
Reduce risk window

20

20

Do not distribute as your own outside of your 
organization 10
P21 WWUG Virtual Connect 2020 8/20/2020

Virtualization HA

(Average failover time: 2m 45s)

21

21

App-Level High Availability

 Redundant distributed data

 Domain controllers & AD replication
 Multiple active file servers using Windows DFS
 SQL Server
 Email in the cloud

22

22

Do not distribute as your own outside of your 
organization 11
P21 WWUG Virtual Connect 2020 8/20/2020

Fault Resiliency

Requirements Suggestions
 Virtualization  Secondary storage
 Replicating authentication  Not same make/model

 Redundant app servers  VM placement

 Redundant user access  Offload backups

 VM replication

23

23

Disaster Recovery

24

24

Do not distribute as your own outside of your 
organization 12
P21 WWUG Virtual Connect 2020 8/20/2020

VM Replication

VM

R
E
A Async
Replication
D Replication
Receiver
S Queue ASYNC STREAM
WRITES

Virtual
Disk Virtual
Disk 25
Primary Datacenter DR Datacenter

25

VM Replication

 VM-level backup set replication  Replicate to:

 Veeam, Rubrik, many others  Secondary datacenter
 In-line VM change block replication  Cloud cold storage
 Zerto  Apply and keep destination hot?
 SAN LUN replication  Depends on SLAs, $$

 VM-level backup directly to cloud

 Clumio

26

26

Do not distribute as your own outside of your 
organization 13
P21 WWUG Virtual Connect 2020 8/20/2020

Database HA

 SQL Server mirroring (deprecated)

 SQL Server Failover Clustered Instances (FCIs)
 Availability Groups (2012+ EE)
 (Basic) Availability Groups (2016 SE)
 Reasons for separate strategy

27

27

Database DR

 SQL Server mirroring (deprecated)

 Availability Groups (2012+ EE)
 (Basic) Availability Groups (2016+ SE)
 Log shipping

28

28

Do not distribute as your own outside of your 
organization 14
P21 WWUG Virtual Connect 2020 8/20/2020

Public Cloud

 Cloud DR is MUCH easier than on-prem

 Still requires testing and validation
 Costs for storage

CONNECT 2020 29

29

Hybrid Cloud

 On-premises HA
 Replicate to cloud
 Promote replicated VMs to live for failover

VM Backup
Tool File server
Cloud file
share
storage
SQL SQL
Server Server
30

30

Do not distribute as your own outside of your 
organization 15
P21 WWUG Virtual Connect 2020 8/20/2020

Testing

 Runbook & test plan

 Must have ability to do test simulations
 MUST actually perform a failover and failback
 Quarterly
 Yearly at least
 Per component
 HA + DR separately
 FAILBACK in addition to FAILOVER

31

31

Put It All Together

 Many options for your toolbox

 No excuses for your management
 HA & DR are a must
 Full failover & failback testing
 Technology choices for SLAs
 Not other way around

32

32

Do not distribute as your own outside of your 
organization 16
P21 WWUG Virtual Connect 2020 8/20/2020

Q&A

33

Next Step!
Free SQL Server on VMware or Hyper-V
for Epicor P21 Build Reference Guide
Setup, performance, maintenance, and efficiency tips and tricks

Available from Heraflux at

https://www.heraflux.com/sqlvmguide/

CONNECT 2020 34

34

Do not distribute as your own outside of your 
organization 17
P21 WWUG Virtual Connect 2020 8/20/2020

David Klee
Founder, Heraflux Technologies
[email protected]
kleegeek
davidaklee
hfxte.ch/youtube

CONNECT 2020 35

35

Do not distribute as your own outside of your 
organization 18