Clone and Upgrade By Dinesh Kumar Raghu

Clone and Upgrade

Dinesh Kumar Raghu , Architect

How does the clone work?

• In response to a clone request, the ServiceNow platform performs the following tasks:
1. Generates a file to preserve operational data on the target server. This file contains the data preserved by data preservers.
2. Creates a new database for the target instance.
3. Restores the database schema and data from the latest backup to the newly created database.
4. Begins the exclude/preserver functions (after all data is copied across).
5. If Exclude options are ticked, truncate tables in the exclude module (All data is cleared out unless this is TPH table where it will be
deleted).
6. Load the file created in Step 1 for preservers into the target database (overwriting any data from the restore).
7. Email functionality is disabled in the target database.
8. Once all is completed the target instance repoints to the newly created database.
9. All Clone cleanup scripts are executed in the target instance.
1. Regenerate Text indexes
2. Clear scheduled job node associations
10.The clone is complete and the instance is accessible for the customer.
11.After the clone completes, a backup is taken of the old Target database
12.Once the backup completes, the old target database is retired.
Clone and Upgrade By Dinesh Kumar Raghu
What do all the options do for cloning?
Field Description
Exclude tables specified in Prevents cloning records from tables on the source instance which are listed under System Clone > Exclude Tables. If a table is on the Exclusion List, clone will
Exclusion List exclude the records on the table as well as records on the child tables.
When excluding tables, their table schema and hierarchy will still be cloned to your target instance. As a result, your target instance will have empty but usable
tables after the clone.
A few important notes:
ServiceNow out-of-the-box table exclusions will still be excluded and are not affected by this setting. These include tables containing auditing, license usage,
logging, and notifications.
You can choose to disable this setting if you need the data from your excluded tables. This is called a ‘Full Clone’ and is recommended for upgrade or deployment
testing.
The legacy clone engine does not support this option.
The default setting is that tables specified in the Exclusion List are excluded from a clone.
Exclude audit and log data Prevents the cloning of audit and log records from the source instance. This will create empty but usable audit and log tables on the target instance.
Note: If you exclude audit and log data from your clone, the Activity Stream for records will not match the source instance. This is because the Activity Stream
relies on the audit table to generate the history.
The default setting is that audit and log data are excluded from a clone.

Exclude attachment data Prevents the cloning of certain files (incl. videos, images, documents, zip files) from the sys_attachment table. Please see exceptions below.

Note: ServiceNow out-of-the-box attachments and other system-relevant files (e.g. catalog item images, theming images, icons) are not affected by this setting.
The following ServiceNow out-of-the-box attachments and other system-relevant files are not affected by this setting and will not be excluded from a clone:
•Table name values that start with ZZ_
•Table names: sys_certificate, sc_cat_item , sys_upgrade_manifest , ecc_agent_jar, ecc_agent_mib , sys_store_app , or invisible.sys_store_app
This option is on by default.

What do all the options do for cloning?

Preserve theme Preserves the theme and CSS elements on the target instance. As a result, your target instance will keep its theme, its look & feel after a clone.
The default setting is to preserve the theme on the target instance.

Lock settings for this clone If you use a clone profile, this option locks the settings and options at the time of the clone request. Any subsequent changes to the clone profile, regardless of
request when the clone runs, do not affect the clone request.
This option is not selected by default.

Amount of data copied from Limits the number of days of historical data for the Task table and its child tables (e.g. Incident, Problem, Change) to 90 days.
the Task tables Important Note: Selecting 'Last 90 days' can increase clone time, due to the post-clone deletion process of records older than 90 days.
To reduce clone time, consider excluding large tables altogether. When excluding tables, your target instance will have the same table schema and hierarchy (i.e.
empty usable tables) as the source instance.
The default setting is to clone all data from the Task table and its child tables to the target instance.

Preserve In Progress Update Preserves the last 90 days of in-progress update sets in the global application scope. This option allows you to keep in-progress, global update sets created within
Sets the last 90 days on your target instance.
Note:
This option does not preserve your in-progress scoped update sets.
Keep in mind that update sets that are older than 90 days will not be saved.
As a best practice, we recommend reviewing and committing your update sets before cloning.
The default is to not preserve update sets.
This option allows you to schedule recurring clones from your source to your target instance. It allows you to define the clone frequency and the maximum number
Clone frequency of occurrences.
By default, the clone frequency is set to None.
For more information about scheduling cloning, see the product documentation: Schedule cloning.
Clone and Upgrade By Dinesh Kumar Raghu
How can I do a full clone?

• A full clone is when you bring over all the information from your source instance and you do not exclude any data from
source. This is normally done when you want a full replica of your prod instance with logs, emails, etc. This is helpful when
doing upgrading, updating sets, application deployments, etc. to see how it would run on your production instance.
• To request a full clone, please toggle these options:
• Exclude audit and log data - Unchecked
• Exclude large attachment data - Unchecked
• Exclude tables specified in Exclusion List - Unchecked
• Preserve Theme - Checked
• Amount of data copied from the Task table - Full
• Note: Emails are disabled during the clone, so even though emails were copied across they will not send after a full clone.

How can I exclude and Preserve Users, Roles,

and Groups?
• In some situations, you would like to not bring over the users from the source instance and only preserve the target instance's users. There
are many different reasons for this and there are a number of ways you can achieve this.
• Method 1 (Recommended): Preserve access and users in target without excluding.
• You can preserve the access in a target instance with a simple group and preserver logic. You will not need any exclusions with this method
(excluding sys_user or sys_user_role can cause broken references).
• Create a user group (e.g. “developers”) that contains all the users that should still have access after cloning
• Clone Source over target
• Disable all user accounts that should not have access after a clone
• Create a cleanup script to check all users that have access and are not part of the “developers” group, if found then disable
• If you need to preserve further roles/groups for the target instance you can add them in preservers
• The cleanup script will remove any new users that are copied from production but do not yet have a record on the target instance. It will find
any users that are not linked to your “developers” group and set them to inactive. Future clones will now have the value as false and it will
always keep working with adding/removing users.
Clone and Upgrade By Dinesh Kumar Raghu
How can I exclude and Preserve Users, Roles,
and Groups?
• Method 2 (Recommended): (Use with caution – can break related tables and instance functionality)
• You can set up data excludes and preservers.
• This is a trickier solution and means that when implemented that if you ever clone with the option to exclude tables unchecked, this will not
work and will break future clones using this method.
• To do this you need to create the following Excludes:
• sys_user
• sys_user_role**
• sys_user_has_role
• sys_user_group
• sys_user_grmember
• sys_group_has_role
• sys_user_role_contains
• cmn_notif_device
• customer_contact (only if you have the Customer Service Management plugin installed)
• This will not bring over any users from the source instance or any of the references to groups and roles, but now you need to
preserve the sys_user in the target instance. Something that many people forget is you also need to preserve roles, groups, and also
the links from users to roles and groups. This is a common mistake, and after cloning you are missing the admin role and cannot log
in, this is because the links to the roles are missing.

How can I exclude and Preserve Users, Roles,

and Groups?
• Now you should create these Preservers (add any conditions depending on your requirements):
• sys_user
• sys_user_role**
• sys_user_group
• sys_user_grmember
• sys_user_has_role
• sys_group_has_role
• sys_user_role_contains
• cmn_notif_device
• customer_contact (only if you have the Customer Service Management plugin installed)
• This will preserve all the users, roles, and groups in the target instance, but it will also preserve the permissions of those users to
those groups and roles.
• If you place these in your instance, after a clone you will have all the same users from your target instance and will still have all the
access from before the clone.
• **Treat the sys_user_role table with caution.
Clone and Upgrade By Dinesh Kumar Raghu
How can I exclude and Preserve Users, Roles,
and Groups?
• If you exclude/preserve the sys_user_role table; plugin-related roles can be excluded/preserved. The role can be missing on the target with
the plugin being in active status if the plugin is in active state on the source and NOT active on the target.
• If a plugin is NOT active on the source and active state on the target, excluding/preserving the sys_user_role will create broken references
as the roles will be available on the target without the plugin being active.
• The sys_user_role table has a unique index on name which makes it impossible to have duplicate role names. So therefore, for records
where index collision happens between source & target, target record will be preserved & source will be excluded. (Recommended to only
preserve roles in target that are not in source, as this can break all references to the source role sys_id)
•
For Example:
If I preserved the role: TestRole
Source: TestRole - sys_id = 77c8e0a61b88d10739e2f48b04bcbdc
Target: TestRole - sys_id = 32f4f3t23b42a23425e1f63g12efase

After the clone we would only have the role from the Target instance, we will not have the role from source as we can't have both with the
unique index.

What tables should I not exclude?

• Upon looking at issues raised through Support we have seen a number of tables that have been excluded and caused issues after the
clone, we recommend that if this table needs to be excluded that you also preserve it, or if you just wish to delete some of the data from that
table to use a clone cleanup script.
• Tables to not exclude:
• sys_properties sys_properties is a base table that works with an instance's metadata and configuration. If excluded, the related information
will be missing on the Target instance, where the config and metadata are transferred. If preserved, the sys_properties are preserved on the
Target, where their related metadata and config are wiped out and replaced with the Source data.
• sys_trigger
• sys_plugins (this can potentially cause the target instance to go down)
• sys_user (as stated before but if preserved with the other tables it should be fine)
• Any sys_* tables custom exclusion should be treated with caution as these table data from the source will be excluded on Target ( E.g.
sys_app, sys_store_app and sys_scope exclusion can kill the modules if the application is not installed on the target)
• Excluding these can put your target instance into a bad state, and you will need to roll back the clone, adjust and clone again.

• Data preservation on cloning target instances (servicenow.com)

Clone and Upgrade By Dinesh Kumar Raghu
How do Profiles work? Is the System Profile my
default and should I use it?

• When requesting a clone, you can select a profile, once selected then the clone profile automatically
populates the clone request with your selected profile settings.
• Here is some more information about Profiles, these profiles can be helpful when you have different settings
to clone over different environments.
• Clone Profiles for Clone Requests
• NOTE: ServiceNow provides one profile called "System Profile" that will only include the OOB defined list of
excluded tables, preserve data entries, and clean-up script. So this will NOT include the excludes /
preservers you have created previously for your default clone. If you wish to keep using your default settings
without profiles then leave the profile field empty when requesting a clone, DO NOT use System Profile
unless you have configured it.

Post-clone tasks
• Perform the following tasks after your production instance is cloned:
• Update the welcome page
• Change the email properties
• Restrict user access
• Modify LDAP to disable imports and updates.
• Disable active scheduled jobs
• Restoration activity.

• Post-Cloning Checklist - Support and Troubleshooting (servicenow.com)

Clone and Upgrade By Dinesh Kumar Raghu
Can I roll back my clone?

• Yes, but only within the 7 days after the clone is run for non-sharded DBs, or
within 2 days for sharded DBs.
• As we take a backup of the database after the cloning process, we can roll back
the clone up to 7 days after the clone was run. After a clone is complete there is a
UI action Rollback clone. If you click this button, it will roll back the clone using
the same automation that Technical Support will use to roll back the clone. If you
are facing any issues or need clarification about your clone rollback, please
contact Technical Support for assistance.

Upgrade Instance:
• Planning:

• Create upgrade schedule with client.

• Ensure that there is a comprehensive test plan for after the upgrade.

• Verify most used apps/modules with client.

• Verify integrations and customizations.

• Once dates are confirmed by the client send the Monthly Schedule Notification.

• Schedule upgrades in the ServiceNow support portal.

• Add upgrade dates to Platform Excellence outlook calendar.

Clone and Upgrade By Dinesh Kumar Raghu
Upgrade Instance:
• Pre-Upgrade Checks:

• - Refer to ServiceNow’s Upgrade Planning Checklist document

• Release notes for upgrading from Washington DC (servicenow.com) / Release notes for upgrading from Vancouver (servicenow.com)

• Release notes for upgrading from Utah (servicenow.com)

• Phase 1 - Read the release notes and plan your upgrade (servicenow.com)

• Phase 2 - Prepare for the development instance upgrade (servicenow.com)

• Phase 3 - Verify your upgrade configurations and schedule the development instance upgrade in Now Support (servicenow.com)

• Phase 4 - Upgrade and validate the development instance (servicenow.com)

• Phase 5 - If applicable: Upgrade and validate your other non-production instances, such as your test instance (servicenow.com)

• Phase 6 - Prepare to upgrade the production instance (servicenow.com)

• Phase 7 - Upgrade the production instance (servicenow.com)

Upgrade Instance:
• Internal Checklist:
• • Ensure sub-production environments have been cloned in preparation for upgrade. If not, schedule clones from PROD to DEV/TEST(QA).

• • Send Upgrade reminder notification 24-48 hours before upgrade.

• Instance Checks-
• Verify the scheduled upgrade time and ensure that there are no conflicts.

• Take snapshots, document, and/or test the following in the clients’ lower instances:

• MID Servers and their status.

• Email Diagnostic settings and status.

• LDAP servers and status.

• Evident customizations or non-OOTB functionalities.

• CMDB discovery sources.

• Verify most used apps/modules.

• Test most used forms and document behaviour.

• Monitor the upgrade.

Clone and Upgrade By Dinesh Kumar Raghu
Upgrade Instance:
• Post-Upgrade Checks:

❖ Check the following are up and running:

• MID Servers

• Email Diagnostics

• LDAP

• Nodes

• Once the upgrade is complete, send relevant parties the completion email/notification with screenshots.

• Go through the skipped changes in Upgrade Monitor.

• Process the skipped changes and capture them in update set(s).

• Once the skipped changes have been processed, complete and save the update set(s).

• Send confirmation email to client of completed skipped changes.

Upgrade Instance:
• Testing, validating, and remediating:

• Ensure that all basic functionalities of the instance are working properly.

• Perform testing as illustrated with the test plan created during the planning phase.

• Review and fix any issues discovered during testing.

• Repeat from 'Pre-Upgrade' phase to 'Testing, validating and remediating' phase for all lower instances.

• Upgrade - PROD:

• Once all lower instances have been upgraded, validated, tested, and signed off successfully, proceed with upgrading PROD.

• Refer to ‘Post-Upgrade Checks’ above.

❖ For skipped changes processing:

• Migrate over the update sets with the skipped changes from lower instances. Check for any additional skipped changes that may appear. If there are additional
skipped changes, process them.

• Perform smoke testing on all vital functionalities. If any issues arise, troubleshoot, and fix accordingly.

• Once the upgrade is complete, send relevant parties the completion emails/notifications.
Clone and Upgrade By Dinesh Kumar Raghu

Thank you.

Clone and Upgrade By Dinesh Kumar Raghu