Dobimigrate-6 4 3
Dobimigrate-6 4 3
4
Revision 4
Table of Contents
1. Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2. Configure Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.7. LDAP/AD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8. Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.10. System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3. Set up Throttling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.6. Multiprotocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
1. Getting Started
Welcome to the online help for DobiMigrate.
Using this help, you can configure, manage, and run migrations. DobiMigrate supports
multiple protocols. For NAS-to-NAS (N2N) migrations, SMB and/or NFS protocols are
supported and for Object-to-Object (O2O) migrations the S3 protocol is used.
Below is a brief description of the steps that are needed to perform a migration with the
DobiMigrate software.
• Configure the DobiMigrate Settings and environment to prepare for a migration. This
involves the configuration of the File servers, Global Settings, and (optionally) items
such as SID Maps, Email Home, LDAP/AD, Users.
• Shares and Exports Management which involves creating Shares and Exports on the
target file servers.
• Switchover is the step when a final scan of the data is done, and any changes are
copied one last time from the source to the target.
• Post Finish Actions can be taken if you are not sure that the migration has finished
correctly or when there are issues with accessing data on the system; you can verify
and try to repair the migration. Exporting migration information is another task that can
be done at this step.
Before you start with the configuration of the DobiMigrate settings and
environment, make sure that the DobiMigrate Core software and the proxies
have been installed as described in the DobiMigrate Installation Guide.
The migration workflow for NAS and Object are similar with a few exceptions. Any differences
between a shared step are identified within the content with related images. Use the chart
below to identify which steps are part of either N2N or O2O.
1
DobiMigrate Online Help 6.4
Configure Settings X X*
Setup Throttling X X
Setup Migration X X*
Manage Migration X X
Monitor Migration X X
Switchover/Dry Run X X
Rollback X
REST API X
Tools X X
We aim to keep this help and other support information up to date with the latest state of our
software. To accomplish this we also publish updates on the Datadobi Support Portal,
available to registered users.
2. Configure Settings
The Configuration module contains the following tabs that allow you to manage:
• File Servers: add a new file server and view, edit, remove configured file servers, and
configure a schedule
• Object Storage: add a new object storage type and view, edit, or remove existing object
storage
2
DobiMigrate Online Help 6.4
• System: restart the DobiMigrate Core, configure HTTPS and upload DobiMigrate
software
• Global Settings: configure settings for the DobiMigrate instance, usage display,
migrations and analytics
• License: view DobiMigrate license key information and request and upload a new
license key
For more information on each of these operations, see the respective topics below.
• Other NAS
3
DobiMigrate Online Help 6.4
NetApp Data ONTAP covers both Data ONTAP 7-mode and Clustered Data
ONTAP.
For VNX/Celerra you need to provide the user name and credentials of the
admin account.
DobiMigrate integrates with the listed server types, for other server types, use Other NAS.
For integrated file servers, you need to specify the IP address (or DNS name) of the file
server’s management interface and the username and password that have been configured
for management access. Click Next when done.
When you have specified the right credentials for accessing the file server via the
management API, a window with the file server details is displayed:
The server details are retrieved via the management API and cannot be changed. In this dialog
you can only enter the DobiMigrate display name of the file server and configure a
schedule via the Configure button (see Configure Schedule). When done click Next.
DobiMigrate automatically starts a discovery of the sub file servers on the file server.
Depending on the file server type, the discovered sub file servers will be either:
4
DobiMigrate Online Help 6.4
• NAS servers
The next step is to select the parameters to set up a connection with the DobiMigrate proxies.
You can set up and test the connection between the DobiMigrate proxies and the file server. To
do so, select the IP addresses of the sub file server to which the proxies should connect. The
list of available interfaces is collected via the management API and you have the option to
manually add more interfaces as described below.
To set up a sub file server via DobiMigrate, select an available IP address from the
address popup for the respective sub file server.
By default, the list displays the available addresses that have been collected from the file
server. Alternatively, you can add other server addresses by selecting the option Manage IP
addresses… from the popup. This will open the following window where you can add specific
server addresses via the Add… button. Once added, select the new server address and
click OK so DobiMigrate can use it.
5
DobiMigrate Online Help 6.4
In the case of SMB, you also need to add the SMB credentials by selecting Manage
credentials… from the popup.
In the New SMB credentials dialog, enter the domain and user name for the account that will
be used to access the data with the SMB protocol. Make sure that the user account is part of
the local backup operators group, and the local administrator’s group, on the file server.
Once you have specified account credentials, DobiMigrate will also make these available as an
option to choose from when setting up the other file servers.
When you have deployed SMB and/or NFS proxies prior to the setup of the file server, these
proxies are listed in the proxies table. Select one or more proxies and click the Test
connection button to validate if the selected proxies can connect to the configured sub file
servers using the given credentials. A separate dialog shows the Connection test result. When
all is OK, close the dialog and click OK to finish the setup of the file server.
In case no proxies are displayed, they probably have not been set up yet. You can complete the
setup ignoring this screen and come back to it later by editing the settings of the file
server after you have set up the proxies as described in the DobiMigrate Installation Guide. See
6
DobiMigrate Online Help 6.4
For Isilon versions older than 7.2, NFS support is only available for the
System access zone. For Isilon access zones, you can specify the pool of IP
addresses to which the proxies can connect.
Once the SMB parameters have been set up, the next step is to setup the shares DobiMigrate
will be using for the migration. This can be done via the SMB data access shares tab.
DobiMigrate shows the set of directories on the server to which access is required.
DobiMigrate allows the user to select which share should be used to access the file system
path in the leftmost column. The options are:
• Automatic: DobiMigrate will automatically create a new migration share with a new,
unique name and will provide full access to the user with the credentials as provided in
the previous step. Click on Validate to check which operations DobiMigrate will perform
to configure the share(s).
• Manual: DobiMigrate will use an existing share on the file server without making any
modification to it.
• None: DobiMigrate will not use this path in the migration process.
The final screen of the file server setup allows you to manage the migration exports and how
these should be accessed. DobiMigrate shows the set of filesystem paths on the server to
which access is required. Access options are:
• Automatic: DobiMigrate will update the existing export or create a new one and allow
full access to the assigned NFS proxies for this file server. Click on Validate to check
which operations DobiMigrate will perform to configure the export(s).
• Manual: DobiMigrate will use the existing export without making any modification to it.
The user needs to give full access to the assigned NFS proxies manually.
• None: DobiMigrate will not use this path in the migration process.
Click OK when you have made your selection. DobiMigrate will ask confirmation for
creating/modifying the shares and exports. Click Finish to finalize the setup of the file servers
and the migration shares and exports. For more information on viewing or updating the
created file server, see Edit/View a File Server.
7
DobiMigrate Online Help 6.4
Other NAS
In this case, you need to enter a name for the file server and specify the IP address (or DNS
name) for data access via the NFS and/or SMB protocol.
In case you have deployed DobiMigrate proxies prior to the setup of the file server, these are
listed in the proxies table. Select one or more proxies and click the Test… button to validate
whether the selected proxies can connect to the selected interfaces using the credentials as
provided. A separate dialog shows the Connection test result. When all is OK, close the dialog
and click Next to manage the SMB data acess shares and/or the NFS data acess exports.
In case no proxies are displayed, they probably have not been set up. For
more information on their setup, see the DobiMigrate Installation Guide.
In the next dialog you can select the SMB shares that DobiMigrate to access the data, options
are:
• Manual: DobiMigrate will use the share on the file server without making any
modification to it.
• None: DobiMigrate will not use this path in the migration process.
8
DobiMigrate Online Help 6.4
Click Next and select the NFS exports that DobiMigrate can access to migrate data. Options
are:
• Manual: DobiMigrate will use the export on the file server without making any
modification to it.
• None: DobiMigrate will not use this path in the migration process.
9
DobiMigrate Online Help 6.4
Click OK and the file server is added to the list with configured file servers.
Go to Configuration > File Servers and select the file server that you want to edit/view from
the list. Click the Edit button. This opens the file server edit dialog on the General tab. The
information shown depends on the type of file server. The dialog allows you to change the
information as given during configuration as described in Add a File Server.
For integrated file servers, you can change the IP address or DNS name and the
username and the password that have been set up for management access.
On the Overview tab, you can change the name of the file server as it will be displayed in the
DobiMigrate GUI. Depending on the file server type, this tab shows:
• Name: the name of the system as provided when it was added to DobiMigrate.
• Total raw capacity: (Isilon and Unity) the capacity used by the file server.
• Filename Encoding: (Isilon only) the encoding that has been set on the file server.
Typically this would be UTF-8.
• Nodes: not for all file server types. Information differs per type of file server.
10
DobiMigrate Online Help 6.4
You can confirm changes at any time and close the edit dialog by clicking OK.
On the tab with the discovered sub file servers, you can change and test the connection
between the DobiMigrate proxies and the access points of the file server. For more
information, see Add a File Server.
Other NAS
On the General tab of Other NAS file servers you can view and edit the connection details for
the SMB share and/or NFS exports as described in Add a File Server. In the Name field you can
change the name that was given when the file server was added to DobiMigrate.
11
DobiMigrate Online Help 6.4
The shares and exports on the file server that DobiMigrate can access to migrate data are
shown on the SMB data access exports tab and the NFS data access exports tab
respectively. For more information on the management of shares and exports, see Add a File
Server.
To view the configuration details of a file server, go to Configuration > File Servers. The
overview shows all (sub) file servers with their configuration settings. To view detailed
information of an individual (sub) file server, select it from the overview and then click on
the Details button under the table. For example, select vfiler1 to view its details:
For all types of file servers, the following tabs are available:
12
DobiMigrate Online Help 6.4
For the integrated file servers, the information depends on the file server type.
Deleting a server will also delete any migrations that use the server.
To add new object storage, go to the Configuration module and click on the Object Storage
tab.
1. Click the Add… button to open the Add New Object Storage dialog.
2. Select the storage type and click Next to specify the credentials for accessing the object
storage via the management API. DobiMigrate supports the following types of object
storage types:
13
DobiMigrate Online Help 6.4
3. Select the required details for the selected object storage type.
14
DobiMigrate Online Help 6.4
◦ Name: The object storage name that will be displayed in the UI.
◦ Protocol: Choose either http or https (Https is the default and is recommended as
it provides a secure connection to the object storage platform.)
15
DobiMigrate Online Help 6.4
◦ Discovery Schedule: this setting defines how frequently DobiMigrate will check
the server to rediscover system settings and information.
◦ Scan Strategy: The sort order used by this S3 server when listing object keys. The
Directory Tree option is used for object servers that do not list object keys in UTF-
8 or UTF-16 sort order.
◦ The Manual mapping option is selected by default, which allows the bucket to be
used as either the source or target for the migration.
◦ If you need to add an unlisted bucket, click on the Add unlisted bucket button.
6. Once the buckets are configured, click Finish to add the object storage.
Go to Configuration > Object Storage and select the object storage that you want to edit/view
from the list.
1. Click the Edit button. This opens the edit dialog on the Overview tab.
16
DobiMigrate Online Help 6.4
2. The information shown depends on the type of storage. The dialog allows you to
change the information as given during configuration.
3. Click on the S3 Data Access Buckets tab to review or edit the selected bucket. Buckets
can be manually added using the Add unlisted buckets button.
17
DobiMigrate Online Help 6.4
You can remove an object storage if it is no longer needed or used. Removing an object
storage also removes any pending or finished migrations, as well any reports that use the
removed object storage.
2. Select the object storage from the list on the Object Storage tab.
Removing a target storage will NOT remove the data that was already
migrated to that target storage.
18
DobiMigrate Online Help 6.4
By default, DobiMigrate is configured to run outside business hours with 64 threads for
scanning and 64 threads for operations. Examples of operations are: copy or delete files,
create symlinks, create or delete directories, transfer metadata or security settings. The
scanning operations serve two purposes:
1. To scan the file systems on the file server to discover changes that need to be inspected
and processed.
2. To scan the metadata of the files residing on the file systems for the generation of
reports that provide insights into the type of data on these systems.
A thread in the context of DobiMigrate refers to the number of concurrent access connections
that are made to a file server in order to manipulate data in parallel.
If you do not want DobiMigrate to interact with a file server from 9:00 to 17:00 during the
weekdays, then draw a block from 9:00 to 17:00 on Monday and use the Copy to
weekdays button to get the following week view.
19
DobiMigrate Online Help 6.4
To configure settings for all users, go to Global Settings > Instance Settings > Edit.
Name: This is the name that will be displayed as the title of your web page or tab when you log
in to this DobiMigrate instance. Changing the name can be handy if you have multiple
instances running and want to distinguish between them.
Session Timeout: Enable this to terminate a session (user logged out) when a session (logged
in user) has been inactive for a set amount of time.
User Lockout Threshold: Set a number value here to determine the number of consecutive
failed login attempts a user can make before being locked out.
If a user account has been locked, you can unlock it via Configuration >
Users.
20
DobiMigrate Online Help 6.4
This backup does not contain detailed states of those migrations, such as
scan files, error lists or chain of custody records. It is not possible to look at
Keeping a backup of the DobiMigrate configuration is highly recommended, as it will allow you
to restore the DobiMigrate core easier and faster in the event of, for example:
• Manual errors: such as accidentally removing a file server and deleting all associated
migrations
DobiMigrate allows you to define a custom schedule that is used to trigger the configuration
backup on a regular basis.
These backups are saved as .zip files and can be encrypted if required. They can be stored in
any location on the filesystem of the machine running DobiMigrate. You can choose to backup
to an NFS mount, for example. This way the configuration backup archives can even be stored
automatically on an external system, safeguarding against the loss of the machine running the
DobiMigrate core.
To change the settings of the configuration backup, click on the Edit button, change the
values, and click the Save button.
Backup location: specifies the place on the core where the backups will be stored.
Schedule: Defines the moments when the backup of the configuration will be made. A backup
21
DobiMigrate Online Help 6.4
will be created every day at midnight by default, but this can be changed. It is possible to
define a custom backup schedule.
Remove old backups: allows you to indicate if you would like to remove old backups:
• If this option is checked, old backups will be removed and only the most recent will be
kept. The number of backups to keep can be specified.
• If the option is unchecked, all backups will be kept and the number of backups to keep
is not relevant.
Number of backups to keep: Specifies the number of backups that are kept. Older backups are
automatically cleaned up.
It is also possible to immediately Take backup now and/or Disable the automated triggering
of the configuration backup.
Iteration Schedule: You can define iteration schedules that can later be applied to individual
jobs. This is useful since often the schedule you want to use is the same for quite a few jobs.
Templates are defined here that apply to jobs afterwards.
Minimum age: with this setting you can define how old the files on the source server need to
be before they are taken into account for the next iteration. With this setting you can avoid that
DobiMigrate will migrate data that is updated constantly during the steady state process. Note
that during dry run and switchover activities, all data will be migrated regardless of this setting.
The modification timestamp of the file is used to check for file updates. This timestamp is
compared with the DobiMigrate server time (UTC) to calculate the file age which will then be
compared with the minimum age setting to decide whether the file needs to be migrated or
not.
To ensure a proper working of the minimum age setting, you need to make
sure that NTP is set up on all involved systems.
For the example below, with the start of the next iteration DobiMigrate will:
• migrate File A: because its age since the last update is higher than the minimum age
setting.
• Not migrate File B: because its metadata has been updated more recently than defined
by the minimum age setting.
22
DobiMigrate Online Help 6.4
• migrate the metadata changes made to Directory C: because its age is older than the
minimum age setting.
Digest algorithm: allows you to specify which algorithm should be used to check the integrity
of copied data during a migration. This global setting defines the default that will be used
when setting up migrations. The digest algorithm is used by DobiMigrate during the migration
process internally, and during the creation of a Chain of Custody. You are able, however, to
deviate from this globally defined default by selecting a different algorithm when you set up an
individual migration.
Max number of concurrent scans: This setting, with a default value of 256, dictates the
number of scans that can be run at any point in time by the DobiMigrate core. For migration
projects with a lot of migrations running in parallel, you may want to increase this number.
This will have an impact on the memory requirements for the DobiMigrate core instance.
Please inquire with Datadobi for guidance.
Discovery Schedule: this setting defines how frequently DobiMigrate will check the file server
to rediscover system settings and information.
Share/Export Operation Timeout: The timeout to use when retrieving or modifying SMB
shares and NFS exports.
23
DobiMigrate Online Help 6.4
Discovery Schedule: this setting defines how frequently DobiMigrate will check the file server
to rediscover system settings and information.
The user can define a text message that will display on the login screen.
In the SMTP Configuration panel you can configure the mail server through which
DobiMigrate has to send health reports, the credentials and the sender’s address.
By default, SMTP is disabled. Click the Enable button to open the configuration form. If SMTP
is already enabled and configured, you can edit the existing settings, by clicking the
Edit button.
• SMTP host: host name of the SMTP server to use (for example: mail.yourdomain.com).
• Use Secure Sockets Layer (SSL): select this option if the SMTP server accepts
connections using Secure Socket Layer/Transport Layer Security (SSL/TLS).
• Authentication: the type of authentication to use with the SMTP server. Possible values
are None and User & password. In case a user and password are required, the following
two fields need to be filled in:
◦ SMTP user: user name to be used for authentication with the SMTP server.
◦ SMTP password: password to be used for authentication with the SMTP server.
• From address: the email address from which to send the health reports.
Once all settings have been configured click on the Save button in the panel.
You can use the Test button to verify the given settings. DobiMigrate will try to connect to the
given SMTP server with the specified authentication settings and will show the result.
24
DobiMigrate Online Help 6.4
A health report gives a view of the overall condition of the migration environment, and job
reports give a detailed view of running jobs.
In the DobiMigrate Health Reports panel you can configure the receiver’s address and the
frequency of the health reports.
The Job reports panel allows you to configure the same settings for the job reports.
If the reports setting is disabled, click the Enable button to open the configuration form. If
already enabled and configured, you can edit the existing settings, by clicking the Edit button.
• To addresses: The email address to send the health reports to. Separate multiple
addresses with commas.
• Frequency: Specifies whether a health report should be sent daily, weekly or monthly.
• Include coverage report: Provides an overview of the migration completion for the files
(buckets).
Once all settings have been configured click on the Save button in the panel.
You can use the Test button to send a test email using the given configuration. Note that
SMTP needs to be configured as well as described in the section above.
25
DobiMigrate Online Help 6.4
2.5.3. Alerts
• To addresses: the email addresses to send the alerts to. Separate multiple addresses
with commas.
• Filesystem roots that define the locations where DobiMigrate keeps its system files,
project data, and where you add more capacity if needed.
• Dropzones that define where DobiMigrate will drop large file exports. The Chain of
Custody files, for example, will also be dropped in these locations.
For a typical DobiMigrate installation one root location will be assigned during the installation
for DobiMigrate to keep all its information. The default path is: /mnt/data/dobiminer/ for .OVA
based deployments. However, when DobiMigrate is deployed in large environments, it is highly
recommended to either provision enough capacity for the default root location or to add extra
paths for DobiMigrate to use. Extra capacity can be added by adding more disks and once
created and configured at the operation system level, configure them in DobiMigrate. More
information on adding extra capacity is described in the DobiMigrate Installation Guide.
• A window appears that allows you to select a path. Note that you can only select a
directory for which DobiMigrate has the UNIX file system permissions read, write,
and execute.
26
DobiMigrate Online Help 6.4
• Once you have made a selection, click on the Select button. In the window that appears
next, the Results option has been selected by default. Keep this setting and click
on Save to save the path settings.
A dropzone allows you to quickly store and retrieve large exported lists with information
related to discovery or migration activities. This approach is recommended when you want to
export and keep large amounts of information such as a list of all files that were migrated to
the target file server. Since these files can be gigabytes in size, it is best to save them in a
dropzone and then retrieve them by connecting to the dropzone on the DobiMigrate instance
using tools as scp or WinSCP.
2.7. LDAP/AD
27
DobiMigrate Online Help 6.4
Instead of the default user management embedded in DobiMigrate you can also use LDAP/AD
to manage user access to DobiMigrate. Make sure that you configure the LDAP server
accordingly.
• Server address: the host name of the LDAP server. For example: ldap.yourdomain.com.
• Server port: the port number on which the LDAP server can be reached. The options are:
◦ Default: the default LDAP port is taken and the Custom server port field is greyed
out. Typically this is port 389 or 636 when using SSL/TLS.
• Use SSL: select this option to use Secure Socket Layer/Transport Layer Security
(SSL/TLS) when connecting to the server.
• Bind DN: the distinguished name (DN) to use when binding to the LDAP server. The DN
needs read-only rights on the server. For
example: DN=read_only,OU=people,DC=yourcompany,DC=com. Not needed when
anonymous binding is used.
• Bind password: the password to use in combination with the Bind DN when binding to
the LDAP server. Not needed when anonymous binding is used.
• Base DN: the base distinguished name under which to find user accounts. For example:
OU=people,DC=yourcompany,DC=com.
• User group DN: the group containing the accounts which will get DobiMigrate user
rights. For example: CN=DobiMigrate-users,OU=groups,DC=yourcompany,DC=com.
• Expert group DN: the group containing accounts which will get DobiMigrate expert
rights. For example: CN=DobiMigrate-experts,OU=groups,DC=yourcompany,DC=com.
• Admin group DN: the group containing accounts which will get DobiMigrate admin
rights. For example: CN=DobiMigrate-admins,OU=group,DC=yourcompany,DC=com.
• Server type: the type of LDAP server that will be used. Options are: Active Directory
, OpenLDAP or custom. When Active Directory or OpenLDAP are selected, the fields UID
attribute and Group member attributes will be greyed out. When custom is selected,
you need to enter values in these fields.
28
DobiMigrate Online Help 6.4
• UID attribute: the attribute used on the LDAP server to contain unique user IDs.
Typically uid on OpenLDAP or sAMAccountName on Active Directory.
• Group member attribute: the attribute used on the LDAP server to indicate group
membership. The member attribute is commonly used for this purpose.
Once all settings have been configured, you can use the Test button to validate the
configuration. DobiMigrate will try to connect to the configured LDAP server and retrieve the
specified Groups.
2.8. Users
To add a new user, click on the Add button in the bottom right corner of the screen.
A dialog box appears in which you can enter the following details:
◦ User: gives users a read-only right to view and export information. Users with this
role can consult some of the settings of DobiMigrate but cannot change them.
◦ Expert: in addition to viewing reports and exporting information, the expert role
allows users to manage all aspects of a job (manage file servers, set up discovery,
set up & manage jobs, manage proxies, create shares and exports, manage global
settings and SID Maps)
◦ Administrator: gives users full rights to DobiMigrate. Specific settings are only
reserved to the administrator such as DobiMigrate application settings. For
example: upgrade DobiMigrate, change licenses, set up Email home or LDAP/AD,
set up secure server, enable logging, manage users and configure DobiMigrate
capacity and dropzones.
Click on Save to save the user configuration. A user account is activated as soon as it has
been created.
29
DobiMigrate Online Help 6.4
To remove a user, select the user that you want to remove in the list and click on the
Remove button in the bottom right corner of the screen.
To edit a user, select the user that you want to edit in the list and click on the Edit button in the
bottom right corner of the screen. The edit option is mainly used to either change the role of
the user or its password.
To configure Session Management for an individual user, go to Configuration > Users. From
here you can select Local Account, External Account, and Show sessions.
Once you select a user, from the bottom of the page you can:
• Add
• Remove
• Edit
• Unlock user
• Terminate Session
See also, Instance Settings, under Configuration > File Server > Global
Settings.
30
DobiMigrate Online Help 6.4
2.10. System
The Configuration > System tab allows you to restart the DobiMigrate core server, to configure
HTTPS settings, and to update DobiMigrate software.
In some cases DobiMigrate might ask you to restart the DobiMigrate core server. To do this,
click on the Restart Server button in the top panel of the System tab. You will be disconnected
from DobiMigrate. After the restart you need to refresh your browser before you log in.
DobiMigrate by default uses standard HTTP (port 80) to provide access to its web GUI. If
required, it is possible to configure DobiMigrate to use HTTPS (port 443) instead.
DobiMigrate can generate a key pair and certificate itself but it also accepts them from a
certificate authority like VeriSign, Thawte, GeoTrust and others.
Depending on the kind of certificate that you want to use, you can:
A dialog box appears showing the certificate that DobiMigrate has generated. The owner
should match the name of the host on which DobiMigrate is running.
Click the Restart button to restart DobiMigrate using this new certificate.
After the restart, connections to the DobiMigrate web GUI will automatically use HTTPS. The
web browser will probably display a warning that the site’s certificate is untrusted. You can
ignore this warning and go to the web GUI using an encrypted connection to DobiMigrate.
31
DobiMigrate Online Help 6.4
• Upload a certificate: For optimal security it is best to obtain a public/private key pair and
a matching certificate (in PEM format) issued by a certificate authority. The host name
in the certificate must match the URL at which DobiMigrate is installed. Please refer to
the certificate authority of your choice for detailed instructions.
A dialog box appears in which you need to copy your key pair and certificate (including the
BEGIN and END lines) to the appropriate fields.
Click Ok to continue.
The next dialog box shows the certificate that you have uploaded. The owner should match
the name of the host on which DobiMigrate is running.
After the restart, connections to the DobiMigrate web GUI will automatically use HTTPS. You
will now have an encrypted connection to the authenticated DobiMigrate web server. Attackers
will no longer be able to intrude your data traffic or redirect your connection to a fake server
without a security notification from your web browser.
Upgrading DobiMigrate
Before you proceed with the upload, ensure the /tmp directory has
enough space to store the upgrade image.
When the file has been uploaded, click the Upload button to upgrade to the new version.
DobiMigrate will automatically restart so you can log in using the new version. The
upgrade itself might take up to five minutes.
Should proxies fail to upgrade automatically, they can be upgraded manually by taking
the following steps:
32
DobiMigrate Online Help 6.4
Proxy
a. Copy the DobiMigrate proxy RPM to the Linux machine running the proxy.
DobiMigrate prompts you to retrieve and upload a license key in the following cases:
• when you log in for the first time after initial installation
Follow the steps below to obtain and upload the license key.
2. In the Request license key dialog, copy the text in the Encoded License Request field
and send it to [email protected] to obtain a new license file.
Do not restart DobiMigrate while waiting for the license key. If needed,
you can close the UI.
3. If you want to request a license key for specific source or target servers, click
Advanced….
4. Select the source or target file servers for which you want to request a license and click
Save.
33
DobiMigrate Online Help 6.4
5. When you have received the license key, go to the Configuration module and open
the License tab. Click on the Upload license key button at the bottom of the screen.
The File Servers tab in the Configuration module shows green check marks for the licensed
file servers. You can now start migrations for these servers.
34
DobiMigrate Online Help 6.4
The License tab in the Configuration module shows the following information on the current
license key:
• Customer: The name of the organization to which the license has been issued.
• DobiMigrate instance ID: A unique identifier for the DobiMigrate instance on which this
license key can be used.
• Migration capacity: The capacity that has been used for migration against the total
licensed migration capacity. Click on the Details button to see more information on the
capacity usage.
• Custom fields: Are fields that are added by DataDobi when creating the license.
• Cluster ID: The list of clusters on which DobiMigrate can be used. These clusters are
identified through their unique Cluster IDs or GUIDs.
• Capacity: The amount of capacity that has been purchased as part of the software
license.
35
DobiMigrate Online Help 6.4
against multiple storage servers, many to one storage server and any other combination; as
long as the proxies are connected to both the source and target system.
The Proxies module allows you to manage the installed proxies as described in the following
sections.
• View detailed information on the installed proxies and check whether the proxies are
online.
• Remove proxies that are no longer needed by selecting them and clicking the
Remove button at the bottom right of the page. A warning message appears to show to
which file servers the proxies are connected.
To schedule each proxy individually, go to the Schedule tab and select the proxy. Click on
the Edit button to open the scheduler.
36
DobiMigrate Online Help 6.4
Add the time slots and the associated bandwidth using the buttons at the right. Click OK to
save the schedule. The schedule is shown in the overview from where you can copy it to
another proxy using the Copy schedule button at the bottom right of the page.
You can assign one or more proxies to one or more file servers, either through the file server
configuration during setup (or when editing these settings), or using the Proxies module via
the Assign tab and the File Servers button at the top.
You can also assign proxies to a specific access zone. Access zones are for
Isilon; however, other integrated file servers have similar concepts and are
also supported.
37
DobiMigrate Online Help 6.4
1. Select one or more file servers and click the Edit button to assign one or more proxies to
it:
3. Click OK to close the dialog if the connection is successful. If not, click Cancel and fix
the issues shown in the test results dialog.
When the proxy is used to migrate data between the source and target file
server, make sure to have it assigned to both these servers.
You can see which file servers are assigned to one or more proxies using the Proxies tab
under the Assign module.
The Edit button allows you to connect or disconnect one or more proxies to or from one or
more file servers.
38
DobiMigrate Online Help 6.4
The Performance tab shows the statistics of all proxies per bandwidth or per scanned, copied
and deleted operations. You can zoom in and out using the Zoom buttons in the top right
corner.
3. Set up Throttling
Throttling can be set up for:
• Load by defining the number of concurrent connections that are allowed against the
source and target file server. With the Configure Schedule setting, you can define the
amount of load during and outside business hours
• Throughput or bandwidth at proxy level. For each individual proxy, you can define a time
schedule with restrictions on bandwidth usage based on a week schedule. For more
information on how to set up bandwidth throttling see below.
In addition you can vary the allocation of the number of proxies that you assign to a source
and target file server. This allows you to create a bigger pipeline between your source and
target file servers, especially when the data to be migrated are mostly large files (>1MB).
In case your migration is scheduled to run for an extended time, also consider the minimal
age setting and the iteration schedule length. The minimal age avoids that files that are
changed frequently are re-migrated with each iteration (e.g. mailbox PST files). These files
should only be migrated as part of the switchover windows to maximize the efficiency of
39
DobiMigrate Online Help 6.4
network usage.
The iteration schedule determines how frequently source and target server are scanned for
changes. If you have run a usage report prior to your migration setup, you can select those
paths that have few ongoing changes to increase the iteration schedule and those paths that
are used very actively to be scanned more often in relation to the total planned time for the
migration and switchover.
1. Go to the Proxies module and select the Schedule tab. This view lists all available
proxies, the time zone they operate in and their weekly schedules.
2. Change the default regimes that are available by clicking on the configuration button
next to the (colored) predefined bandwidths at the top right corner.
3. Select a proxy for which you want to apply restrictions and click the Edit button below
the list.
40
DobiMigrate Online Help 6.4
4. Specify the time zone of the proxy so that its bandwidth restrictions are aligned with the
local business activities.
6. (Optional) Use the button on the right-hand side to duplicate the slot across the week
schedule.
Throughout the day and week, you can create multiple slots, each with their specific
regime.
41
DobiMigrate Online Help 6.4
7. Confirm the schedule with the OK button. You can reuse a defined schedule across
multiple proxies:
a. Select the proxy and clicking the Copy schedule button under the list.
The Copy schedule dialog lists the proxies to which you can copy the schedule.
At any point in time, you can check whether a certain proxy is subject to a bandwidth
restriction through the Proxies Dashboard.
If all your migrations take place in the same time zone as the time zone in which the
DobiMigrate core server is running, then no specific time zone settings are required.
• Proxy level
It is best to align the proxies with the time zone in which they operate. This would normally be
the same location as where the source and target file servers reside.
The reason for this is that the bandwidth restrictions need to align with local business context
and operations.
For example, if you are migrating infrastructure based in one location, but you are monitoring
and managing the project from elsewhere, then you set the time zone of the proxies. If a
customer does not want any migration activities to occur during the busiest part of the
working day, for example between 9:00 AM to 6:00 PM, then you just draw a slot in the proxy
schedule of the local proxy and set its regime to 0Mbit/s bandwidth speed.
If you are also migrating from infrastructure in a local data center, facing the same bandwidth
restrictions during local business hours, you select the select Copy Schedule and select those
proxies. Once this is done, you can then change their time zone via the Edit button.
42
DobiMigrate Online Help 6.4
The same approach as for proxies needs to be taken when you want to configure the
maximum load (expressed by the maximum amount of concurrent threads) against your
source and target file server. You can configure the time zone and business hours for which
load restrictions apply via Configure Schedule.
Migrations can be added by manually adding migration paths, or by using bulk import to
quickly add multiple migrations using a common set of migration options.
2. First select the source file server from the drop down at the top (see screenshot, below),
and then select the paths to be migrated.
3. Select the path(s) that you want to include in the migration. You can migrate at the root
level or at any subdirectory level.
43
DobiMigrate Online Help 6.4
4. Once you have made your selection, click Next to go to the Select target path dialog.
Here you can select the target file server and location and create additional directories
to serve as the root level for the migration data.
44
DobiMigrate Online Help 6.4
When you have selected the migration paths, a dialog opens listing the select migration
paths and corresponding security information and migration-types.
5. Verify that the Protocol is filled in correctly. In case there are multiple options, you can
select the correct protocol using the drop-down in this column. The following values are
possible:
45
DobiMigrate Online Help 6.4
◦ Multiprotocol: Copy data over NFS and metadata using both NFS and SMB.
◦ SMB + symlinks over NFS: Copy files and directories over SMB. For symbolic
links, use NFS to create them and then apply SMB metadata.
When you select this option for the protocol, you can no longer select "Normal
symbolic link handling" for the "Symbolic links" option, only the following options
are possible:
◦ NetApp Mixed to Isilon: DobiMigrate will choose the most appropriate copy
protocol, NFS or SMB, for the data and metadata of each individual file or
directory.
46
DobiMigrate Online Help 6.4
Make sure the relevant job options are selected, then click Next. Read Configure
Migration Options for more information about all of the different choices.
47
DobiMigrate Online Help 6.4
◦ The right permissions have been given to DobiMigrate to perform the migrations
from the source to the target location.
After review of the results, you can change the migration settings if needed or click Finish to
start the migration process.
migration setup. Cancel the migration setup, resolve the issue and start the
setup of the migration again. When in doubt, contact Datadobi because a
failed pre-check could lead to an incomplete or incorrect migration result.
The bulk import feature allows you to create multiple migrations by uploading a CSV or Excel
file. Templates for these files can be downloaded from the migration wizard as detailed in the
following steps.
48
DobiMigrate Online Help 6.4
• Full Import: Allows you to import migrations for multiple file server pairs. In this case
you need to specify the source and target server for each of the migration paths.
• Paths Only Import Allows you to import migration paths for one specific file server pair.
In this case, specify only the source and target paths in the template. You will be asked
to choose the source and target file servers, after uploading.
To add multiple migration paths in one go, via bulk import, take the following steps:
2.
If you have not done so already, download the applicable template using the
button.
3. Add the migration path definitions to the template. The templates contain a description
of the data that is needed and how it should be filled in.
4. Upload the completed template using the Import button. From here you can browse to
the correct file.
If you are uploading the paths only import template verify that the
correct source and target file servers are selected in the dropdowns at
the top of the wizard.
49
DobiMigrate Online Help 6.4
DobiMigrate will process the import, perform some prechecks, and display the results
as a summary of imported migrations, including security mappings and protocols used.
Incorrect imports will be flagged and will need to be corrected in order to continue.
5. Proceed with the migration setup as usual as described in step 6 in the section Add a
NAS Migration.
2. Select the source object storage server from the dropdown at the top and then select
the paths to be migrated.
3. Select the path(s) that you want to include in the migration. Select the source bucket
that you want to migrate. Click Next to go to the Select target path dialog.
4. Select the target bucket. (DobiMigrate will prevent you from selecting migration paths
that are from the same bucket.)
50
DobiMigrate Online Help 6.4
5. When you have selected the migration paths. Click Next to view Job options.
6. DobiMigrate will present an overview of the job options. Make sure the relevant job
options are selected. Click Next or the Advanced options… button.
◦ Iteration schedule: allows you to define when migration iterations will run. By
default the migration will start an iteration at midnight, but there are lots of
different values to choose from:
▪ Use default: refers to the default schedule as defined in global settings. The
value for the default global iteration schedule setting can be changed under
Configuration > Global Settings > Job Settings.
▪ Manual: In this mode you have to manually start the next process iteration
in the detail screen of the respective processes. Once started it will execute
51
DobiMigrate Online Help 6.4
the process for the migration path and pause it again before starting the
next process step or iteration.
▪ Presets: Predefined schedules are sets of starting times, for example every
6 hours would be 0:00, 6:00, 12:00 and 18:00.
◦ Start first iteration: allows you to bypass the Iteration schedule and start the first
iteration immediately.
7. Select any desired Advanced options… and then click Close to return to the Job options
dialog..
Digest algorithm: Allows you to choose the hashing algorithm used to compute the
digest value, to verify the migrated data and potentially generated a CoC. The default
option indicates that the digest algorithm specified in Configuration > Global Settings >
Job Settings will be used. You can choose to use a different algorithm for this specific
migration by selecting a different value in the dropdown list. Note that when you change
this value during a running migration, your final CoC will contain a mix of different digest
algorithms.
Target read-back verification: Read-back object content from target after copying for
verification.
◦ The right permissions have been given to DobiMigrate to perform the migrations
52
DobiMigrate Online Help 6.4
◦ The target locations are empty or not. Regardless, of empty or not, a full mirror of
the source will ensure that the source and target match.
9. After review of the precheck results, you can change the migration settings if needed or
click Finish to start the migration process.
migration setup. Cancel the migration setup, resolve the issue and start the
setup of the migration again. When in doubt, contact Datadobi because a
failed pre-check could lead to an incomplete or incorrect migration result.
• SMB Options
• NFS Options
Migration options are also available from the Jobs > Running tab.
From the Job Options window, there is a button to access the Advanced options. which
consist of:
• Iteration schedule: allows you to define when migration iterations will run. By default
the migration will start an iteration at midnight, but there are lots of different values to
choose from:
53
DobiMigrate Online Help 6.4
◦ Use default: refers to the default schedule as defined in global settings. The value
for the default global iteration schedule setting can be changed
under Configuration > Global Settings > Job Settings.
◦ Manual: In this mode you have to manually start the next process iteration in the
detail screen of the respective processes. Once started it will execute the process
for the migration path and pause it again before starting the next process step or
iteration.
• Start first iteration: allows you to bypass the Iteration schedule and start the first
iteration immediately.
• Minimum age: defines the minimum amount of time a file needs to be unmodified
before it will be taken into account for the next iteration. With this setting you can avoid
that DobiMigrate will migrate data that is updated constantly during the steady state
process. The globally defined default is set to 1 hour and can be changed
in Configuration > Global Settings, Migration Settings.
During dry run and switchover activities, all data will be migrated
regardless of this setting.
• Root directory handling: allows you to determine how DobiMigrate should copy the
security (permission bits or ACL) of the root folder of the migration to the target. The
possibilities are:
◦ Copy security: will ensure DobiMigrate copies the security as is defined on the
source. This is the default setting.
◦ Don’t copy security: will instruct DobiMigrate not to copy any security settings
from the source. Choose this option in cases where it is not possible to read the
source root directory permissions or to apply these permissions to the target root
directory. This is, for example, the case when you cannot grant DobiMigrate the
permissions to do these actions.
54
DobiMigrate Online Help 6.4
ACE inheritance algorithm will delete the inherited ACEs on the migration root
folder. It removes them because they are not present in the parent ACL.
◦ Consolidate root folder: allows you to migrate directories from several source
paths into one target path. DobiMigrate has built-in detection and protection
mechanisms for consolidation scenarios and will warn you to apply consolidation
if several paths are migrated to a single target folder. Data from several source
paths are merged to a target root folder. Any changes happening under the
consolidation jobs source paths are propagated to the target during the iterations
and switchover.
▪ The root security of any of the consolidated source paths is not copied, as it
is impossible to determine which exact source security settings to use on
the target path.
▪ Files directly in the source root folder are skipped. Only directories in the
root folder are migrated.
▪ If several consolidation jobs contain folders at the root level with the same
name, a blocking error message will be shown. In this case, you should use
the Consolidate sub folders option or rename the clashing folder on one of
the source root paths to make it unique.
◦ Consolidate sub folders: Allows you to run the consolidation for the root folder
and its 1st level sub-folders. The consolidation of the 1st level sub-folders is
executed in the same way as the consolidation of root folders, which is described
in the previous bullet.
• Skipped path/file patterns: directories or files matching these patterns will be skipped
during iterations that are not part of a dry run or switchover. This can be used to avoid
recopying files that will change often during the migration lifecycle.
55
DobiMigrate Online Help 6.4
◦ The \ character can be used to escape other characters that would otherwise
have a special meaning. For example: \\ matches the backslash character and \{
matches the left brace character.
◦ The hyphen – indicates a range of characters; [a-z] specifies all characters from a
to z (inclusive). A combination is also possible: [abce-g] matches a, b, c, e, f, or g.
If the ! character follows the left bracket, then it is used for negation: [!x-z]
matches any character except x, y and z.
◦ The characters *, ? and \ match their actual characters. The – character matches
its actual character if it follows the left bracket or the ! after the left bracket.
◦ Examples:
▪ /departments/{hr,mgmt} will
exclude /departments/hr and /departments/mgmt
/invoices/20\{12*,13*,140[1-9],141[0-1]} will
exclude /invoices/20120301,/invoices/20140901, /invoices/20141101 but
not /invoices/20141201 or /invoices/20150101
◦ You can enter multiple patterns in a text box via the Edit button on separate lines.
◦ You must specify each pattern in a separate line otherwise none of the patterns
on the same line will work. Press the Enter key after every pattern.
56
DobiMigrate Online Help 6.4
source and target file server types. The default patterns exclude the system
files and directories that should typically not be migrated.
• Allowed operations: gives you the possibility to restrict the operations that are allowed
during this migration. Note that this refers to operations being carried out on the target.
As part of the migration process, DobiMigrate may delete directories/files from the
target that are not on the source as part of the process to match the target with the
source. DobiMigrate will never delete or update anything on the source.
◦ All operations allowed: is the default value for this migration option. Allowing all
operations without restrictions is the required value to be able to perform a
regular full migration.
◦ No deletes: allows you to ensure no directories nor files will be deleted from the
target at all during this migration process even when the directory/file no longer
exist on the source. The updates and/or additions that are made to
directories/files on the source will continue to be made/reflected on the target by
DobiMigrate.
◦ No deletes or updates: allows you to ensure no directories nor files will be deleted
nor updated from the target at all during this migration process. The updates that
are made to directories/files on the source will not be made/reflected on the
target. This option can be useful when there is existing data on the target that
may not be modified by the migration.
Preserve access times on source: allows you to preserve the access time of the source files
when migrating the data to the target file server.
This will impact performance and may require write-access on the source.
In addition to the general migration options, protocol-specific options and advanced options
are available for the data of the selected migration paths as described in the sections below.
57
DobiMigrate Online Help 6.4
• Chain of custody: specifies for which set of data you would like to generate a Chain of
Custody (CoC) entry in the Chain of Custody File (CoCf). This feature provides a full
record of migration activity, including timestamps and digests, which is important for
auditing purposes.
If you start a migration with a non-empty target, the CoC will issue ‘resync’ commands
that compare the full data of each item on the target with the corresponding data on the
source. While doing that, the digest of each of these items is calculated and included in
the CoC. This can be expensive; in which case you may want to change this setting from
‘Full’ to ‘Migrated data’. This avoids the ‘resync’ commands, but be aware that it omits
the pre-existing target data from the CoC. There should be no need to set this option to
‘None’.
◦ Full (default): to generate a CoC for all of the data. This means that a CoC will be
generated for pre-existing data on the target in addition to generating a CoC for
the data that is migrated during this process. The pre-existing data will be
validated by comparing it with the source.
58
DobiMigrate Online Help 6.4
◦ Migrated data: to limit the generation of a CoC to the data that is copied during
this migration process. This means that the data that already exists on the target
will not be included in the CoCf of this migration.
In addition, the safety net provided by the Full migration option will not be used.
Thus, inadvertent target modifications made by users will be overwritten when
DobiMigrate synchronizes the modified items with their corresponding source
items.
◦ None: if you don’t wish to keep a CoC for this migration at all. Additionally, files
that are modified on the target before a switchover, will be overwritten because
there is no CoC.
• Advanced Integrity Protection: allows you to enable or disable the Advanced Integrity
Protection (AIP) setting. Enabling this setting ensures that DobiMigrate generates a
timely warning when the data on the target is externally modified.
This option also adds a safety net by trying to detect inadvertent target modifications
made by users before a switchover is completed. When such modifications are
detected, DobiMigrate flags the modified items as "Modified externally" and will not
synchronize them with their corresponding source items during subsequent iterations.
It is recommended that you disable this setting only in exceptional circumstances. For
example, if you keep seeing false positives for AIP and cannot track down the root cause
for them.
• Digest algorithm: allows you to choose the hashing algorithm used to compute the
digest value, to verify the migrated data and potentially generated a CoC. The default
option indicates that the digest algorithm specified in Configuration > Global Settings >
Job Settings will be used. You can choose to use a different algorithm for this specific
migration by selecting a different value in the dropdown list. Note that when you change
this value during a running migration, your final CoC will contain a mix of different digest
algorithms.
• Source read-back verification: causes DobiMigrate to read back the data from the
source a second time, after it has been copied, to verify that the source potentially did
not supply any corrupt data. This option is disabled by default. Note that enabling this
extra verification, will impact the performance of the migration process.
59
DobiMigrate Online Help 6.4
• Follow junctions: By default, DobiMigrate will skip junctions when these are discovered.
◦ When this option is enabled, DobiMigrate will not ignore the junction and migrate
all the content it refers to as if it was a directory with that content.
• Copy owner: This will copy the owner of the Security Descriptor. When not set, it will be
left blank on the data on the target file server.
• Override owner: This owner will be used for all the migrated files and directories.
• Copy group owner: This will copy the group owner of the Security Descriptor. When not
set, it will be left blank on the data on the target file server.
• Override group: This group will be used for all the migrated files and directories.
• Copy discretionary access control list (DACL): A DACL identifies the trustees that are
allowed or denied access to files or directories.
• Copy system access control list (SACL): A SACL contains access control entries (ACEs)
that specify the types of access attempts that generate audit reports. Each ACE
identifies a trustee, a set of access rights, and a set of flags that indicate whether the
system generates audit messages for failed access attempts, successful access
60
DobiMigrate Online Help 6.4
attempts, or both.
This controls whether it will be copied to the target or not. Not all file servers
support SACLs (for instance: Isilon stores them but doesn’t use them).
• SID map: to specify SID mapping from source to target. This option is described in more
detail in section User Mapping.
• Clean invalid security descriptors: DobiMigrate will clean up any invalid security
descriptors found on the source platform before copying them to the target.
• Replace "Creator Owner" and "Creator Group" in effective ACEs. These are two well-
known SIDs that can be used in inheritable ACEs. Some file servers are tolerant to them
appearing in effective (inherited) ACEs; others will refuse to do that. When this option is
enabled, DobiMigrate will replace these SIDs in effective ACEs by the actual owner and
group SIDs of the file.
◦ Copy, ignore errors: tries to copy alternate data streams, but does not report
errors when it fails.
61
DobiMigrate Online Help 6.4
• Copy access control list (ACL): copy the POSIX ACL (NFSv3 only) or NFSv4 ACL if
present. For NFSv3 to NFSv4 migrations, DobiMigrate will convert POSIX ACLs to NFSv4
ACLs automatically.
• UID map: Here you can specify UID mapping to be used between source and target. This
option is described in more detail in section User Mapping.
• GID map: Here you can specify GID mapping to be used between source and target(This
option is described in more detail in section User Mapping.
• Override permissions: allows you override the value of each individual POSIX
permission. Click on the edit symbol on the right to configure the copying of NFS
permissions. For each permission you can select to either copy it from source to target,
or allow or deny the permission on the target regardless of the value on the source.
62
DobiMigrate Online Help 6.4
◦ Copy FIFO (named pipe) files: allows you to copy FIFO (named pipe) files. A FIFO
file is a special file used for exchanging data between processes executing on the
same Unix host. Options include Do not copy, Copy (with ACLs) and Copy (without
ACLs).
◦ Copy Unix domain socket files: allows you to copy domain socket files. A Unix
domain socket file is a special file used for exchanging data between processes
executing on the same Unix host. Options include Do not copy, Copy (with ACLs)
and Copy (without ACLs).
◦ Copy block device files: controls whether to recreate block device files on the
target.
◦ Copy character device files: controls whether to recreate character device files on
the target.
• Iteration schedule: allows you to define when migration iterations will run. By default
the migration will start an iteration at midnight, but there are lots of different values to
choose from:
63
DobiMigrate Online Help 6.4
◦ Use default: refers to the default schedule as defined in global settings. The value
for the default global iteration schedule setting can be changed under
Configuration > Global Settings > Job Settings.
◦ Manual: In this mode you have to manually start the next process iteration in the
detail screen of the respective processes. Once started it will execute the process
for the migration path and pause it again before starting the next process step or
iteration.
◦ Presets: Predefined schedules are sets of starting times, for example every 6
hours would be 0:00, 6:00, 12:00 and 18:00.
• Start first iteration: allows you to bypass the Iteration schedule and start the first
iteration immediately.
Digest algorithm: Allows you to choose the hashing algorithm used to compute the digest
value, to verify the migrated data and potentially generated a CoC. The default option indicates
that the digest algorithm specified in Configuration > Global Settings > Job Settings will be
used. You can choose to use a different algorithm for this specific migration by selecting a
different value in the dropdown list. Note that when you change this value during a running
migration, your final CoC will contain a mix of different digest algorithms.
Target read-back verification: Read-back object content from target after copying for
verification. This is disabled by default and the user should be aware that some cloud vendors
charge high egress fees for reading data from the cloud.
• Invalid SMB Filenames: select the behavior for when a file or directory can’t reliably
access over SMB because of, for example, invalid characters in file names, or multiple
file names with different casing for example.
64
DobiMigrate Online Help 6.4
◦ Generate errors: to report an error. No corrective action is taken and the file is not
migrated.
◦ Migrate files using NFS only, generate error for directories: to report an error for
directories, fall back to an NFS only copy for files (SMB security descriptors and
alternate data streams will not be copied).
◦ Migrate files and directories using NFS only: to fall back to an NFS only copy for
files and directories (and the complete subtrees below them). SMB security
descriptors and alternate data streams will not be copied.
◦ Create over NFS, apply metadata over NFS and SMB: all symbolic links are
created using the NFS protocol. Metadata is copied using both the NFS and SMB
protocol. This mode must be used for file servers that do not support the creation
of symbolic links over SMB but support symbolic links for all other operations.
◦ Create and apply metadata over NFS: all symbolic links are only copied using the
NFS protocol. No operations are done over SMB. This mode must be used for file
servers without support for symbolic links over SMB.
• Changing some options can require that all metadata of files and directories will need to
be re-migrated.
1. In the Running tab, under Jobs, select an item, and click Options.
2. Select Change Protocol…. The Migrations wizard is shown at the Selected Jobs Shares
and Exports page.
65
DobiMigrate Online Help 6.4
• If both source and target have SMB configured then the SID Map upload button will
become available.
• If both source and target have NFS configured then the UID Map and GID Map upload
buttons will become available.
In a Windows NTFS file system, all file access permissions and auditing controls are
associated with Security Identifiers (SIDs). A SID is a variable-length bit string which is
guaranteed to be unique worldwide. The relationship between a SID and a domain\account or
domain\group name is contained in the data structures maintained by a Domain Controller or
the Security Accounts Manager of a local machine. DobiMigrate provides an easy way of
(re)assigning files and permissions to a new account or group. This can be done by uploading
a SID map during the setup of a migration. At that time DobiMigrate validates its content
for syntax and the ability to resolve the imported user accounts.
66
DobiMigrate Online Help 6.4
1. [S-1-5-21-2694190442-3250320402-2687946070-1111]:[S-1-5-21-4248949456-
2451647171-897633524-17198]
2. [S-1-5-21-2694190442-3250320402-2687946070-1111]:CORP.COM\Administrator
In the first example above, a SID has been replaced with another, absolute SID. In the second
example, the SID will be replaced with the user Administrator that is part of the corp.com
domain. In the third example, the SID will be replaced with the user Administrator from the
domain in which the respective proxy is running.
SIDs that are no longer valid can be removed by specifying the REVOKED as
the target value.
UID stands for User Identifier, GID stands for Group Identifier. Both are numbers identifying
each user/group on systems with NFS exports (typically UNIX systems). There is a 1:1
mapping between the user name and the UID, and between the group name and GID. The UID
and GID are unique.
1. Admin_source:Admin_target
2. 100:Admin
3. 100-103:500
◦ In the first example above, files with owner or group Admin_source on the source
will be transferred to the target with owner/group Admin_target.
◦ In the second example, files with owner or group 100, will be transferred to the
target with owner/group Admin.
◦ In the third example, files with owner or group 100, 101, 102 and 103 will be
transferred to the target with owner/group 500.
UID/GID name mapping is only available for NetApp 7mode, CDOT and
Isilon. VNX and Unity only support UID/GID number mapping.
Thus, Admin_source:Admin_target will not work on Unity, whereas100:101
will work.
To view the details of uploaded mapping files, go to Configuration > User Mapping. The
screen below shows the details of a file that has been uploaded for a specific migration. At the
67
DobiMigrate Online Help 6.4
top, summary information is shown such as the File name, the number of Entries and the
number of Blank lines, Rejected lines, Source or Target user account errors.
The User mapping table shows the user mapping pairs that have been imported. These
mappings will be applied when you set up migrations using this user mapping.
DobiMigrate will not validate whether these user mappings exist, it will use
them as is. So it is critical that they apply in your environment and that
their mapping is correct. DobiMigrate only resolves the provided SMB
usernames to SIDs.
In case you want to use SID mappings, it is important that their syntax is correct and that the
user accounts exist for the mappings to use. DobiMigrate will only proceed with setting up a
migration when there are no rejected lines and user account errors. In case of errors, first
review and resolve the errors in your mapping file. Go back to the migration options and delete
the current user mapping. Then upload the updated mapping again for validation by
DobiMigrate.
68
DobiMigrate Online Help 6.4
Note that it is also possible to assign an owner SID and/or a group owner SID when you set up
a new migration. In these fields in the Advanced job options window, you can specify only the
SID values (e.g. [S-1-5-21-2694190442-3250320402-2687946070-1111]). This setting will be
applied independently from any SID maps that are set up at the migration level. Whenever a
SID value has been entered for one of these fields, the old SID values will be ignored and
replaced with the new ones for all the data in your migration.
69
DobiMigrate Online Help 6.4
4.6. Multiprotocol
There are two different types of file system security models: NTFS (ACLs) and Unix (Posix
mode bits), and in most cases a file system will use either one of these.
It’s also possible, however, for a file system to use both, or a mix of, NTFS and Unix security
settings. Depending on the access protocol, one of two security settings is used: NTFS
security for objects accessed over SMB, and POSIX mode bits for objects accessed over
NFSv3.
70
DobiMigrate Online Help 6.4
information.
4.6.2. Challenges
Multiprotocol migrations can present difficulties when migrating between different file server
product families because there are no industry standards governing security. This can lead to
confusion, because while both SMB and NFS are standard protocols, the simultaneous
multiprotocol access of file system objects is not.
As a result, any vendor that provides a multiprotocol function has created a proprietary
method of maintaining security as it relates to the storage of permissions and the resulting
evaluation and enforcement of access control.
• When migrating to NTFS security style, an SMB migration is required. If the source
contains symbolic links, then use SMB+symlinks over NFS.
Characters
• The following characters are not permitted in SMB filenames: \ / : * < > |
Trying to retrieve a file named with invalid characters will result in an error.
Encoding
SMB uses UTF-16 encoding for its filenames while NFS uses the encoding supplied by the
client. It’s therefore quite possible that a file can have a different name, based on whether it’s
viewed through NFS or SMB.
There are special file types that are only available on NFS, such as sockets and pipes.
71
DobiMigrate Online Help 6.4
1. DobiMigrate proxies first scan the data via both SMB and NFS protocols.
2. Once the scan is completed, all the data is copied to the target platform using the NFS
protocol.
3. Both protocol’s security information is applied to the target files as SMB and NFS
metadata copy operations.
Before configuring a Multiprotocol migration, verify that SMB data shares and NFS data
exports are both configured for the source and target file server paths you are planning a
migration for. Otherwise, the 'Multiprotocol' migration option will not be available.
1. Once you’re logged in to DobiMigrate, go to the Jobs module, and click on the New
button and select New migration.
2. Select the source file server and folder to migrate. Click Next.
3. In the Selected Jobs, Shares and Exports popup, select Multiprotocol in the Protocol
column:
72
DobiMigrate Online Help 6.4
4. In the Selected Jobs, Shares and Exports popup, select Multiprotocol in the Protocol
column. Click Next.
If the source server contains symbolic links, select SMB + symlinks over NFS migration
mode to ensure that all data and metadata are migrated through SMB; however,
symlinks is created through NFS because ONTAP does not support symbolic links
creation over SMB.
5. The general Job options display (refer to the Configure Migration Options section for
details). Click Next or Advanced options.
73
DobiMigrate Online Help 6.4
6. Click on the Advanced button to view the advanced options for multiprotocol.
Invalid SMB Filenames: select the behavior for when a file or directory can’t reliably be
accessed over SMB because of, for example, invalid characters in file names, or multiple
file names with different casing.
◦ Generate errors: to report an error. No corrective action is taken and the file is not
migrated.
◦ Migrate files using NFS only, generate error for directories: to report an error for
directories, fall back to an NFS only copy for files (SMB security descriptors and
alternate data streams will not be copied).
◦ Migrate files and directories using NFS only: to fall back to an NFS only copy for
files and directories (and the complete subtrees below them). SMB security
descriptors and alternate data streams will not be copied.
◦ Create over NFS, apply metadata over NFS and SMB: all symbolic links are
created using the NFS protocol. Metadata is copied using both the NFS and SMB
protocol. This mode must be used for file servers that do not support the creation
of symbolic links over SMB but support symbolic links over NFS.
◦ Create and apply metadata over NFS: all symbolic links are only copied using the
NFS protocol. No operations are done over SMB. This mode must be used for file
74
DobiMigrate Online Help 6.4
8. The system runs prechecks and identifies any errors or warnings. Click Finish to start
the migration.
9. The first scan begins and can be tracked under the Jobs > Running tab. The Protocol
column identifies the multiprotocol migration.
10. Click on the magnifying glass to view the progress in real time.
75
DobiMigrate Online Help 6.4
11. When the scan is complete, click on the scan details icon (Source checkbox) to look at
the migration logs.
12. The Source Scan Results lists the source paths that were scanned.
76
DobiMigrate Online Help 6.4
13. Click on the Running tab again to see that the first copy is running. Data is copied using
one protocol (NFS). Metadata is copied using both.
14. Click on the magnifying glass to view the migration activity log. Data and metadata are
both copied using both NFS and SMB protocol.
Below are examples of errors that could occur when running a multiprotocol migration. Errors
are identified with a red exclamation icon. Click on the icon to view error details.
77
DobiMigrate Online Help 6.4
• Two files and two directories have the same name but in different cases.
Errors can be mitigated by reviewing the Job Options > Advanced options for multiprotocol,
which include:
In computer and information technology, a WORM device (write once, read many) is a type of
data storage that allows data to be written to a storage medium only one time. Once it is
recorded, the data cannot be erased, modified, or re-encrypted. Because of this, data stored on
WORM storage are immutable and permanent where it can be read or accessed by users, but
they cannot alter it in any way.
WORM is the practice of saving data that cannot be deleted until some period of time has
elapsed. It’s intended largely to help organizations meet regulatory and legal retention
requirements on their datasets, such the IRS, SEC, or European Parliament. Setting up a
migration with WORM data is like setting up a normal migration, except that the migration type
WORM must be selected when adding a migration path.
All migration states, including switchover, are the same as for a normal migration and the
digest calculations are also performed to ensure integrity of all migrated data and metadata.
Note however, that during the migration iterations of a WORM migration, the retention period
and the commit state are not transferred to the destination so that files are not locked until the
switchover time on the target.
78
DobiMigrate Online Help 6.4
1. Go to the Jobs module and click on the New > New migration button in the bottom right
of screen to open the Select source path(s) window.
2. First, select the source file server from the drop-down at the top, and then select the
paths to be migrated.
3. Select the path(s) that you want to include in the migration. You can migrate at the root
level or at any subdirectory level.
4. Once you have made your selection, click Next to go to the Select target path dialog.
Here you can select the target file server and location and create additional directories
to serve as the root level for the migration data.
79
DobiMigrate Online Help 6.4
5. When you have selected the migration paths, a dialog opens listing the selected
migration paths and corresponding security information and migration-types. Verify
that the Protocol is correct. In case there are multiple options, you can select the correct
protocol using the drop-down in this column. The following values are possible:
6. Click Next when all the values are correct. DobiMigrate displays an overview of the Job
options: make sure the relevant job options are selected, then click Next (refer to the
Configure Migration Options section for details).
80
DobiMigrate Online Help 6.4
81
DobiMigrate Online Help 6.4
9. From the Running tab, the status and activity of the migration can be tracked. After
setting up a WORM migration, the Type column for this migration path displays WORM.
10. Click on the magnifying glass icon to view details of migration activity. When the scan is
complete, the Status is marked as Success.
82
DobiMigrate Online Help 6.4
A switchover is a step in the migration flow during which a final scan of the data is done, and
any changes are copied one last time from source to target.
When a WORM migration has been switched over, the data is fully in sync between the source
and target and can be committed on the target side.
Do not use the target side until the commit operation is finished.
1. From the Switchover module, click on the Start new event… button.
3. When scanning is complete, the path moves to Copy Completed and the Summary tile
displays 100%. Click on the Summary tile to access and select the Close Event… button.
4. Select the Move to Copy WORM Phase option from the Decision column.
83
DobiMigrate Online Help 6.4
5. When a WORM migration has been set up, the WORM Commit* tab displays which
shows the state of a WORM migration after switchover. To do the actual Copy WORM
operation, click on the WORM Commit* tab, select the paths that are in the WORM
state waiting for manual start and click the Start button.
7. Select one of the following options and click Next. (The option definitions display when
selected.)
The first option will re-hash every file to save the SHA-1 Digest and
will take a very long time (as long as the First Copy) to complete. Since
the SHA-1 Digest is already in the Chain of Custody report, it is
84
DobiMigrate Online Help 6.4
When there are no issues and the Copy WORM has finished, you can finish the WORM
migration by selecting the path(s) and clicking the Finish button. The WORM migration
will move to the Finished state.
9. To view the results of the migration, click on the magnifying glass to view the iteration
details. When there are issues, you can view these by clicking on the error icon next to
the iteration number.
85
DobiMigrate Online Help 6.4
◦ WORM report: provides data regarding the target retention and commit success.
From the iteration detail view, select the Switchover iteration in the top section.
Next to Status, click the icon to the right to view the Chain of Custody report.
The Chain of Custody can also be downloaded from the Finished tab by clicking on the
‘blue ribbon’ icon.
86
DobiMigrate Online Help 6.4
12. The second report is the WORM report. Go back to the Iteration detail view and select
the Copy retention iteration in the top section. Then in the bottom-middle section
where you see “Copy WORM”, click the icon next to Done.
87
DobiMigrate Online Help 6.4
• For the path /vol/vol_ntfs the next iteration will start within 6 hours and 56
minutes.
• The migration for /vol/vol0 will only start after manual intervention.
The orange color indicates the paths that are waiting for an action to proceed.
Irrespective of the configured iteration schedule setting, you can start a new iteration
immediately by selecting the migration path and clicking the Start button under the table. For
more information, see Manually start a migration iteration.
Use the Pause and Resume button to manually pause and resume the iteration of a migration
path.
Use the Options to configure the migration options or to change the migration protocol.
88
DobiMigrate Online Help 6.4
Click the magnifying glass to view more details of the iterations for a migration path.
You can start the next iteration or a special iteration of a migration path by selecting the path
from the list and clicking the Start button under the table. The start iteration wizard will first
ask you to specify which type of iteration you would like to start:
• Incremental: The goal of an incremental iteration is to detect if any changes were made
on the source since the previous iteration and if there are any, apply them to the target.
▪ Normal: Applies all source changes that occurred since the previous
iteration to the target. This option allows you to manually trigger the same
type of iteration that is triggered automatically based on a defined
schedule.
▪ Force metadata copy: Recopies all metadata from source to target for all
items in the migration, even if it has not changed. The actions of a normal
iteration, as described in the previous bullet are also applied.
• Retry items with errors (xx): allows you to run an incremental iteration limited to a
subset of items that gave errors during the migration, instead of a normal iteration that
runs for all items. The number between the parentheses is the total number of errors in
this migration. The Selective Retry option can be used:
a. Select the Retry items with errors (xx errors) iteration type and then click
Next.
DobiMigrate will then display a list of all the items that gave errors (limited
to a max. of 5000), with the possibility to narrow the list of items down by
89
DobiMigrate Online Help 6.4
filtering on a particular string or via the filter button that allows you to filter
different criteria such as on a certain category of errors.
b. Select one or more of the errors, and click Next. You can also specify a filter
to display a subset of the errors.
c. Confirm you want to retry the selected errors, and click Next. The iteration
will then be started and all the selected errors will be retried.
All the errors will be retried. However, not all data will be
recopied. For example, if the command that failed was a
metadata copy then only that metadata copy will be
redone, and the data will be kept. Another example is if the
command was to create a directory or delete a file, then
that will be retried. In addition, this option will not
overwrite files that have been marked as ‘modified
externally’.
• Verification: A verification iteration compares content and metadata of all items in order
to check that they are consistent and that there is no data corruption.
▪ All which will verify that there is no data corruption nor metadata
inconsistencies.
▪ Only content which will only verify that there is no data corruption.
▪ Only metadata which will only verify that there are no inconsistencies in the
metadata.
When an option has been selected and the migration iteration has started by clicking Next, you
90
DobiMigrate Online Help 6.4
can view the results in the Iteration details window (see Iteration Details View).
• SMB shares
• NFS exports
In the example below, the Exports tile is red which indicates that a migration has been defined
for this source/target file server pair, but there are issues with some (or all) configuration
settings. In this case there is one issue:
When you click on the Exports tile, a more detailed view on the discovered settings is opened.
91
DobiMigrate Online Help 6.4
The Target status column shows the status of the share/export on the target server:
• Share/export not found on target: migration has been defined but no corresponding
share/export is found on the target file server.
• OK (read-only): an export or share configuration has been created on the target file
server but user(s) and group(s) have been granted read-only access.
• OK (switched over): an export or share configuration has been created on the target file
server and all rights have been inherited from the source file server.
• Multiple target paths: the source path maps to multiple target paths (for example on a
case-insensitive file system).
• Unsupported target type: the target does not support the creation/transfer of share or
export settings.
• Error determining target path: there was a problem determining the target path. For
example, scanning of the target file system failed.
• Target path does not exist: the target path does not exist.
• Unsupported source type: the source does not support transferring of share settings.
• Existing share/export: Share/export already exists on target, but no action has been
taken yet.
92
DobiMigrate Online Help 6.4
The Compatibility column shows if there are issues with the transfer of the share/export
definition from the source to the target platform. The color of the exclamation mark in
the Action column indicates whether it is a warning (yellow) or an issue that needs to be
resolved (red).
• OK: There are no issues for converting the source share/export to the target file server.
• Unsupported target type: The target does not support the creation/transfer of share or
export settings.
• Conversion issues: There are minor issues converting the share/export to the target file
server but conversion is possible (mostly not 100% correct because the target does not
support all features from the source). To investigate the issues, click on the yellow
exclamation icon.
• Conversion errors: There are known errors converting the share/export to the target
filer. The current share/export definition cannot be converted to the target. To
investigate the errors, click on the red exclamation icon.
• Unsupported source type: The source does not support transfer of the share settings.
Click on the Action icon to open a popup window that lists all compatibility issues.
In case there are configuration settings with an issue (Not found on target), select the setting
from the list and click the Fix button.
The window below shows an SMB share that is missing on the target fileserver.
93
DobiMigrate Online Help 6.4
There are multiple options for the creation of the target shares:
• Keep the same name as the source shares. In this case, click the Create button. This will
create the shares on the target file server and DobiMigrate will check their availability.
When OK, the status for these shares is flagged green with the message OK.
• Add a prefix to the selected shares. To do this, enter the prefix string in the text field next
to the Set prefix button and when done, click Set prefix. This will add the prefix to the
target share names. In case you have set the wrong prefix, use the Revert button to
reset the names to match the source share names. When done, click the Create button
to create the shares on the target file server.
• Change the name of the target shares one by one by editing the Target Share field.
When done, click the Create button to create the shares on the target file server.
When clicking the Create button, a dialog asks to confirm the share name and the matching
target path. Click OK to create the shares/exports on the target file server. When done, the
status of the shares/exports is updated.
Click Close to go back to the overview window which shows an updated overall status. If
needed, select the checkbox Show resolved settings to display the newly created share.
In case there are configuration settings with an issue (Not found on target), select the setting
from the list and click the Fix button.
94
DobiMigrate Online Help 6.4
The window below shows which exports that are missing on the target file server.
When clicking the Create button, a dialog asks to confirm the creation of the export and which
access rights these should be granted:
95
DobiMigrate Online Help 6.4
When clicking OK, the selected export will be created on the target file server. When done, the
export status is updated to OK (read-only). Click Close to go back to the overview window
which shows an updated overall status. If needed, select the checkbox Show resolved
settings to display the newly created exports.
The Source Status column, which only displays a value for a defined migration, indicates the
access level to the share/export on the source file server. It’s good to do this during a
switchover to ensure that nobody makes changes to the source data while a switchover is
taking place.
• Original: this is the access level as it has been set for the share/export prior to any
manipulations done by DobiMigrate.
• Read-only: this value is shown after the share/export has been set to read only via
DobiMigrate.
• Unsupported: indicates that DobiMigrate cannot set access to this source to Read only.
To change the access to the source shares/exports, select the share/export and click
the Modify source access button. In the dialog that opens you can select the shares/exports
for which that you want to change the access. Select the option Set shares access rights to
derived read-only if you want DobiMigrate to put the selected shares/exports to read only.
When you select a share and click the Show details button, extra information is displayed in
the bottom frames.
96
DobiMigrate Online Help 6.4
The left frame provides settings information for the selected configuration on the source file
server. For a share, it shows the user(s) and group(s) that have access to that share together
with their permissions. This information is collected from the source file server when you open
the Settings window. The frame on the right lists the details of the share that has been created
by DobiMigrate on the target file server:
The settings that generate warnings/errors with the conversion from source
• Share Name: the name of the share as given at the time of creation by DobiMigrate or
the name that was given by the person who created the share outside the DobiMigrate
environment.
• Home Directory Provisioning: these are the home directory settings that were applied
when creating these shares. DobiMigrate uses the default settings of the target file
server. Possible values for Isilon are: Path variables will NOT be expanded, Path
variables will be expanded, Directories will NOT be automatically created, and
Directories will be automatically created. For more information on these settings, please
consult the Isilon online help.
• Users and Groups: the list of user(s) and group(s) that have been given read-only
access to the share.
97
DobiMigrate Online Help 6.4
In case the project team has decided not to migrate a specific migration path
by excluding it from the scope, the matching share(s) are also excluded
automatically in the Settings view. The shares are labeled green with the
text Excluded path.
Whenever shares have been created directly of the file server, these will show
up in the file server list.
When you select an export and click the Show details button, extra information is displayed in
the bottom frames.
The left frame provides settings information for the selected export on the source file server.
For an export, it shows the clients that have access to that export together with
their permissions. The frame on the right lists the details for the export that has been created
by DobiMigrate on the target file server. Please note that the type of configuration settings that
can be migrated and their detailed representation will vary according to the source and target
file server types.
Shares and exports are always given read-only access when they are created before the
actual switchover. This is to avoid that users or groups start to write or manipulate data on the
new shares or exports while they still have full access to the data on the old environment. This
could lead to confusion about which shares and exports should be used during the migration
project. The read-only access has been put in place to allow users and groups to validate
whether they can access their data on the target file server before a switchover is finalized.
One of the final steps of the switchover is to transfer all rights from the source file server to the
98
DobiMigrate Online Help 6.4
The creation of shares can be done at any point during the migration process as soon as the
matching target migration path is known. Whenever shares/exports are added to the source
file server during the migration project, the shares/exports are shown when the settings
window is opened.
In case you have created shares/exports on the target file server incorrectly, you will have to
contact the file server administrator to have these shares/exports removed.
The Rescan button under the overview pane instructs DobiMigrate to rescan the settings that
can be migrated on the source file server. Use the Mark as Resolved, Mark as
Unresolved buttons to explicitly flag an issue or pending decision as resolved in case the issue
does not apply, or to revert from that decision. This allows the project team to clear any
irrelevant pending decisions and issues.
In case the project team has decided not to migrate a specific migration path
by excluding it from the scope, the matching configuration settings are also
excluded automatically in the Settings view.
99
DobiMigrate Online Help 6.4
You can control what’s displayed in the dashboard by selecting a predefined filter from
the Group by dropdown list and/or entering details in the Filter field.
Green indicates that the job is running without issues and that no action is required.
• The user needs to start the next process step or iteration for a job that is running in
manual mode. The process tile containing a manual job will be orange when it is in
waiting mode. A process can also be in waiting mode because the user has defined
specific iteration schedules.
• Not all job paths have been set up for a specific source/target storage server pair.
• Not all shares or exports found on the source storage server have been created on the
target storage server.
Red indicates that there are issues that need to be resolved in order to successfully complete
the job. Examples are:
• Errors that pop up during the execution of a process step and that are out of control of
DobiMigrate.
• You try to complete switchover windows for shares and exports that do not exist on the
target environment or for which no explicit decision was taken to migrate them.
In the Shares & Exports module, you can view interactive tiles under Dashboard, and use
the Filter field to select source and/or target names.
100
DobiMigrate Online Help 6.4
Once Shares or Exports is selected, you can select a source/target storage server pair and,
using the buttons at the bottom of screen:
• Rescan: Rescan the fileserver’s metadata and recalculate the target status. This
updates share information in the DobiMigrate UI, for instance, if a share had been
changed manually.
• Modify source access: Change source shares and exports to read-only or export scripts
to change the source access. This is typically used during a switchover event so that
source data is not changed.
• Restore source access: Change source shares and exports from read-only back to their
original state. It is the opposite action to the Modify source access button.
• Mark as Unresolved: Removes the Resolved status. It puts the share or export back into
its original status. (Note: This can be used to unresolve any “Undecided” shares or
exports.)
• Mark as Resolved: Mark the selected shares and exports as resolved regardless of their
status. This is typically used when the end user changes the shares/exports on the
storage servers manually and want to show in DobiMigrate UI that the share/export has
been dealt with. (Note: This can be used to resolve any “Undecided” shares or exports.)
• Fix…: Allows the addition of read-only shares or exports on the target server during
migration. This is useful to create read-only access to users on the target to verify the
target data ahead of a switchover, without being able to change it.
101
DobiMigrate Online Help 6.4
automatically unless the user explicitly has chosen to run it in manual mode.
The rows are groups of migrations as selected in the Group by dropdown list. Clicking on a tile
will show all migrations from the grouping in the selected state.
When no migration paths are processed in a specific process step, then the
view can be empty.
While the dashboard provides an overview, the four stages of migrations can be viewed under
the following tabs:
For each running process, progress details are displayed in the dashboard on tiles, and each
tile gives an overview of the following information:
• Planning: a complete overview of the migration paths, and the current migration status
for each of those paths.
• First Scan: the migration paths for which the first scan is running.
• First Copy: the migration paths for which the first copy is running.
• Steady State: the migration paths for which scans are executed on a scheduled basis.
Any (meta)data differences between source and target are then migrated. A migration
path will remain in this state until it is scheduled for switchover.
• Post Finish: the migration paths for which additional actions are being executed, such
as verify content, for example.
102
DobiMigrate Online Help 6.4
7.2.1. Planning
This view provides a clear overview of the migration of the different directories during a large-
scale migration. It shows the different migration paths that have been defined, what the status
of their migration is, and provides the necessary insight to ensure that all of the directories
have been migrated successfully and that none have been missed.
As you can see, DobiMigrate will retrieve and present the sibling directories on each of the
levels for the defined migration paths.
The key feature of this view is the Show coverage option that you can select by clicking the
settings button in the top left of screen and selecting the Show coverage option. By turning
this option on, the view is supplemented with a visual indication of the completeness of a
migration per path. We call this path coverage, and it is displayed in the status column on this
view.
When Show coverage is selected, which is the default, DobiMigrate will use the status column
to provide path coverage information. Each row of the planning table will display one of the
following path coverage values:
• migrate: a migration has been defined for this directory. The directory and all its
contents will be migrated to the target server.
103
DobiMigrate Online Help 6.4
• excluded: this directory has been marked as excluded from the coverage report, and will
not be migrated.
You can use the Exclude from coverage report shortcut menu (right-
click option) to exclude directories and files from the coverage report.
• incomplete: the contents of this directory will only be migrated partially. One or more
migrations exist for child directories of this directory, but not for all child content.
• complete: all child directories of this directory are either part of a migration or have been
marked as excluded.
You can filter this view on paths by clicking on the button at the top left of the screen and
selecting the root(s) whose paths you would like to see, or by using the filter wizard by clicking
on its button in the top right of the screen.
7.2.2. Running
This includes first scan, first copy, and steady state. You can choose what to display through
the View and Group by dropdown lists.
7.2.3. Finished
This shows the paths that have been switched over. Choose what’s displayed through
the Group by and Paths dropdown lists.
This includes further actions that may be required such as verification, fixing and metadata for
example. Choose what to display through the View, Group by, and Paths dropdown lists.
7.2.5. Statistics
This presents a visual overview of how the source data is structured and how it might impact
migration speeds. The view is updated after each iteration to allow for any changes that may
have occurred since the previous iteration.
Use the dropdown options of View, Job paths and Jobs to refine your view.
104
DobiMigrate Online Help 6.4
You can also access the tabs by clicking on an active tile under its corresponding column.
This view is called the Iterations Details View. It allows you to monitor running migrations
closely by presenting all of the iteration’s details, and allows you take any actions needed
during the running of the migration.
The source and target paths of this migration are presented at the very top of this view
together with the following action buttons.
105
DobiMigrate Online Help 6.4
• Statistics: takes you to a view of the statistics and histograms that you can consult
regarding this migration.
• Start: allows you to manually start a new iteration for this migration, see Manually start
a migration iteration for more details.
• Options: allows you to view/edit the options of this running migration, see Configure
Migration Options for more details.
The iterations details view consists of the following five main parts that are explained in the
sections hereafter:
• Summary is the table at the top of this view and presents a summary of the most recent
iterations for this migration, including the First Scan and First copy iterations.
• Iteration details, directly underneath the Summary on the left, presents the main
characteristics of the iteration that is selected in the Summary, including histograms
that present the migration data graphically.
• Scan details, underneath the Iteration details on the left, displays the activity that
happened for the selected iteration of the migration during source and target scan.
• Operation details, underneath the Iteration details in the middle, displays the activity
during the copy phase of the selected iteration.
• Error details, underneath the Iteration details on the right, aggregates the errors from
the Scan and Operation details, and provides error categorization.
106
DobiMigrate Online Help 6.4
• Status: indicates the current status of the iteration. The possible values are:
◦ Dry Run
◦ Switchover
◦ First Scan
◦ First Copy
◦ Recopy metadata
◦ Verify content
◦ Verify metadata
◦ Verify all
◦ Copy WORM
◦ Verify WORM
• To Copy: the total amount of capacity to be copied from source to target (may include
verification depending on the validation options).
• Capacity copied: is the amount of data that has already been copied (may include
verification depending on the validation options).
• Skipped: the number of operations skipped. These are the items that were part of the
iteration that did not meet the set minimum age criteria for that iteration. If for instance
107
DobiMigrate Online Help 6.4
the minimum age has been set to 1 day and DobiMigrate comes across a file that was
stored 3 hours ago, then it will be skipped in this iteration.
• Capacity Skipped: the amount of data that is not copied based on the minimum age
setting.
• Errors: the number of errors that occurred during this iteration, and in between
parentheses the number of errors that are recurring.
Some types of errors block the iteration. When this is the case, the
You can view additional details of these errors by looking at the Error Details table.
You can select a specific iteration by clicking on one of the rows in this table. The Iteration
details view underneath this table will then present details of this selected iteration.
The table underneath the Summary is the Iteration Details table. This table displays the details
of the iteration that is selected in the Summary.
The left side of the Iteration details lists basic statistics. The type of statistics that is shown
here depend on the status of the iteration. The statistics are described in more detail further
on in this paragraph. The right side of this view presents a number of Histograms, which give
you a graphical representation of the migration. The four histograms, which you can cycle
through using the arrows on each side of the displayed histogram are: Directory size, File
modification time, Directory depth, and File size, see Scan Statistics for more information.
108
DobiMigrate Online Help 6.4
As long as the status of the selected iteration is Running, the following statistics are displayed:
• Status: this is the status of the iteration. This status matches the one displayed in the
Summary for this iteration, but gives more detail when it is running by presenting sub-
steps, the total number of sub-steps, and the current sub-step number. This can give
you an idea of how the iteration is going.
When you click on the icon next to this status, the Activity view is opened which shows
the details of the current activity. See Activity view.
• Duration: the exact duration that the iteration has been running up till now.
• Throughput: the actual number of items scanned per second. An item can be a file,
directory, symlink, or metadata.
• Progress (throughput): the percentage of items that have been processed out of the
total amount.
• ETA (throughput): the estimated time of completion based on the number of items
processed so far and the current throughput.
• Progress (bandwidth): the percentage of capacity copied out of the total amount.
• ETA (bandwidth): the estimated time of completion based on the capacity processed so
far and the current bandwidth.
When the selected iteration is completed with Success, the following information is displayed:
• Status, start and stop time of the iteration. These are the same values as displayed in
the Summary above.
When you click on the icon next to this status, the Chain of custody results is opened
which shows the chain of custody data related to this iteration. See Chain of Custody
Results.
• Duration: the exact value for the total duration of the iteration..
• Source Scan Duration: the total time DobiMigrate took to scan the path on the source
storage server.
109
DobiMigrate Online Help 6.4
• Target Scan Duration: the total time DobiMigrate to scan the path on the target storage
server.
• Operation Duration: the total time DobiMigrate took to execute the operations.
An iteration’s status is set to Failure when one or more blocking errors occurred while it was
running. The following information is displayed:
When you click on the icon next to this status, the Chain of Custody results view is
opened which shows the chain of custody data related to this iteration. See Chain of
Custody Results.
• Failure reason: presents the blocking error that caused the iteration to fail.
• start and stop time of the iteration. These are the same values as displayed in the
Summary above.
• Duration: the exact value for the total duration of the iteration is shown here instead of
the rounded value in the Summary.
• Source Scan Duration: the total time DobiMigrate took to scan the path on the source
storage server.
• Target Scan Duration: the total time DobiMigrate to scan the path on the target storage
server.
• Operation Duration: the total time DobiMigrate took to execute the operations.
might take some minutes before the statistics are refreshed. DobiMigrate
waits for these operations to complete before updating the statistics.
110
DobiMigrate Online Help 6.4
The Scan Details table presents a summary of what has been discovered during the scanning
phase of the selected iteration. It lists the number of items found on the source as well as on
the target. This information gives you an idea of what the migration entails, which types of
items that will be processed, and the progress of the migration.
• Other: the number of other items (symbolic links, junctions, sockets, pipes..) that have
been discovered
• Exclusions: The number of items that matched the exclude pattern of this migration
• Errors: the number of items that caused an error during the scanning phase of this
iteration.
You can view a complete listing of the scan results by clicking on the small icon next to either
the Source heading or Target heading to get the complete listing of either the Source or target
scan results respectively. Note that the icon is only available when there are results for the
selected iteration.
You can also retry specific errors in the Target Scan Results table by selecting the error,
clicking the Details button and then clicking the Remigrate button.
111
DobiMigrate Online Help 6.4
The Operation Details table presents a summary of the activity during the running operation
phase. It lists which operations have been executed and how many times they have been
executed during this iteration.
You can view a complete listing of the operations that need to be performed and that have
been performed by clicking on the small icon next to either the To do heading or Done heading
respectively. The icon is only available when relevant.
Note that for an iteration of the type Verification, this table is replaced with the Verification
Details table.
The Error Details table aggregates and displays the number of errors encountered during
source scan, target scan, and the execution of the s.
The most common errors have been classified in the following categories:
112
DobiMigrate Online Help 6.4
• Permission denied: The DobiMigrate proxy shown has not been given the permission to
perform the operation.
• Out of space: The target storage server does not have enough capacity to accept the
file.
• Not found: The file is not found on the source storage server. This can happen when the
file was deleted between the scan operation and the subsequent copy process.
• In use: The file on the source storage server has been locked and is not available to
DobiMigrate to read it for a copy operation.
• Timed out: The operation timed out because the DobiMigrate Core did not get a timely
response from the DobiMigrate proxy.
Target-modified detection uses the modification time, maintained by the storage server.
This information is read-only and is incremented by the storage server every time an
item is changed, for example, when a file is written to, a directory has an item added or
deleted, and so on. DobiMigrate records the modification time every time it copies a new
item or updates an already migrated item. If content was changed on the target, then
the modification time will not match the recorded change time value. This indicates to
DobiMigrate that the item was externally modified. Normally, this would also mean that
those changes would be overwritten with the changes on the source. To prevent this
issue, DobiMigrate triggers a Modified externally error. Any item with a Modified
externally error will no longer be changed by DobiMigrate and will not be kept in sync
with its corresponding source item.
What can you do in case you encounter a modified externally error? Examine the
cause for external changes on the target. What caused the modification time to be
incremented? Who made the change? Was it a process or a person? Some of the more
obscure reasons for the change are: file changes by a virus checker, attribute changes
by a backup software, quota setting on a target filesystem, and so on. Verify that the
changes are valuable or not. If they are, make sure you take a backup before you
continue.
To resume source to target syncing for items that have the "Modified externally" error,
you can:
113
DobiMigrate Online Help 6.4
With this option, DobiMigrate will resume keeping target items in sync with the
source, and any changes done on the target will be overwritten.
• Other: None of the known categories apply. In this case, you can find out more about the
error by viewing the complete listing.
You can view a complete listing of the items with errors by clicking on the small icon next to
one of the column headers. The icon is only available when relevant.
The Verification Details table: lists the results of the chosen verification option for the selected
iteration. This table is only presented when viewing the details of a verification iteration. In this
case it replaces the Operation Details table, which is less relevant during a verification
iteration as the operations will always be a type of verification.
• Failed: The verification itself failed; the item could not be read because of permission
issues, network not available, or other reasons.
• Not verified: The item has not been verified. This column should always be empty. Any
value in this column indicates an issue.
• To be copied: The content of the item has been modified on the source storage server
and will be (re)copied to the target storage server during the next iteration.
• To update metadata: The metadata of the item has been modified on the source
storage server and will be (re)copied to the target storage server during the next
iteration.
• To be deleted: The item is no longer available on the source storage server and will be
deleted on the target storage server during the next iteration.
• Content mismatch: The timestamps of the items are the same but either the size or the
content is different between the source and target storage server. This has to be
investigated in more detail and will not be fixed automatically during the next iteration.
• Metadata mismatch: The timestamps on the metadata between the source and target
storage server indicate that the source metadata has been copied to the target storage
server, but there is still a mismatch between the actual metadata. This has to be
114
DobiMigrate Online Help 6.4
investigated in more detail and will not be fixed automatically during the next iteration.
Not all storage servers store metadata the same way. If you migrate
between different storage server types, it is very likely that the
• Modified externally: The item has been modified on the target storage server. This may
indicate that there were changes on the target. This has to be investigated in more detail
and will not be fixed automatically during the next iteration.
The validation is done hierarchically; if some criteria are not met for an item,
the validation will stop.
You can access the additional information by clicking on the small icons you may find next to
the Status in the Iteration Details table or by clicking on one of the small icons in the headers
of the Scan Details, Operation Details, Error Details, and Verification Details tables. It is in the
description of those tables that you will find when/which additional detailed information is
available.
The additional Activity view displays detailed information on all the activity of the iteration.
You can access this view by clicking on the small icon next to the status, when the iteration is
running in the Iteration Details table.
115
DobiMigrate Online Help 6.4
This additional view shows the details of which operations are being executed for this running
iteration. DobiMigrate presents a list of each and every operation it is executing and specifies
whether it is on the source or target, through which proxy, with which protocol, etc.
You can specify which columns are included in this view with the icon at the
far right of the header row.
This view shows a maximum of 1000 entries, sorted on age. You can export the complete
overview of activity, for this iteration, using the export button at the bottom left of the screen.
Occasionally it might look as if nothing is happening, the most common causes are:
• the migration is moving from scan activity to operation activity (solution: wait), or;
• the proxies are not running (solution: start up the relevant proxies).
116
DobiMigrate Online Help 6.4
Key to a Chain of Custody is proving the integrity of an item throughout its lifetime. This can be
proven through the use of content digests. A content digest is calculated for every individual
item encountered during the migration and logged in a Chain of Custody record along with its
identifiers. A second content digest is calculated immediately after migrating the item, and
compared with the first to prove the integrity of the item after migration.
• Path: which helps you identify the item whose integrity is evaluated.
• Status: in other words, the result of the integrity evaluation. The possible values are:
◦ In sync (S): which means that the item is the same on source and target.
◦ Excluded (E): due to exclusions defined in this migration’s options. See Configure
Migration Options for more information.
◦ Modified externally: indicates the files on the target have been modified by users
before a migration has been switched over. Files marked with this flag will not be
synchronized with the source because updated files will be overwritten by the
content in the source.
◦ Retained: which means that the item would have been deleted to be in sync with
the source, but was retained due to the no deletes option defined in the
migration’s options.
◦ Out of sync: which means that a change on the source was not propagated to the
target, see further details in Info.
117
DobiMigrate Online Help 6.4
◦ Unknown: in this case you should have a look at the details in Info.
• Info: which gives additional information in case the status is Excluded, Out of sync or
Unknown.
You can export the Chain of custody results, for this iteration, using the export button at the
bottom left of the screen.
Each of the information tables at the bottom of the Iteration Details View (Scan Details,
Operation Details, Error Details, and Verification Details) provides links in the headers, given
there is data in the columns, to get additional information as the tables on the main Iteration
Details View only provide a summary.
The additional information is provided by DobiMigrate to allow you to follow up the activity of
the iteration in very fine grained detail. This is typically useful during trouble-shooting.
You can specify which columns are included in these additional information
views with the icon at the far right of the header row, and you can export the
complete overview of activity, for this iteration, using the export button at the
bottom left of the screen.
To export a list of issues, locate your migration(s) under their corresponding tab (Planning,
Running, or Finished), select them, and click on the error button in the bottom left corner:
• Filter: select a filter to export all failures or a subcategory based on the definition of
exception categories (see below).
• Destination: you can download the list to your desktop via a browser or put the exported
file in DobiMigrate’s dropzone.
118
DobiMigrate Online Help 6.4
When you select Based on the Exception Category, an extra table is shown allowing you to
make a more granular selection for the records to be exported.
Select which categories you want to export, confirm the Destination and press the
Export button.
For large files, it is advised to use the dropzone otherwise your browser could
be locked for a while when the information is collected and downloaded.
Depending on the amount of entries you have selected, the files size can be
GBs or TBs.
119
DobiMigrate Online Help 6.4
• storage server: Name of the storage server for which the records are exported.
• Operation: The operation performed for the respective path. The values for Operation
can be:
◦ COPY FILE METADATA (CFM): Copy of the metadata of the file (for instance a
change in the file attributes)
◦ COPY DIRECTORY METADATA (CDM): Copy of the metadata of the directory (for
instance a change in the directory attributes)
◦ DELETE FILE (DF): Deletion of the file on the target storage server
◦ OK: Operation is successful.
◦ Permission Denied: The respective DobiMigrate proxy is not given the permission
to perform the operation.
◦ Out of Space: The target storage server does not have enough capacity to accept
the file.
◦ Not Found: The file is not found on the source storage server. This can happen
when the file was deleted between the scan operation and the subsequent copy
120
DobiMigrate Online Help 6.4
process.
◦ In Use: The file on the source storage server has been locked and is not available
to DobiMigrate to read it for a copy operation.
◦ Timed Out: The operation timed out because DobiMigrate Core did not get a
timely response from the DobiMigrate proxy.
◦ Other: In case none of the known categories apply, the exception field will
contain more details.
• Exception: The exception message that came with the processing of this path. This
message provides extra context to help root cause and resolve the issue.
• Stack Trace: For an exception, the stack trace can be used to determine where the error
occurred within DobiMigrate.
The statistics page gives you a visual overview of how the source data is structured and how it
might impact migration speeds. The view is updated after each iteration to allow for any
changes that may have occurred since the previous iteration.
There are five different histograms used to display specific data information:
• File size: This groups the files into different size categories and shows the number of
files in each category. It also shows the total capacity in each category.
• Directory size: This groups the directories based on the number of files they contain.
The numbers of files per group are also shown.
• File modification time: This groups the files based on when they were last modified.
• Directory depth: This groups the directories based on the number of subdirectories they
contain, and shows the number of directories per group.
121
DobiMigrate Online Help 6.4
• File type categories: This categorizes files per file type and also shows the number of
files per file type, along with their total capacity.
You can also get an overview of the ten largest files and directories:
Scan statistics also appear under Migrations > Running (Histogram with a
carousel interface).
When a migration path completes the first scan and copy it is in steady state. At this time, the
system allows the source system to operate normally and be available for both reading and
writing during most of the migration period.
During the very final stage, switchover, the source system must go to maintenance mode or at
least become read-only. During the switchover, users and applications cannot use the source
or target file servers. Changing the source to read-only is done by moving the source shares
and exports to read-only state, which is detailed below.
122
DobiMigrate Online Help 6.4
During the switchover window, a final synchronization is performed between the source and
target. Target shares and exports can also be configured by copying source shares and
exports definitions on the target.
When a switchover is finished, the target system becomes the authoritative source for the
data. Its shares and exports are made writeable, and users have to reconnect or be
reconfigured to use the target system. At the end of the switchover, users and applications are
redirected outside of the system to the new environment.
No additional steady state iterations will occur for the paths that were part of the switchover,
and the source system should no longer be used for this dataset.
Switchover Events
• Switchover Errors
• Cancellation
NOTES:
• The Minimum Age setting is not applied for dry run or switchover, meaning that any files
that were skipped due to Minimum Age setting will be migrated at this point.
• Any files that are skipped due to the Skipped file pattern option are migrated at this
point.
123
DobiMigrate Online Help 6.4
• During a switchover, it is not possible to set the exports to read-only on a source generic
NFS file server. (API call to do this is not available.)
• It is not possible to apply the source export settings on a target generic NFS file server.
(API call to do this is not available.)
• You cannot change the protocol during a switchover. You can change the protocol only
during the iterations before a switchover.
8.2. Planning
If there are no unexplained errors, it is time to plan for the switchover. During the switchover,
users and applications cannot use the source or target file servers. That is why you typically
do a switchover during a service window. The most critical aspect when planning the
switchover is to know its duration so you can tune and optimize the scope of each service
window. Dry runs provide that insight for a switchover.
To be able to perform a dry run or a switchover, you need to define a switchover group first. A
switchover group is a selection of migrations that is switched over at the same time.
1. From the Switchover module, select the Pending groups tab and click on the plus (+)
button.
2. Enter a logical name for the group. Click on the calendar icon to open the scheduler.
124
DobiMigrate Online Help 6.4
3. Click on a date/time and drag the box to set up the switchover time. Click OK.
4. The Parellism field can be left at the default value of 64. This determines how many
paths will be scheduled at the same time during the switchover. Click OK to continue.
6. Click on the add/edit paths icon to add the paths that are to be part of the switchover
group.
125
DobiMigrate Online Help 6.4
8. The path(s) are listed and dry runs can be set up by clicking the Add/Edit upcoming dry
run event icon.
9. Click and drag the desired dry run times and click OK.
126
DobiMigrate Online Help 6.4
Before initiating a switchover, a dry run can be used to estimate how long a switchover will
take. A dry run is a simulation of a switchover. A dry run can be set up to run automatically or
manually triggered as needed.
• The migration paths for the dry run are part of a switchover group. (See Switchover
Group content.)
The dry run automatically starts based on the selected start time, runs its course, and then
finishes. The results are displayed in the Completed dry runs block. At the start time of a
scheduled dry run window, the Current window tile in the Switchover view becomes active and
accessible.
Setting up a dry run is the same as creating a slot for a switchover, but it needs to be marked
as a dry run and have migration paths assigned to it.
1. Go to the Current event tab in the Switchover module and click on Start new event
button. (If an event is currently running, click on Close Event).
127
DobiMigrate Online Help 6.4
3. Select a switchover group from the Group dropdown list. If you need to define a
switchover group, choose Other in the dropdown. Then click OK.
128
DobiMigrate Online Help 6.4
• A scan and copy of the changes are done for the paths that are part of the dry run but
no shares or exports are altered. Once the dry run is finished, the paths are moved back
to steady state.
• A dry run detects any final errors that would potentially make a switchover fail.
• Paths that are part of the dry run are scanned as well as any changes copied from
source to target so that target is in sync with source. This activity is visible under
Scheduled.
• Once all paths are finished, the dry run automatically completes, and the results are
visible under the Completed dry runs block under the Pending groups tab.
• All the processes not directly contributing to the switchover/dry run are paused
automatically to provide maximum CPU and network support to the scheduled
operation.
129
DobiMigrate Online Help 6.4
NOTES:
• When the allocated time has passed, the dry run window is closed regardless of the
execution state of the migration path. The paths that are not finished will not have ETA
calculations available for planning purposes. In case the paths were executed prior to
the scheduled time, the window automatically closes and the end time of the slot is set
to the closing time.
• The amount of migration paths that will be executed in parallel is based on the value as
defined in the field Parallelism when creating the dry run window.
• The switchover ETA for a group is the average of the last three dry runs.
• As the dry run progresses, the tile colors might change indicating bottlenecks, delays or
issues. Click on the respective tile to get more information on the risks or issues that
occur.
Manual Switchover
1. Go to the Current event tab in the Switchover module and click on Start new event
button.
130
DobiMigrate Online Help 6.4
2. Select Switchover from the Type dropdown list. Select a switchover group from the
Group list, if available.
3. If you need to define a switchover group, choose Other in the dropdown. Then click OK.
5. Set the shares to read-only to prevent source data from being changed during the
switchover.
131
DobiMigrate Online Help 6.4
6. A pop-up window displays with the option to set shares access to read-only. Click OK.
132
DobiMigrate Online Help 6.4
8. The active user sessions on the file server need to be disconnected. The disconnect can
be managed externally, or within the UI using the connector icon for the related path.
133
DobiMigrate Online Help 6.4
11. The paths are moved to the Scheduled tile where the final scan and copy is performed.
This syncs the target with the source, and then moves the paths to the Copy Completed
tile.
12. If there are no errors, update the target shares and exports.
134
DobiMigrate Online Help 6.4
13. Click the Create button to create shares on the target file server replicating the share
name and permissions.
135
DobiMigrate Online Help 6.4
3. Click on the Fix button. Note: It is not possible to revert the target shares and
exports to their original system configuration. Make sure that the switchover is
executed successfully, and no issues remain, before doing these updates.
16. Close the switchover event. Select the Summary tile and click on the Close Event
button.
17. Verify that the paths are in the Move to Finished status under the Decision column.
18. Click OK to move the switched over migrations to the Finished tile on the Migrations
dashboard.
136
DobiMigrate Online Help 6.4
Automatic Switchover
• The migration paths for the switchover are part of a switchover group.
Cancel a Switchover
A switchover can be cancelled after the last scan and copy have occurred but before the target
shares and exports have been updated.
An example of why you might want to cancel a switchover is that an issue was found that
requires you to revert to Steady State so the issue can be resolved. This means that the source
shares and exports need to be put back to their original state. Additionally, the migration paths
that are part of the switchover need to be moved back to Steady State .
1. From the Switchover > Current event window, click on the Share Settings and Export
Settings tiles to restore them to their original state.
137
DobiMigrate Online Help 6.4
2. Click on the Restore Source access access button, respectively. Click OK to complete.
3. Move the migration paths to Steady State. Select the Summary tile and click on the
Close Event button.
4. Change the Decision column values to Move Back to Steady State. Click OK.
5. The Migrations dashboard now displays the paths are moved back to Steady State.
138
DobiMigrate Online Help 6.4
Click on the error icon (with the exclamation mark) to view the error details.
After you have fixed errors, you can reschedule the switchover and retry the items with errors.
1. Select the path you want to reschedule in the current event window and click the
Reschedule.. button at the bottom.
139
DobiMigrate Online Help 6.4
3. Select the items with errors you would like to retry and click Next.
4. Verify that all errors have been resolved after the rescheduled iteration has finished.
140
DobiMigrate Online Help 6.4
In order to complete the switchover, all errors should have been resolved or
accepted.
141
DobiMigrate Online Help 6.4
Select the migrations that you want to verify/repair and click on the Actions button under the
overview table. This opens a dialog from where you can select one of the following options:
• Recreate: to recreates this migration, restarting in first scan. You have the ability to
change the migration options before the migration is recreated.
• To steady state: to put the selected paths back to steady state. The selected paths will
appear in the Steady State tile and can be rescheduled from there for migration.
• Repair metadata: to forces a copy of metadata for content that is present on both
source and . New content or updates to the content will not be migrated, and content
only present on the target will not be deleted.
• Copy new content: to only copy new content from source to target. Updates to content
on the source will not be migrated, updates to content on the target will not be
overwritten and new content on the target will not be deleted.
• Verification: to verify the consistency between the source content and the target
content and report inconsistencies, if there are any.
Go to the Post Finish tile to view the results of the verification and repair actions, for example:
• If there are failures they will show up in the Finished and/or Post Finish tile and in the
Issues tile (summary).
• If no blocking errors are found, the error will be part of the Finished tile.
• To view the error details, click on the red icon with the exclamation mark
142
DobiMigrate Online Help 6.4
Rollback as implemented in the current version of DobiMigrate has the following limitations:
• Deletes are not propagated. To avoid modifying the source during a switchover, deletes
are not propagated. For example, if someone accidently deleted files on the target after
the switchover, the only remaining copy at the source should not be deleted.
• No automatic reversal of the SID mapping. If the original migration used SID mapping,
you need to create the opposite SID map manually and assign that to the rollback
migration.
• A migration must be defined to migrate in the opposite direction. If the original migration
was from FilerA to FilerB, a migration from FilerB to FilerA must exist.
• End users should not have access to the original source when you perform a rollback.
Action Recommendations
Initiate rollback Ensure original source is still read-only so that users no longer
have access to it. The target you are rolling back from, can still
be in read-write mode at this point until the rollback switchover.
143
DobiMigrate Online Help 6.4
Action Recommendations
Initiate rollback Ensure source and target are marked as read-only. This is the
switchover last incremental copy and users should not have access to
either side during rollback switchover.
The Rollback job wizard dialog is displayed, summarizing the rollback that will be
performed.
2. Click Next. The wizard checks the prerequisites and runs pre-checks to verify that
everything is in place to run the migration rollback.
144
DobiMigrate Online Help 6.4
Once these pre-checks are completed, the system displays a summary of the jobs and
shares that will be used for the rollback.
3. Click Next. The Rollback job options page displays the migration options.
By default, the Rollback migration wizard displays the same options as the original
migration wizard.
4. Click Next. The Precheck Results page is displayed. This is the final step where
DobiMigrate will run additional pre-checks based on the displayed paths and options.
145
DobiMigrate Online Help 6.4
5. Click Finish. The rollback migration will be performed. It will run as a normal migration
and the details will be displayed on the Iteration details screen.
6. On the Jobs Dashboard you can see there is now a reverse migration running.
7. Once the migration is in steady state, remove the migration path in DobiMigrate, as the
target is synced back to the source at this point. Click the Remove button.
146
DobiMigrate Online Help 6.4
8. The last step is to select the migration path that you rolled back from the Finished
migrations and click the Actions button.
9. Select To steady state and click on Next. This will start the migration path in steady
state, copying changes again from source to target, until a new switchover can be
planned.
147
DobiMigrate Online Help 6.4
Once the rollback migration is in "steady state", it must be switched over. For
more information, see Data Migration: Switchover and Dry Run.
11.1. About
Apart from browser-based access to the Datadobi software suite, the REST API (Application
Programming Interface) offers an additional way to automate the configuration and
management of data migrations.
An API is a set of interfaces that enable software programs to “talk” to each other. REST
(Representational State Transfer) is an architectural pattern that describes how distributed
systems can expose a consistent interface. The term "REST API" generally refers to an API
accessed using the HTTP protocol via predefined set of URLs and HTTP verbs.
148
DobiMigrate Online Help 6.4
It is currently not possible to use the REST API to start or manage a running
The REST API is exposed over the HTTP or HTTPS protocol. It uses the JSON format for the
request and response body of the API calls.
1. Before a user can use the API, an administrator has to generate an API token for that
user, and give the token to the user. As an administator, log in to the UI and go to
Configuration > Users. Select the User and click Edit.
3. The administrator copies the token from the popup window. The token is available for all
API calls and does not expire. The administrator distributes the API token to the user in
a secure way.
149
DobiMigrate Online Help 6.4
2. To use the REST API, the API token has to be included in the HTTP Authorization
header. The text below shows examples using the curl command, but any other REST
API client can also be used e.g. https://github.com/socialwifi/jsonapi-requests/ . In the
examples it is assumed that the user has set an environment variable API_TOKEN. The
first API call will retrieve the list of servers known in the DobiMigrate system, two
servers are found.
150
DobiMigrate Online Help 6.4
"subServers" : {
"links" : {
"self" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9/relationships/subServers",
"related" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9/subServers"
}
}
}
} ],
"links" : {
"self" : "http://10.10.97.5/api/servers/"
}
}
• Use API to list the migrations and inspect the migration details
1. After adding one or more file servers in the in the DobiMigrate UI, query the list of
servers using the API. The example output of that full list is shown above. Two servers
were returned named Source Server and Target Server. The id of each server is
present in the response. The example below shows how to find the details of the server
having ID 2797bbbd-fd0d-4d35-99ad-052e3a660ed9. The response shows that
there are five sub-servers named System, and VirtualServer001 through
VirtualServer004. The IDs of those sub-servers are needed to determine the source
and target for the migrations to be created.
151
DobiMigrate Online Help 6.4
http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9?include=subServers
{
"data" : {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9",
"type" : "servers",
"links" : {
"self" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9"
},
"attributes" : {
"name" : "Source Server"
},
"relationships" : {
"subServers" : {
"data" : [ {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:System:I",
"type" : "subservers"
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer002:I",
"type" : "subservers"
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer003:I",
"type" : "subservers"
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer004:I",
"type" : "subservers"
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer001:I",
"type" : "subservers"
} ],
"links" : {
"self" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9/relationships/subServers",
"related" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9/subServers"
}
}
}
},
"included" : [ {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:System:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:System:I"
},
"attributes" : {
"name" : "System"
},
152
DobiMigrate Online Help 6.4
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:System:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:System:I/server"
}
}
}
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer001:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:VirtualServer001:I"
},
"attributes" : {
"name" : "VirtualServer001"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer001:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer001:I/server"
}
}
}
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer002:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:VirtualServer002:I"
},
"attributes" : {
"name" : "VirtualServer002"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer002:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer002:I/server"
}
}
}
153
DobiMigrate Online Help 6.4
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer003:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:VirtualServer003:I"
},
"attributes" : {
"name" : "VirtualServer003"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer003:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer003:I/server"
}
}
}
}, {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer004:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:VirtualServer004:I"
},
"attributes" : {
"name" : "VirtualServer004"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer004:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer004:I/server"
}
}
}
} ],
"links" : {
"self" : "http://10.10.97.5/api/servers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9?include=subServers"
}
}
2. Create a migration in the DobiMigrate UI. Put the schedule on Manual if you do not
want to automatically start the migration.
154
DobiMigrate Online Help 6.4
155
DobiMigrate Online Help 6.4
"copyPipeFilesMode" : "DONT_COPY",
"copyCharacterDeviceFilesMode" : "DONT_COPY",
"copyBlockDeviceFileMode" : "DONT_COPY",
"copyPermissionModifier" : "sssssssss"
},
"smbOptions" : {
"desiredOwner" : "",
"desiredGroup" : "",
"copySacl" : false,
"followJunctions" : false,
"cleanInvalidSecurityDescriptors" : false,
"copyUserDefinedAttributesMode" : "COPY",
"replaceCreatorOwnerGroup" : false,
"copyOwner" : false,
"copyGroup" : false,
"copyDacl" : false,
"copyHiddenAttribute" : false,
"copyNotContentIndexedAttribute" : false,
"copyReadOnlyAttribute" : false,
"copySystemAttribute" : false,
"copyTemporaryAttribute" : false
}
},
"specialUseCase" : "NONE"
},
"relationships" : {
"uidMapping" : {
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/uidMapping",
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/uidMapping"
}
},
"gidMapping" : {
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/gidMapping",
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/gidMapping"
}
},
"destinationSubServer" : {
"data" : {
"id" : "2c5ad57a-e29b-4aa5-9497-102585e567c1:System:I",
"type" : "subservers"
},
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/destinationSubServer",
156
DobiMigrate Online Help 6.4
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/destinationSubServer"
}
},
"switchoverGroup" : {
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/switchoverGroup",
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/switchoverGroup"
}
},
"sidMapping" : {
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/sidMapping",
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/sidMapping"
}
},
"sourceSubServer" : {
"data" : {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer001:I",
"type" : "subservers"
},
"links" : {
"self" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/relationships/sourceSubServer",
"related" : "http://10.10.97.5/api/jobs/23179d06-646f-4af3-bb8c-
fe56d9bf32d4/sourceSubServer"
}
}
}
} ],
"included" : [ {
"id" : "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer001:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-99ad-
052e3a660ed9:VirtualServer001:I"
},
"attributes" : {
"name" : "VirtualServer001"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
99ad-052e3a660ed9:VirtualServer001:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2797bbbd-fd0d-4d35-
157
DobiMigrate Online Help 6.4
99ad-052e3a660ed9:VirtualServer001:I/server"
}
}
}
}, {
"id" : "2c5ad57a-e29b-4aa5-9497-102585e567c1:System:I",
"type" : "subservers",
"links" : {
"self" : "http://10.10.97.5/api/subservers/2c5ad57a-e29b-4aa5-9497-
102585e567c1:System:I"
},
"attributes" : {
"name" : "System"
},
"relationships" : {
"server" : {
"links" : {
"self" : "http://10.10.97.5/api/subservers/2c5ad57a-e29b-4aa5-
9497-102585e567c1:System:I/relationships/server",
"related" : "http://10.10.97.5/api/subservers/2c5ad57a-e29b-4aa5-
9497-102585e567c1:System:I/server"
}
}
}
} ],
"links" : {
"self" :
"http://10.10.97.5/api/jobs/?include=sourceSubServer%2CdestinationSubServer
"
}
}
4. To query one specific migration (instead of all migrations), use the migration job id to
get details for that one migration job.
...
158
DobiMigrate Online Help 6.4
5. To create a new migration, it is a common REST practice to use the HTTP POST verb. In
order to do this, create a file migration_info.txt which contains the migration
options that is copied from the previous output. In this simple example, a new migration
is created which migrates the Projects2 folder using the same source and target
servers as the migration created in the UI.
{
"data": {
"relationships": {
"sourceSubServer": {
"data": {
"id": "2797bbbd-fd0d-4d35-99ad-052e3a660ed9:VirtualServer001:I",
"type": "subservers"
}
},
"destinationSubServer": {
"data": {
"id": "2c5ad57a-e29b-4aa5-9497-102585e567c1:System:I",
"type": "subservers"
}
}
},
"attributes": {
"options": {
"configuration": {
"protocol": "NFS"
},
"iterationScheduling": {
"strategy": "MANUAL",
"cronSchedule": null
},
"startFirstIterationNow": false
},
"jobType": "NAS_MIGRATION",
"destinationPath": "/ifs/output/Projects2",
"sourcePath": "/ifs/Windows/Projects2"
},
"type": "jobs"
}
}
159
DobiMigrate Online Help 6.4
{
"data" : {
"id" : "73e1450e-6952-49af-9d66-d735a9bb5cd9",
"type" : "createjobstatuses",
"links" : {
"self" : "http://10.10.97.5/api/createjobstatuses/73e1450e-6952-49af-
9d66-d735a9bb5cd9"
},
"attributes" : {
"status" : "QUEUED",
"errors" : { }
}
},
"links" : {
"self" : "http://10.10.97.5/api/jobs"
}
}
7. Query the “createjobstatuses” resource to view the actual status of the queued
creation. Use the id that was returned when the job was created. The job has
successfully been created, and the migration job URL is returned in the Location
response header. This new migration is now displayed in the UI.
160
DobiMigrate Online Help 6.4
{
"errors" : [ {
"status" : "303",
"title" : "SEE_OTHER",
"detail" : "Job created"
} ]
}
8. Verify that the migration was created in the UI. The migration can also be inspected
using the API. There are two ways to do this.
# Query the migration directly, based on the migration job URL that was
returned earlier in the Location response header
curl -H "Authorization: Bearer $API_TOKEN"
http://10.10.97.5/api/jobs/b28f85e5-f9cf-44de-bfe0-1d238a8b1a2a
9. In case the migration creation fails, the “createjobstatuses” resource can be queried to
find the failure reason. For example, a migration already exists on that path.
161
DobiMigrate Online Help 6.4
{
"data" : {
"id" : "e6b7bb65-df13-4de2-b171-a71e0ab9555a",
"type" : "createjobstatuses",
"links" : {
"self" : "http://10.10.97.5/api/createjobstatuses/e6b7bb65-df13-4de2-
b171-a71e0ab9555a"
},
"attributes" : {
"status" : "FAILED",
"errors" : {
"VALIDATING_SOURCE" : [ "there are existing migrations with the
given source path /ifs/Windows/Projects2" ]
}
}
},
"links" : {
"self" : "http://10.10.97.5/api/createjobstatuses/e6b7bb65-df13-4de2-
b171-a71e0ab9555a"
}
}
10. Another potential issue is that the migration directory does not exist on the source or
target.
{
"data" : {
"id" : "2a33513c-1430-456a-975e-cd071bfc5aab",
"type" : "createjobstatuses",
"links" : {
"self" : "http://10.10.97.5/api/createjobstatuses/2a33513c-1430-456a-
975e-cd071bfc5aab"
},
"attributes" : {
"status" : "FAILED",
"errors" : {
"VALIDATING_SOURCE" : [ "Could not determine if source path
existed. Reason: NfsV3Exception.2: //ProjectsX: 'Lookup' failed with status
NFS3ERR_NOENT[2 - No such file or directory]" ]
}
}
},
"links" : {
"self" : "http://10.10.97.5/api/createjobstatuses/2a33513c-1430-456a-
975e-cd071bfc5aab"
}
}
162
DobiMigrate Online Help 6.4
multiple migrations, a script has to be written to issue multiple such API calls
and to follow up on their completion.
For security, it is recommended to only allow HTTPS access to the DobiMigrate UI and API.
The host firewall can be used to close the HTTP port. Additionally, a proper SSL certificate
should be deployed (preferably not self-signed) so that the communication of UI and API users
is protected.
Role Resource
11.8.1. Resources
[graph]
Resource Description
163
DobiMigrate Online Help 6.4
Resource Description
Createjobstatuses
Attributes
Jobs
164
DobiMigrate Online Help 6.4
Attributes
id Job id String ✓ ✓ ✓
Relationships
165
DobiMigrate Online Help 6.4
Principalmaps
Attributes
Servers
Server definition
Attributes
id Server id String ✓ ✓
166
DobiMigrate Online Help 6.4
Relationships
Subservers
Attributes
Relationships
Switchovergroups
167
DobiMigrate Online Help 6.4
Attributes
Relationships
11.8.2. Types
Type Description
168
DobiMigrate Online Help 6.4
Type Description
169
DobiMigrate Online Help 6.4
Type Description
ChainOfCustodyMode
Literals
Name Description
170
DobiMigrate Online Help 6.4
Name Description
Configuration
Use this to override the configuration for paths in the migration source.
Attributes
CopyNamedAttributesMode
Literals
Name Description
171
DobiMigrate Online Help 6.4
Name Description
CopyRootDirectoryMode
Literals
Name Description
172
DobiMigrate Online Help 6.4
Name Description
CopySpecialFilesMode
Literals
Name Description
CopyUserDefinedAttributesMode
173
DobiMigrate Online Help 6.4
Literals
Name Description
CronScheduleDto
Attributes
https://docs.oracle.com/en/
java/javase/11/docs/api/
java.base/java/time/
ZoneId.html#
of(java.lang.String)
174
DobiMigrate Online Help 6.4
DigestAlgorithm
Literals
Name Description
MD5 MD5
SHA1 SHA1
SHA256 SHA256
SHA512 SHA512
XXH128 XXH128
IterationScheduling
Attributes
175
DobiMigrate Online Help 6.4
JobControl
Attributes
JobOptions
Attributes
176
DobiMigrate Online Help 6.4
177
DobiMigrate Online Help 6.4
• Immediately
• According to the
iteration schedule
• when the
iterationScheduling is
set to MANUAL, the
default for
startFirstIterationNow
is FALSE
• when the
iterationScheduling is
set to DEFAULT, and
that default iteration
scheduling is set to
MANUAL, the default
for
startFirstIterationNow
is FALSE
178
DobiMigrate Online Help 6.4
179
DobiMigrate Online Help 6.4
JobType
Type of job
Literals
180
DobiMigrate Online Help 6.4
Name Description
MinimumAge
Minimum age
Literals
Name Description
ZERO_SECONDS
FIFTEEN_SECONDS
THIRTY_SECONDS
ONE_MINUTE
FIVE_MINUTES
TEN_MINUTES
THIRTY_MINUTES
ONE_HOUR
TWO_HOURS
THREE_HOURS
SIX_HOURS
NINE_HOURS
181
DobiMigrate Online Help 6.4
Name Description
TWELVE_HOURS
ONE_DAY
TWO_DAYS
THREE_DAYS
ONE_WEEK
MultiProtocolScanErrorMode
Literals
Name Description
182
DobiMigrate Online Help 6.4
NfsConstraints
Literals
Name Description
V3 NFSv3 migration
V4 NFSv4 migration
NfsOptions
Attributes
183
DobiMigrate Online Help 6.4
184
DobiMigrate Online Help 6.4
PrecheckWarnings
Literals
Name Description
186
DobiMigrate Online Help 6.4
Name Description
PrincipalMapType
Literals
187
DobiMigrate Online Help 6.4
Name Description
Protocol
Literals
Name Description
MULTIPROTOCOL Multiprotocol
188
DobiMigrate Online Help 6.4
SmbOptions
Attributes
189
DobiMigrate Online Help 6.4
190
DobiMigrate Online Help 6.4
SmbSymlinkTargetMode
Literals
Name Description
191
DobiMigrate Online Help 6.4
Name Description
SpecialUseCase
Literals
Name Description
NONE Normal
Status
Literals
192
DobiMigrate Online Help 6.4
Name Description
UNKNOWN
Strategy
Literals
193
DobiMigrate Online Help 6.4
Name Description
12. Tools
The following tabs are available:
• Performance
• Activity
• Logging
• Console
• File Browser
• Sessions
12.1. Performance
This tab provides performance information for a chosen source - target file server pair. The
slider at the bottom allows you to define the time period for which you want to see the data.
• Bandwidth
• Throughput
194
DobiMigrate Online Help 6.4
12.2. Activity
The Activity view shows the ongoing migration activity (if any) for all of the entered migrations.
In the screenshot below you can see an example of different migrations that are in the copy
phase:
The Blocked state for the Commit Directory is not an issue. It simply means
that these directories are waiting until all files are copied, after which the
directory commit will be done.
12.3. Logging
This tab shows the logging for the different components in DobiMigrate. This can be a very
handy tool to debug issues.
The left-hand side of the window shows the items that can be selected for logging with their
logging level. You can search for specific items or reload the last-used logging configuration
195
DobiMigrate Online Help 6.4
change the logging levels or other settings as this may seriously impact
performance.
Logging details are cumulative, with each subsequent level building on the previous one.
Logging settings are hierarchical. When a particular logger is ‘not set’ the level of detail setting
of its parent will apply.
• Debug: for this item only information that might be relevant for debugging is logged.
At the right side of the window, a snapshot of the current logging information is shown (last
1000 lines). This includes:
• Level: the logging level indicated by an icon with the following color coding:
◦ red = error/fatals
◦ yellow = warning
◦ green = info
◦ no color = debug/trace
196
DobiMigrate Online Help 6.4
• Class: the internal DobiMigrate class that generated this log message.
To pause and resume the current logging information, click on the Pause button above the
logging overview table.
To download the log files, click on the download button above the logging overview table. A
dialog appears allowing you to select the logging information to include in the log files:
• Source: from where should the logging details be retrieved, options are:
◦ DobiMigrate: download the log files from DobiMigrate itself. If there are proxies
configured, there will be additional options.
◦ All proxies: to download the log files for all configured proxies.
• Destination: the location to save the log files to. The log files can be downloaded
directly within the web browser or they can be downloaded to an existing dropzone. The
download will be a ZIP file containing one or more TXT files with all logging details.
12.4. Console
This allows Datadobi support to execute commands in order to troubleshoot reported issues.
It is also possible Datadobi will ask you to execute certain commands and send back the
output, as part of an ongoing issue investigation.
197
DobiMigrate Online Help 6.4
S3. You can choose the file server, and then drill down to the file, directory, or object in
question.
This tool is useful to, e.g. root cause permission differences on source and target during a
migration, as you can check how DobiMigrate sees the permissions via SMB or NFS on the
source and on the Target file server. It also provides visibility to metadata attributes of objects.
Alerts are communicated through a red indication marker in the GUI, or by email (SMTP), and
these emails can be configured to send out alerts of varying levels of severity or detail.
You can configure alerts through both the Configuration and Tools modules, where the user
can chose from an extensive list of possible event-types and decide whether or not to be
notified through an alert.
Configuration module
You should first configure the SMTP host in the SMTP Configuration
panel at the top of the page.
198
DobiMigrate Online Help 6.4
There isn’t one email per event - each email includes all events that may have occurred during
the set time frame. So for example if the frequency is set to 'Every 15 minutes', DobiMigrate
will check every 15 minutes if any events have occurred, and then bundle all events into a
single email alert if needed.
Tools module
The Events & Alerts tab in the Tools module allows you to view the events and alerts of a
given DobiMigrate instance.
You can select either All events & alerts or Unresolved alerts. You can also filter by type,
source, or details:
You can configure the Events & Alerts by clicking on the cog wheel symbol on the right. From
here you can select the event or alert-types you would like to receive:
199
DobiMigrate Online Help 6.4
You can then view the details of an event or alert by clicking on its corresponding icon in the
column on the left.
12.7. Sessions
The Sessions tab displays the open SMB connections to a certain file server. In the example
below, 2 active connections are present, both of them originating from DobiMigrate:
During a switchover, this tab is useful to check if there are any open SMB connections to the
file servers that can impact the switchover. If so, it is possible to end these open Sessions by
clicking on the Disconnect button.
200