Data Migration
Data Migration
Data migration is the process of selecting, preparing, and moving existing data from one
computing environment to another.
1. Profiling:
Data profiling involves analysing the source data to understand its structure, quality,
and content. This helps identify any issues or anomalies in the data that need to be
addressed before migration.
2. Development:
In this phase, you develop the tools, scripts, or programs needed to perform the data
migration. This often involves writing code or configuring software to extract,
transform, and load (ETL) data.
3. Testing:
Testing ensures that the data migration processes work correctly and that the data is
accurately transferred. This phase involves validating that the data has been moved
as expected without errors or losses.
UAT is conducted by end-users to ensure that the migrated data meets their needs
and expectations. It confirms that the migration process delivers the desired
outcomes from the user perspective.
6. Volume Testing:
This testing involves validating that the migration process can handle the volume of
data being transferred without performance degradation or failures.
7. Dry Run:
A dry run is a practice run of the data migration process to identify and resolve any
issues before the actual migration takes place.
8. Production:
This is the final stage where the actual data migration occurs in the live environment.
Executing the migration plan in the production environment, monitoring the process
closely, and addressing any issues that arise to ensure a smooth transition.
Data sources for migration : File systems , Data warehouses , Legacy systems , API
BSCS (Billing and Customer Care Systems) : Capture the transaction level data
LDB: LDB refers to a logical model of data that abstracts the physical data storage details. It
represents the organization and structure of data as perceived by users and applications,
independent of the physical storage or database implementation. During migration, the
logical data model helps map data from the source system's logical structure to the target
system’s logical structure.
SDB : System Database is a type of database that stores metadata, configuration settings,
and system-level information for a database management system (DBMS). It typically
contains details about database schemas, user permissions, and system settings.
EDB (Execution Database) : EDB is a replica of SDB. Any interface can access EDB.
DQ Rule :Data Quality (DQ) rules in data migration are guidelines or standards used to
ensure that data remains accurate, consistent, complete, and reliable throughout the
migration process.
Downstream Systems :
After the migration, the data is available for downstream systems like Finance, IVR, DWH
(Data Warehouse), and others.