Scribd
Upload
9 views2 pages

Validationns Mapping

The document outlines various validation processes essential for ensuring data integrity and accuracy during data integration and processing. It includes mapping document validation, job validations, configuration validations, schema validation, and checks for constraints, record counts, duplicates, data quality, date and number formats, special characters, and both full and incremental load validations. Additionally, it addresses the verification of Slowly Changing Dimensions (SCD) to maintain historical data tracking.

Uploaded by

mmyybabybaby
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
9 views2 pages

Validationns Mapping

The document outlines various validation processes essential for ensuring data integrity and accuracy during data integration and processing. It includes mapping document validation, job validations, configuration validations, schema validation, and checks for constraints, record counts, duplicates, data quality, date and number formats, special characters, and both full and incremental load validations. Additionally, it addresses the verification of Slowly Changing Dimensions (SCD) to maintain historical data tracking.

Uploaded by

mmyybabybaby
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

● Mapping document validation: Ensuring that the document

describing how data fields are transferred and transformed between


systems is accurate, complete, and consistent with the actual
implementation.
● Job Validations: Verifying that data integration or processing jobs (e.g.,
ETL jobs) run successfully, complete within expected timeframes, and
produce the intended outputs without errors.
● Configuration validations: Checking that the settings and parameters
of systems, applications, or data pipelines are correctly configured
according to requirements and standards.
● Schema Validation: Verifying that the structure and organization of
data (e.g., tables, files, data models) conform to the defined schema,
including data types, lengths, and presence of fields.
● Constraints Validation: Ensuring that data adheres to the rules and
limitations defined on data fields (e.g., primary keys, foreign keys,
unique constraints, not null constraints).
● Record Count Validation: Comparing the number of records in source
and target systems or between processing steps to ensure no data loss
or unintended duplication has occurred.
● Duplicate Validations: Identifying and flagging or removing identical or
highly similar records within a dataset to ensure data uniqueness and
accuracy.
● Data validations (including transformations): Checking the quality
and correctness of the actual data values, often after they have
undergone transformations (e.g., cleaning, formatting, calculations).
● Date format check: Verifying that date values conform to a predefined
format (e.g., YYYY-MM-DD, MM/DD/YYYY) to ensure consistency and
proper interpretation.
● Number format check: Ensuring that numerical values adhere to a
specified format (e.g., decimal places, thousands separators) and are
valid numerical entries.
● Special Characters check: Identifying and handling the presence of
unwanted or invalid special characters in data fields based on defined
rules.
● Full load validation: Performing comprehensive validation on all data
when a system or process is initialized or completely refreshed.
● Incremental load validations: Validating only the new or updated data
during periodic updates to a system, ensuring the integrity of the
changes.
● SCD's Validations (Slowly Changing Dimensions Validations):
Verifying that the historical tracking of changes to dimension data is
correctly implemented and maintained according to the defined SCD
type (e.g., Type 1, Type 2).

You might also like