Scribd
Upload
25 views2 pages

Validations

The document outlines various SQL queries and checks for data validation, including schema validation, data type checks, total count checks, and data accuracy. It emphasizes the importance of ensuring data consistency, completeness, and uniqueness, while also detailing methods for identifying and handling null values and duplicates. Additionally, it describes the use of MINUS or EXCEPT queries for source and target validation, as well as the use of Common Table Expressions (CTE) for managing duplicate records.

Uploaded by

mmyybabybaby
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
25 views2 pages

Validations

The document outlines various SQL queries and checks for data validation, including schema validation, data type checks, total count checks, and data accuracy. It emphasizes the importance of ensuring data consistency, completeness, and uniqueness, while also detailing methods for identifying and handling null values and duplicates. Additionally, it describes the use of MINUS or EXCEPT queries for source and target validation, as well as the use of Common Table Expressions (CTE) for managing duplicate records.

Uploaded by

mmyybabybaby
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Please help me to understand the below with SQL query

 Schema Validation:

 Mapping Logic check for each field As per ETL mapping Document. It helps us to
check the column name and number we can do it in informatica. It is kind of white box
checking.

 Data-type/Data Size checks in database - Verify the table structure of data type, data
length, keys, constrains, views, stored procedure. Data type check confirms that the data
entered has the correct data type.

In Second column we find city_name column varchar (100) But in BRD document it Is like
varchar (28) but developer given 100 so we can make it as bug and report to dev team.
For example, a field might only accept numeric data. If this is the case, then any data
containing other characters such as letters or special symbols should be rejected by the
system.

 Total Count check/Data completeness query: This query checks whether all the
expected data has been loaded into the data warehouse or not. Verify that counts should
match in source and target data file. To count the number of records in a data base, we
can use the COUNT function in SQL query.

 Data accuracy/ Data Validation: ata validation is the method for checking the accuracy
and quality of data. It is often performed prior to adding, updating, or processing data.
Similarly, when we want to merge data from disparate sources we often talk of
‘cleansing’ the data – in other words validating it. When validating data, we can check if
the data is:
a. complete (ie no blank or null values)
b. unique (ie no duplicate values)
c. consistent with what we expect (eg a decimal between a certain range)

 Data consistency: to checks that the data in the data warehouse is consistent or not.
The query will give us list of columns that are not consistent across different source
systems. We can then investigate these columns to identify the cause of the
inconsistency.

 Data profiling: This query helps in identifying data patterns and relationships in the
source data.

 Data validation Check : will check whether the data in the data warehouse is valid or
not.

 Source and Target validation: MINUS query or EXCEPT query is used to subtract out
the result of second query from the result of the first query. It takes the distinct
 result set of the first query, then filter out the records which appear in the result set of the
second query. If the second query includes the records which are not in the first query,
they will be ignored.
Data Count

MINUS or EXCEPT

 Null Values Check- A field with a NULL value is a field with no value. A NULL value is
different from a zero value or a field that contains spaces. A field with a NULL value is
one that has been left blank during record creation.
Checking whether null values populated in target table if data is replace with values as
per business logic. We can use the IS NULL or IS NOT NULL operator in WHERE
clause.

 Duplicate Check Testing-


Query to get the number of duplicate rows in the target table. We can then investigate
these rows to identify the cause of the duplication. Some data like IDs or e-mail
addresses are unique by nature. A database should likely have unique entries on these
fields. A uniqueness check ensures that an item is not entered multiple times into a
database.

You can get duplicate recodes:

To find only duplicates duplicats rows:

Display all rows


Display columns only duplicates.

To find without duplicates;

Removing duplicate records:

Method 1: Using Distinct by creating a temporary table.

Method 1: create and drop another table

Method 2: By using Auto_id column

Method 3: Using CTE –


CTE allows you to define a temporary named result set that is available temporarily in the
execution scope of a statement such as SELECT, INSERT, UPDATE, DELETE, or MERGE.

You might also like