A Final Year Internship Report

On
Data Intern
At
eXtenso Data

Submitted To

Department of Computer Science and Information Technology

Swastik College
Chardobato, Bhaktapur

Under the Supervision of

Sristi Khatiwada
Coordinator, Swastik College

In partial fulfillment of the requirements for the Bachelor’s in Computer

Science and Information Technology

Submitted By
Kebal khadka

TUReg. No. 5-2-8-1131-20-2020

May,2025
 Supervisor’s Recommendation
I hereby recommend that this report, prepared under my supervision by Kebal khadka(TU
Roll No. 26861/077), be accepted as fulfilling in partial requirement for the degree of
Bachelor of Science in Computer Science and Information Technology. In my best
knowledge, this is an original work in Computer Science and Information Technology.

........................................

Ms. Sristi Khatiwada

Swastik College
 Letter of Approval

This is to certify that this report, prepared by Kebal khadka (26861/77) in partial
fulfillment of the requirement for the degree of Bachelor of Science in Computer Science
and Information Technology, has been well studied. In our opinion, it is satisfactory in the
scope and quality of the project for the required degree.

………………... ………………… …………………….

Ms. Sristi Khatiwada External Examiner Ms. Sristi Khatiwada

Supervisor CDSCSIT Head of Department
Swastik College Tribhuvan University Swastik College
 Acknowledgement

I want to sincerely thank eXtenso Data for giving me the chance to work as a data intern.
I am incredibly appreciative of the priceless influence this experience has had on my
development on both a personal and professional level. It has been a pillar of my career
path.
I am incredibly grateful to Mr. Suresh Gautam, CEO of eXtenso Data, for giving me this
internship opportunity and allowing me to learn a great deal about a variety of industries.
His mentorship and continuous support during my internship have been instrumental in
shaping my professional development, and I deeply value his guidance and
encouragement.
I am sincerely thankful to my supervisor, Ms. Sristi Khatiwada, for her exceptional
guidance, unwavering support, and inspiring encouragement throughout my internship.
Report generation is one of the areas where her wise criticism and guidance have greatly
improved my abilities.
Lastly, I would like to extend my sincerest regards and heartfelt gratitude to all of my
esteemed colleagues, fellow workers, and any other individuals who have provided me
with unwavering support throughout the entirety of this period.
 Abstract
Data engineering involves designing and building robust systems that facilitate the
collection, transformation, and management of large-scale data to support strategic
decision-making. This report summarizes my internship as a Data Engineering Intern at
eXtenso Data, a Big Data Analytics company dedicated to enhancing operational
efficiency, optimizing costs, and uncovering new business opportunities through data-
driven insights. During my time at eXtenso Data, I was actively involved in developing
and maintaining data pipelines using Python and SQL, and I worked extensively with
MySQL for data storage and querying. Additionally, I gained hands-on experience with
big data tools such as Apache Spark for large-scale data processing and Apache Airflow
for orchestrating complex data workflows.
This internship deepened my understanding of the complete data engineering lifecycle—
from data ingestion and transformation to scheduling and automation—and provided me
with valuable experience in building scalable data solutions in a real-world business
setting.

Keywords: Data Engineering, Big Data, Python, SQL, MySQL, Apache Spark, Apache
Airflow, Data Pipelines, Data Ingestion, Data Transformation.

Table of Contents
Supervisor’s Recommendation..........................................................................................i

Letter of Approval..............................................................................................................ii

Acknowledgement.............................................................................................................iii

Abstract..............................................................................................................................iv

List of Tables.....................................................................................................................vii

Chapter 1: Introduction.....................................................................................................1

1.1 Introduction..........................................................................................................1

1.2 Problem Statement...............................................................................................1

1.3 Objectives.............................................................................................................2

1.4 Scopes..................................................................................................................2

1.5 Limitations...........................................................................................................2

1.6 Report Organization.............................................................................................2

Chapter 2: Organization Details and Literature Review...............................................4

2.1 Background Study......................................................................................................4

2.1.1 Introduction to Organization................................................................................4

2.1.2 Organizational Hierarchy.....................................................................................4

2.1.3 Working Domains of Organization......................................................................5

2.1.4 Description of Intern Department/Unit................................................................6

2.2 Literature Review.......................................................................................................7

Chapter 3: Internship Activities........................................................................................8

3.1 Roles and Responsibilities...................................................................................8

3.2 Weekly log...........................................................................................................9

3.3 Description of the Project(s) Involved During Internship..................................10

3.4 Tasks / Activities Performed..............................................................................11

Chapter 4: Conclusion and Learning Outcomes...........................................................13

4.1 Conclusion................................................................................................................13

4.2 Learning Outcome....................................................................................................13

References.........................................................................................................................15

Annex.................................................................................................................................16

List of Tables
Table 2.1 Organization Details.........................................................................................12
Table 2.2 Internship Period Details.................................................................................14
Table 3.1 weekly Log.........................................................................................................17
 Chapter 1: Introduction
1.1 Introduction
Data engineering is a crucial field that focuses on designing, building, and managing the
infrastructure and tools needed to collect, store, process, and analyze large volumes of
data. It plays a vital role in enabling organizations to make data-driven decisions and gain
valuable insights from their data. During my ongoing internship, I am building a strong
foundation in data engineering by working on data collection, transformation, and
pipeline development. I am actively involved in creating scalable data workflows,
managing databases, and ensuring data quality across various stages of the pipeline.
During this internship, I focused on building end-to-end data pipelines to support reliable
and scalable data workflows. I started by developing ETL scripts in Python to collect and
transform data from various sources using tools like Selenium Base for web automation
and Pandas for data cleaning and transformation.
I gained hands-on experience with SQL, which I used extensively for querying and
transforming data from structured databases. This laid a strong foundation in data
wrangling, joins, aggregations, and subqueries—essential operations in any data
engineering role.
As the internship progressed, I was introduced to modern big data tools such as Apache
Airflow for scheduling and orchestrating complex data workflows, and Apache Spark
for distributed processing of large datasets. These technologies allowed me to scale data
processing tasks beyond traditional scripting and move toward production-ready
pipelines.

1.2 Problem Statement

In today's data-driven world, organizations increasingly rely on large volumes of data
from diverse sources to make informed decisions. However, collecting, processing, and
managing this data efficiently poses significant challenges due to issues such as
inconsistent data formats, missing or noisy data, lack of automation, and the growing
scale of data. Manual data handling processes are time-consuming, error-prone, and not
scalable.
To address this, there is a critical need for automated, reliable, and scalable data
engineering solutions that can:
  Extract data from multiple sources,
 Clean and transform it into a structured format,
 Load it into storage systems or databases for further analysis.

1.3 Objectives
 To develop and implement automated ETL (Extract, Transform, Load) pipelines
using Python and SQL to efficiently ingest and process structured and
unstructured data from multiple sources.
 To gain practical experience with modern data engineering tools and frameworks,
including Apache Airflow for workflow orchestration and Apache Spark for
distributed big data processing.
 To ensure data quality and integrity through effective data cleaning,
transformation, and validation processes, enabling reliable storage and
downstream use by analytics or reporting systems.

1.4 Scopes

 To build and manage ETL pipelines using Python and SQL for transforming raw
sanction list data into structured, analyzable formats.
 To work with tools like Apache Airflow and Spark for understanding scalable data
processing and workflow automation in a big data environment.
 To ensure data consistency and integrity by applying data cleaning techniques,
handling missing values, and standardizing formats using Pandas

1.5 Limitations

 Data collection is restricted by the structure/availability of the website. Data not

collected data gathered only if it is available on the target websites.
 Data cleaning can only do so much- some errors or biases in the raw data will
remain uncorrected.
 Matplotlib visualizations are approximately correct, not accurate

1.6 Report Organization

The contents of this report are organized into the following sections.
Chapter1 : Introduction
This chapter gives a summary of the project’s objectives, limitations, and scope.
Chapter 2: Organizational Details and Literature Review
An overview of the organization is provided in this chapter, which also includes
descriptions of the intern department/unit and its functional areas and organizational
structure. This chapter covers the key theories, concepts, and terminology related to the
internship project in order to provide the context for the background study. A literature
study, an evaluation of projects that are comparable to the internship, and theories and
outcomes that are comparable to the projects during the internship are also included.
Chapter 3: Internship Activities
This chapter covers the complete internship program. It includes details about the project
completed during the internship, the decisions made, the roles and duties assumed, and
the weekly logs kept.
Chapter 4: Conclusion and Learning Outcomes
This report’s conclusion and discussion of the internship’s learning objectives are covered
in this chapter.
Chapter 2: Organization Details and Literature Review

2.1 Background Study

2.1.1 Introduction to Organization

eXtensoData, a prominent business vertical of F1Soft Group, was founded in 2018 and is
led by CEO Suresh Gautam. It is a Big Data Analytics company focused on helping
businesses harness the power of their data to improve operational efficiency, optimize
costs, and uncover new opportunities. With a mission to turn raw data into actionable
intelligence, eXtensoData provides a broad suite of advanced data services tailored to
modern business needs
The company’s key areas of expertise include Data Engineering, Process Automation,
Business Analysis, Forecasting, Process Optimization, and Big Data Consulting. Its data
engineering services are designed to transform complex organizational data into
intelligent, timely insights, enabling data-driven decisions. Through process automation,
eXtensoData streamlines repetitive business tasks and eliminates inefficiencies by
leveraging enterprise data and building robust automation platforms.
In addition, the company offers business analysis support at both operational and strategic
levels, enhancing daily performance and delivering insights aligned with emerging
business trends. Its forecasting solutions empower clients with technology-driven
financial foresight, seamlessly integrating predictive models with operational strategies.

Table 2.1 Organization Details

Organization Name: eXtenso Data

Address: Pulchowk, Lalitpur, Nepal

Contact Number: +01-5439131

Email: [email protected]

2.1.2 Organizational Hierarchy

A company’s organizational structure changes throughout time in order to provide its

many functions and services in an efficient and effective manner. Organizations are
usually organized according to their functions to guarantee an efficient operation. eXtenso
Data employs a functional organizational structure that supports the diverse functions
within the company. This organizational structure consists of multiple divisions that work
closely to support business operations. The organization at eXtenso data is structured with
a top-down hierarchical structure of authority. This hierarchy ensures clear lines of
command and responsibility, allowing for effective management and oversight.

2.1.3 Working Domains of Organization

Core Services at eXtenso Data (eXtenso Data, n.d.)

i. Data Engineering:
We offer data engineering services that transform organizational data into meaningful,
intelligent insights. Our comprehensive data solutions are designed to address diverse
business challenges, enabling our clients to make timely and informed decisions.

ii. Process Automation:

We specialize in extracting enterprise data and insights from existing systems and
processes. By automating repetitive tasks, we build robust automation platforms that
ensure organizations are not constrained by outdated or inefficient workflows.

iii. Business Analysis:

Our business analysis services operate at both operational and strategic levels. At the
operational level, we help clients achieve excellence in daily functions. Strategically, we
analyze current market trends to deliver future-ready solutions tailored to each
organization’s needs.

iv. Forecasting:
Forecasting is a key component of effective business planning. Our technologies
automate the forecasting process, making it easier for organizations to align financial
projections with operational strategies for sustained success.

v. Big Data Consulting:

Our Big Data consulting services are focused on processing and analyzing large volumes
of data. The insights gained enable our clients to make data-driven decisions and strategic
plans based on accurate and timely information.
2.1.4 Description of Intern Department/Unit

An internship is a type of practical learning that combines classroom knowledge and

theory with real-world application in a professional or community setting. The resources,
tools, and facilities provided during the internship enable students to gain valuable
experience in a work environment, equipping them with transferable skills and practical
insights.
At eXtenso Data, a total of two intern students from Swastik College participated in the
internship program, which was structured into two distinct phases: the learning phase and
the project phase. During the learning phase, interns were provided with study materials
and technical resources by the mentor to build foundational knowledge. In the project
phase, interns were assigned real-world tasks to apply what they had learned in practical
scenarios.
Throughout the internship, peer learning was encouraged while mentors provided
continuous guidance and supervision. Daily stand-up meetings were led by the mentor,
where interns presented progress updates on assigned tasks. The mentor also maintained
close interaction with the interns through regular check-ins and visits. An internship
working plan was followed, which outlined the procedures, techniques, and standards to
ensure that the data collected and tasks performed met quality and compliance
requirements.
Table 2.2 Internship Period Details

Internship Start Date: 30ᵗʰ March, 2025

Internship End Date: 30ᵗʰ June, 2025
Office hours: 9:00 AM to 6:00 PM
Working days: Sunday to Thursday
Position: Data Intern
Supervisor: Sristi Khatiwada
2.2 Literature Review
Throughout the internship at eXtenso Data, although direct literature specifically tailored
to the projects undertaken was limited, relevant studies in ETL processes, Big Data
Analytics, data visualization, and automation testing provided critical insights that guided
the work and ensured best practices in Data Engineering.
The study titled “Study of ETL Process and Its Testing Techniques” by Mr. Sujit Prajapati
and Mrs. Sarala Mary (2022) explores the fundamental role of the ETL (Extract,
Transform, Load) process in the data warehousing lifecycle. The ETL process forms the
backbone of data integration by extracting data from multiple sources, transforming it in a
staging area, and finally loading it into the data warehouse. The study further delves into
ETL testing techniques, which are essential for validating data accuracy and integrity
post-transformation.
In another relevant paper titled “Big Data Analytics: A Literature Review Paper”
presented at ENCADEMS 2020, the authors Nikhil Madaan, Umang Kumar, and Suman
Kr Jha (2020) address the challenges posed by the three Vs of Big Data: Volume, Velocity,
and Variety. The paper highlights the limitations of traditional data handling tools in
managing such complex data sets and explores how Big Data Analytics enables
organizations to derive valuable insights from rapidly growing and dynamic data.
Further insight was gained from the article “Evolving Paradigms of Data Engineering in
the Modern Era: Challenges, Innovations, and Strategies” by Alekhya Achanta and Roja
Boina (2023). The paper explores the shift from traditional batch data pipelines to real-
time streaming architectures, driven by the need for speed and scalability. Innovations
such as cloud computing, data lakes, machine learning automation, and self-service
platforms are presented as solutions to modern data engineering challenges
 Chapter 3: Internship Activities

3.1Roles and Responsibilities

During my internship at eXtenso Data, I was responsible for designing and implementing
a data pipeline to automate the extraction, transformation, and loading of sanction-related
data into a centralized MySQL database for further analysis and regulatory compliance
checks. The primary goal of this project was to consolidate and standardize sanctions
data, which is critical for compliance with anti-money laundering (AML), know-your-
customer (KYC), and financial crime prevention regulations. The sanctions data I worked
with came from multiple international regulatory bodies, including the United Nations
(UN), US Department of the Treasury (OFAC), UK Her Majesty's Treasury (HMT),
European Union (EU), Interpol, and Canada
Each of these sources published sanctions lists that contained critical information on
individuals, organizations, and entities subject to financial sanctions.
My Key Responsibilities include
 Data Extraction: Collected data from the various official sources in formats such
as CSV, XML, JSON, and HTML. This involved writing scripts to fetch and parse
data from different URLs.
 Data Cleaning and Transformation: Standardized and normalized the data into
a unified schema by cleaning duplicates, handling missing values, and ensuring
that fields were consistent across all sources. This transformation process was
essential to ensure that all data could be analyzed cohesively.
 Database Design and Loading: Designed relational database tables in MySQL to
store the transformed data. Developed an ETL script to automate the loading of
the data into the database, ensuring a smooth and reliable process.
 Automation and Optimization: Automated the ETL workflow, reducing manual
effort and ensuring the data was updated regularly. Optimized the pipeline to
handle large datasets efficiently.
 Data Export: Used MySQL dump to export the final consolidated dataset into
a .sql file, making it easy to share, back up, and integrate with other systems.
3.2Weekly log

Table 3.1 weekly Log

Week Task
1 1. Introduction to SQL and relational database concepts.
2. Learning basic to advanced SQL queries (Joins, Subqueries, Window
Functions).
3. Hands-on practice with SQL on sample datasets.

2 1. Introduction to Linux OS and terminal usage, and installation on the

device using Docker Desktop.
2. Learning basic Linux commands and shell scripting.
3. Understanding file permissions, process management, and cron jobs.
3 1. Introduction to Big Data and data engineering concepts.
2. Learning how to integrate SQL knowledge with Python.
3. Practiced data manipulation and automation using Python.
4 1. Introduction to Apache Spark and its architecture.
2. Hands-on with PySpark for handling large datasets.
3. Overview of Airflow and creating simple DAGs (Directed Acyclic
Graphs).
5 1. Continued practice with Spark transformations and actions.
2. Implementing more complex Airflow pipelines for scheduling and
monitoring data workflows.
3. Presentation of learning and use-cases on Spark and Airflow, created a
simple DAG that compresses files when uploaded to a minio bucket.
6 1. Introduction to the sanctioned data ETL (Extract, Transform, Load)
task.
2. Extracting data from source files and performing transformations
using Python and Pandas.
3. Storing transformed data into CSV and loading into databases.
7 1. Continued work on the sanctioned data ETL task.
2. Optimization and automation of the ETL process using Python scripts.
 3. Presentation of the entire ETL workflow and documentation of the
process.

3.3 Description of the Project(s) Involved During Internship

My internship at eXtenso Data was centered around ETL pipeline development and data
integration.
My project’s main objective was to collect, standardize, and store global sanctions data
published by multiple international regulatory bodies. These sanctions lists are essential
for compliance processes such as KYC (Know Your Customer), AML (Anti-Money
Laundering), and supplier screening in industries like automotive and fintech. However,
the sanctions data is scattered across different formats and sources, making it difficult to
consolidate into a single, reliable database. By implementing an efficient ETL (Extract,
Transform, Load) pipeline and storing the processed data in a structured MySQL
database, my project aimed to bridge this gap. The project involved several key steps:

1. Requirement Analysis: Understanding the broader compliance and data

integration challenges related to global sanctions data. This involved identifying
the types of information needed from diverse international sources such as the
UN, US OFAC, UK HMT, EU, Canada, and Interpol, and determining how to
extract, normalize, and store this data in a consistent and usable format for
downstream processing.

2. Data Collection: Developing scripts to extract sanctions data from at least five
official international sources, each available in different formats (CSV, XML,
HTML, JSON) and structures.

3. Data Cleaning and Processing: Parsing, standardizing, and transforming the data
into a unified tabular format while resolving inconsistencies, missing fields, and
schema mismatches.

4. Database Integration: Storing the cleaned and structured data into a MySQL
relational database designed for easy querying, analysis, and compliance checks.
 5. Reporting: Exporting the entire consolidated dataset using mysqldump into a .sql
file for backup, archival, and integration into compliance systems.

3.4 Tasks / Activities Performed

During my internship, I participated in a range of duties and endeavors that supported the
development of a global sanctions data integration pipeline. They are:

1. Research and Planning:

 Conducted research on international sanctions lists published by agencies

such as the UN, US OFAC, UK HMT, EU, and others.
 Studied their file formats (CSV, XML, HTML, JSON) and schema
variations.
 Planned the overall project timeline and defined milestones for extraction,
transformation, loading, and reporting phases.

2. Data Collection Development:

 Developed Python scripts to download and extract data from at least five
different official sanctions sources.
 Handled different data structures and formats using libraries such as
requests, xml.etree, json.

3. Data Cleaning and Processing:

 Used Python libraries such as Pandas to parse and standardize data fields
across all sanctions lists.
 Resolved inconsistencies in naming conventions, removed duplicates, and
structured the data into a uniform format.
 Ensured all records followed a unified schema to allow smooth integration
into the database.

4. Database Integration:

 Designed a MySQL database schema optimized for storing structured

sanctions data.
  Used mysql.connector in Python to insert processed data into the MySQL
database.
 Verified referential integrity and ensured that all data could be queried
efficiently.

5. Reporting and Documentation:

 Created a MySQL dump file using mysqldump to generate a .sql backup of

the entire consolidated dataset.
 Documented the complete ETL process, including data sources, Python
scripts used, cleaning steps, and schema definitions.

6. Collaboration and Feedback:

 Regularly communicated with my mentor to provide updates on project

progress.
 Participated in team meetings to discuss challenges, share findings, and
receive feedback for improvements.
 Chapter 4: Conclusion and Learning Outcomes

4.1 Conclusion
My time as an intern at eXtenso Data. has been an ongoing journey of growth and
learning. Working on a challenging project that involves data extraction, transformation,
and loading from global sanctions sources has allowed me to enhance my technical skills
in Python and MySQL while gaining valuable insights into real-world data engineering
workflows.
Although the internship is still in progress, I have already gained hands-on experience in
addressing real business needs through designing an ETL pipeline and dealing with
diverse data formats. Collaborating with the technical team and receiving mentorship has
improved my communication and problem-solving skills, while also deepening my
interest in the fields of data engineering and compliance analytics.
I look forward to completing the internship and continuing to apply what I’ve learned to
the remaining phases of the project. This experience is shaping a strong foundation for
my future academic and professional aspirations, and I’m grateful for the opportunity to
contribute meaningfully while continuing to learn.

4.2 Learning Outcome

As an intern at eXtenso Data, I have been actively applying my academic knowledge in
real-world scenarios and gaining valuable insights into the field of data engineering.
Although the internship is still ongoing, I have already achieved several key learning
milestones:
1. Technical Skill Enhancement
 Gained hands-on experience in web scraping using Python libraries such
as Selenium and Selenium Base.
 Learned how to clean and preprocess raw data using pandas to ensure
accuracy and consistency.
2. Understanding of ETL Concepts
 Started understanding the ETL (Extract, Transform, Load) process by
collecting and transforming data into structured formats.
  Initiated work on integrating cleaned data into a MySQL database.

3. Problem Solving and Adaptability

 Faced real-time challenges such as dynamic web page elements and
inconsistent data formats, helping sharpen my analytical thinking.
 Adapted to changing requirements and learned to debug and improve my
scraping scripts accordingly.
4. Collaboration and Communication
 Collaborated with mentors and IT team members to receive feedback and
stay aligned with project goals.
 Improved my ability to clearly communicate progress and technical
challenges.
5. Ongoing Professional Development
 This experience is reinforcing my interest in data engineering and data
science.
 I am gaining a deeper understanding of how data-driven decisions are
made in a professional setting.
 References
Achanta, A., & Boina, R. (2023). Evolving Paradigms of Data Engineering in the Modern Era:
Challenges, Innovations, and Strategies. International Journal of Science and Research
(IJSR), 12(10), 606–610. https://doi.org/10.21275/SR231007071729
eXtenso Data. (n.d.). Services - eXtensoData . Retrieved May 13, 2025, from
https://www.extensodata.com/services
Madaan, N., Kumar, U., & Jha, S. K. (2020). Big Data Analytics: A Literature Review Paper.
International Journal of Engineering Research & Technology, 8(10).
https://doi.org/10.17577/IJERTCONV8IS10003
Prajapati, Mr. S., & Mary, Mrs. S. (2022). Study of ETL Process and Its Testing Techniques.
International Journal for Research in Applied Science and Engineering Technology,
10(6), 871–877. https://doi.org/10.22214/IJRASET.2022.43931
 Annex
i) Snapshot of code used for performing ETL
ii) Snapshot of code used for inserting Data into the Database
iii)Snapshot of parsing Data from the Data source