CST8333 Programming Language Research Project

Reflective Report
Name-Raghav Dhir
Student No.-
Evidence of Learning – CST8333 Programming Language Research Project
Throughout the CST8333 Programming Language Research Project, I have significantly
enhanced my understanding of Python, deepening both my technical proficiency and my
ability to solve real-world problems. This journey allowed me to explore essential
programming concepts and apply them to complex, practical tasks. In this reflection, I will
elaborate on the key skills and knowledge I have gained, particularly in the areas of data
cleaning, visualization, database integration, CRUD operations, threading, and overall software
development practices. This reflective process not only covers what I have learned but also
how these skills have shaped my approach to programming and problem-solving.

Page 1 of 7
1. Data Cleaning and Preprocessing
Before this project, I had a limited understanding of the complexities involved in working with
real-world datasets. I was accustomed to working with clean, well-structured data in academic
examples, but real-world data is often messy, incomplete, and noisy. The data cleaning and
preprocessing phase was a pivotal learning experience.
 Handling Missing Data: I started by exploring how to handle missing or incomplete data.
The pandas library proved invaluable for this task. I learned to identify missing data
using methods like isnull() and notnull(), which helped me assess the extent of missing
values in the dataset. From there, I explored two primary approaches: filling in missing
values (fillna()) and removing rows with missing data (dropna()). Understanding the
trade-offs between these approaches was crucial; for example, filling missing data with
the mean or median could introduce bias, while dropping rows could result in a loss of
valuable data. This learning process taught me how to assess the impact of missing data
on analysis and how to choose the most appropriate method for handling it.
 Data Transformation: Another critical aspect was converting data types to ensure
consistency across the dataset. I learned how to use pandas to cast columns to
appropriate data types (e.g., converting a string column to datetime or integer types)
using methods like astype(). These transformations were essential for ensuring that
numerical values were in the correct format, preventing errors during data analysis and
modeling.
 Filtering and Cleaning Irrelevant Information: Data cleaning also involved removing
irrelevant or unnecessary information. I used pandas functions such as drop() to remove
unwanted columns, and filtering operations like query() or boolean indexing to keep
only the necessary rows. For instance, I often filtered out rows with invalid or erroneous
values (such as negative prices in a dataset that should only contain non-negative
values).
This phase not only helped me prepare data for further analysis but also taught me the
importance of being meticulous in cleaning data before jumping into more advanced tasks like
modeling or visualization.

2. Data Visualization
Page 2 of 7
Data visualization became one of the most enjoyable and rewarding aspects of this project. It
allowed me to turn raw data into visual insights, making it easier to understand trends,
distributions, and relationships within the data.
 Matplotlib and Seaborn: I started with basic visualizations using matplotlib, learning
how to plot line charts, histograms, scatter plots, and bar charts. These visualizations
were helpful for understanding trends, like changes in a variable over time or the
distribution of values across different categories. For example, using a histogram, I was
able to determine whether a dataset had a normal distribution or if skewness was
present.
 Boxplots and Identifying Outliers: One of the most valuable insights I gained was from
using boxplots to identify outliers. Boxplots offered a clear visual representation of data
spread and showed potential outliers that might need special handling. This was a key
step in ensuring data quality, as outliers can distort statistical analyses or modeling.
 Customizing Plots: As I progressed, I learned how to customize plots to improve their
readability and aesthetic appeal. Adding titles, axis labels, legends, and color schemes to
the plots made them more informative and professional. This customization helped me
communicate the insights derived from the data more effectively. For example, I used
color coding to distinguish between different categories in a bar chart or adjusted axis
limits to better focus on a specific range of values.
Visualization taught me the importance of clarity and context in presenting data. Effective data
visualization not only enhances understanding but also supports decision-making by conveying
insights in a digestible format.

3. Database Integration with SQLite

Page 3 of 7
Before this project, I had learned about databases primarily in a theoretical context, but this
project gave me the opportunity to apply database concepts practically by integrating SQLite
into my Python applications.
 Using SQLite in Python: I learned how to use the sqlite3 module to connect Python with
an SQLite database. This allowed me to perform database operations like creating a
database, defining tables, and executing SQL queries directly from within my Python
code. I became proficient in creating tables, inserting data, and structuring databases in
a way that made it easy to retrieve and manipulate data efficiently.
 Executing SQL Queries: I also became comfortable with writing SQL queries to interact
with the database. I learned how to use basic SQL commands, such as SELECT, INSERT,
UPDATE, and DELETE. For example, I used SELECT to retrieve specific records based on
certain conditions, which helped me filter the data based on user input or business logic.
 Data Retrieval and Presentation: A significant aspect of working with SQLite was
learning how to retrieve data from the database and present it in a meaningful way. I
created Python functions that queried the database, processed the results, and
displayed them in a user-friendly format. For instance, I formatted the results of SQL
queries into tables or lists that users could easily read and understand.
By integrating SQLite with Python, I not only enhanced my programming skills but also
developed a deeper understanding of how databases can be used in real-world applications to
manage and manipulate data efficiently.

4. CRUD Operations

Page 4 of 7
The ability to implement CRUD (Create, Read, Update, Delete) operations was a key learning
milestone in this project. These operations are fundamental to most real-world applications,
and mastering them allowed me to build an interactive and dynamic console-based
application.
 Building Interactive Applications: By using Python's input functions and combining
them with CRUD operations, I was able to build a simple yet functional application
where users could add, view, update, and delete records in the dataset. For example, I
implemented an interactive menu system that allowed users to select what operation
they wanted to perform. After the user made a selection, the program would execute
the corresponding CRUD operation on the database or dataset.
 Designing User-Friendly Interfaces: This project emphasized the importance of
designing user-friendly interfaces, even for console-based applications. I made sure that
the prompts were clear, and the output was structured in a way that was easy for users
to follow. This not only improved the usability of the program but also deepened my
understanding of user experience (UX) design principles, even in a text-based
environment.
Through this experience, I developed a better appreciation for the complexity of software
development, as real-world applications often require handling user input, managing data
persistence, and ensuring robust error handling.

5. Threading for Performance

Page 5 of 7
In the latter stages of the project, I delved into the concept of threading to improve the
performance and responsiveness of my application. Initially, I was unfamiliar with the
challenges of concurrent programming, but threading helped me understand how to execute
multiple tasks simultaneously.
 Concurrent Execution: I learned how to use Python's threading module to run multiple
tasks concurrently, allowing my application to remain responsive even when performing
long-running tasks such as data processing or calculations. For example, when querying
the database or performing time-consuming computations, I used threads to run these
tasks in parallel, which significantly reduced the overall execution time of the program.
 Improving User Experience: Threading allowed my application to handle multiple tasks
at once without freezing or blocking the user interface. This was especially important
when working with large datasets, as it ensured the application could continue running
smoothly without delays.
Threading not only enhanced the performance of my project but also taught me the principles
of concurrent programming and how to balance the trade-offs between simplicity and
performance in software design.

6. Reflections on Learning

Page 6 of 7
Reflecting on my journey through the CST8333 Programming Language Research Project, I can
confidently say that I have gained a deep understanding of Python's capabilities and how to
apply them to real-world problems. The project taught me how to break down complex tasks
into manageable chunks, ensuring that I could tackle each challenge systematically.
I also gained valuable experience with libraries like pandas, matplotlib, seaborn, sqlite3, and
threading, which are commonly used in data science and software development. This
knowledge has equipped me with the skills needed to build data-driven applications, manage
databases, visualize insights, and optimize program performance.
Moreover, the project helped me develop essential software engineering practices, such as
debugging, troubleshooting, and time management. I learned how to approach problems
methodically, conduct research to find solutions, and test my code thoroughly to ensure its
functionality.
In conclusion, this project has not only sharpened my technical programming skills but also
enhanced my problem-solving abilities and prepared me for future challenges in software
development. The hands-on experience with Python and its vast ecosystem of libraries has laid
a strong foundation for my continued growth as a programmer.
in cleansing data before jumping into

Page 7 of 7