Assignment #6

Creating a final project for a NoSQL database

Project members:
● Dias Rysbekov, CSE-2401M - main frontend of the project
● Bekzat Aitbekov, CSE-2401M - main backend of the project

Github:
● https://github.com/Yangdux/AdvancedDB
Frontend code under “main” branch
Backend code under “backend” branch

Project goals:
1. Apply NoSQL database principles to store and retrieve user data and product
information.
2. Implement a collaborative filtering recommendation system.
3. Develop a user-friendly environment with personalized recommendations.
4. Gain experience in performance testing and optimization.

Project deliverables:
● A detailed project proposal outlining the chosen NoSQL database, data modeling
approach, and recommendation algorithm.
● A functioning with the recommendation system integrated.
● Documentation on how to set up and run the project.
● Performance analysis and optimization report.

Project description:
● Our project is a mobile application for a shop selling technical appliances such as
laptops, mobile phones, computer accessories and internal parts. The application was
written in Flutter mobile development language created by Google and MongoDB
NoSQL database by integration with the Java environment of development. Project
has all the requested functionality:
○ Users can register and login to their respective accounts.
○ Users have profile pages that show their account data, such as full name,
email, age, gender and their balance of money they have in their account.
○ Users can see all the products that are listed in our database, including
respective data of the products such as description, quantity, price, category,
number of likes on that product and its image. The working database had 120
products and 10 user accounts.
○ We have implemented a recommendation engine and collaborative filtering
functionalities. First is based on user activities such as likes given and
purchases made. Collaborative filter assesses all users’ actions in order to give
the product list to the users. This will be explained more below.
 ○ The app has a user history page which shows all the products that have been
purchased by the user and products that have been liked, and viewed.
○ It also manages the search functionality through the backend api call which
receives a string type keyword and returns a list of products satisfying this
request.
○ All the requests to the database are done through api calls.
○ We also completed performance testing on our recommendation engine, which
will be shown and discussed below.
Project stack:
● Flutter mobile development language, with its packages (libraries).
● Java 17
● Libraries under Spring Framework: Data JPA, Security, Boot, MVC
● Ngrok is used as a server to connect the backend with frontend

Stack reasoning:
● We have decided to write a mobile application over web application because Dias
Rysbekov is a Junior Mobile Developer at WonkAI company. He has a sufficient level
of professionalism to write mobile apps that are both efficient and pleasant in UX/UI.
Flutter is an open source cross-platform language framework written in Dart
programming language created by Google. It can be used to write efficient web and
mobile apps, for ios and android devices. It has great community support and demand
on the market.
● The reason for choosing Java as a backend is that Bekzat Aitbekov is an Java intern in
EPAM Systems. He had plenty of experience implementing API using Java and
Spring Framework. Java is a versatile language, used all around the world to write
web and other applications. The Spring Framework is an application framework and
inversion of control container for the Java platform. The framework's core features
can be used by any Java application, but there are extensions for building web
applications on top of the Java EE (Enterprise Edition) platform.
● MongoDB is used as a NoSQL database, because both Bekzat and Dias are the most
familiar with it, compared to other DBs. In addition, since our main choice for
frontend was Flutter MongoDB was a go-to choice, because Flutter has the most
amount of supported libraries for NoSQL databases for MongoDB after Firebase DB.
We did not choose Firebase because it has many predefined supporting systems that
can be used and thus making the goals of the project too easy to reach.

Project analysis:
Frontend of the project:
1. User registration and profiles
For this section of our project Dias have created three pages: Login page, Registration
page and Profile page. In our github repository, which we have provided two pages are
responsible for these pages: LoginPage.dart and ProfilePage.dart. In the login page users can
either login using their emails and passwords or press the “Sign Up” text button to move to
the registration page and create a new account filling their name, email and password to the
forms provided. Passwords have validation checks both on front and back end of the app to
be sure that no password is less than 8 symbols, has 1 upper case, 1 special symbol
characters. Here are the images of the successful pages.

Picture 1. Login page Picture 2. Registration page Picture 3. Profile page

 2. Product catalog:
Product catalog can be seen in two ways. These are first, on our home page which is
also our recommendation page, where we output the results of recommendation API call and
of our collaborative filtering API call. Second is, on the categories page, where all the
products are presorted by their categories. We have 12 categories, such as laptops, phones,
tablets, gpus and others. The respective api calls for returning a list of the categories and for
the list of products in each category are written in our app, so that whenever our database
grows in the possible future the project will not break. Here are the respective images of the
pages.

Picture 4. Home page Picture 5. Categories page Picture 6. Products page

Picture 4 is a Home page which shows the results of the recommendation engine and in case
if a person does not have recommendations it returns the whole list of the items. Picture 5 is a
Categories page where all of our 12 categories are returned. Ant the Picture 6 is a products
page for monitors category as an example. As you can see here the images of products are the
same because we fairly did not have enough time to properly search for the pictures of items
and place them appropriately. Thus we wrote a script to assign every item a picture of their
category. Thus all the monitors have the same picture, same as for all other categories.
3. User History
Our flutter application also has a user history page which stores the purchase and likes
history of the user account. This data is also retrieved from our database since every
 purchase and like is recorded. From the front part we simply wrote the respective API
call to access that data. On this page there are two tabs, first is the “Purchases” tab
and second is the “Likes” tab. Both likes and purchases history lists are returned from
the database through our API calls.

Picture 7. Purchase History Picture 8. Likes History

4. Recommendation Engine and Collaborative Filtering

Our home page is where both recommendation engine and collaborative filtering
functionalities are implemented. Picture 4 is the home page, as you can see there are
two sliders under “Our hot trends” and “Recommendations for you” texts. Hot trends
is the horizontal slider which shows the collaborative filtering results and
recommendations is a vertical slider which shows the recommendations results.
Vertical slider is what we may call limitless since it does return all the items if the
recommended items are finished, while horizontal slider is limited to 15 of the most
purchased and liked items. This was made because they will be pretty similar lists if
they are both limitless.

5. Search Functionality
Search functionality in the frontend of the project is pretty similar to everything else.
We have search bars across our application and when we write a certain word to
search and press the search icon on the search bar it makes an API call which returns
results for that search. These results are shown as a separate list over the page where
the search bar was located. To finish the searching session users simply need to press
the return text button. Pictures 9 and 10 are the examples of how this is done.

Picture 9. Search bar functionality Picture 10. Search results

Backend
Stack of the backend in more details:
● Java 17,
Spring Boot,
● Spring MVC,
● Spring Data JPA,
Spring Security,
● Hibernate Validation

1) User Registration and Profiles:

User registration and overall security is implemented with Spring Security. Each time an
existing user tries to login, he sends a POST request to the server with email and password in
body. In return, the server sends JWT Token, which times out after 24 hours.

Picture 11. Login

2) Models and catalog in MongoDB

We have 4 main models:

● Users
● Products
● Purchases
● Categories

The next screenshots are representation of these models in Java and MongoDB respectively:
 ● User

Picture 12. User model

● Product
 Picture 13. Product model

● Purchases

Picture 14. Purchases model

● Category
 Picture 15. Category model

3) User History
User can view his purchase history and list of liked products.
Purchase history under this endpoint: /api/purchases/{userId}

Picture 16. Purchase history

List of liked products under this endpoint: api/products/liked/{userId}

 Picture 17. Liked products

4) Recommendation Engine
Recommendation engine work by the following logic:

We get the lists of categories of purchased and liked products for a particular user. We append
the rest of products from other categories. This list is sorted by likes in descending order and
returned.

If the user is new, the list of liked and purchased categories is empty. Then all products under
all categories is sorted by likes and returned

Picture 18. Api call for recommendations

Picture 19. Recommendation engine code
 Picture 20. Recommendation engine work

5) Collaborative filtering
This functionality works with almost the same logic as the recommendation engine.
Products that were liked the most are put into the list, sorted by likes, and the top 15 products
are returned.

Picture 21. Api call for collaborations

Picture 22. Collaborations code

 Picture 23. Collaborations work

6) Search Functionality
The search query of a user is sent to this endpoint:

Picture 24. Search API call

Firstly we compare the query among categories. If nothing is found, compare it with the
product description. All found products are returned to the user.

Picture 25. Search function code

Examples:
Looking for products with “16” in description

Picture 25. Search function example

Looking for products under category “cpus”

Picture 26. Search function example

7) Database
The connection to the database is with Spring Data JPA for MongoDB

The connection url:

spring.data.mongodb.uri=mongodb://localhost:27017/prac6
Basic CRUD operations are implemented automatically by Repositories

Example:

Picture 27. Repository code

All default implementations use id of the document

If custom methods needed (find by name, find by category, etc.), method names must clearly
indicate that:

Picture 28. Repository code

8) Performance

We are testing the performance of the app based on time to load two sets of products
The first set (which is used in the app) contains 120 products. In the second set we have 1000
decoy products. The screenshots show these sets respectively.

Here you can see the screenshots of documents from MongoCompass.

First set:

Picture 29. 120 products collection

Second set:

Picture 30. 1000 products collection

Here is how the load time is measured:

1) First, we created a separate endpoint for the test.

Picture 31. API calls

For this test, we created a separate model as well, “ProductTest”. It has the same
fields as Product, except “img” and “likedBy”.
As shown in this screenshot, first we start the timer, call for a method that loads all
documents from a collection, and stop the timer, showing it in console.
2) We go to this endpoint in Postman
Getting 120 recommendations:

Picture 32. 120 products test

Getting 1000 recommendations:

Picture 33. 1000 products test

 Results:

Picture 34. Test results

As can be seen from the results, load time significantly increases with the number of products
in the collection.