Report on Backend
NOTE: This detailed report outlines the backend API's current state and identifies
several issues that need resolution. After conducting thorough research and review, I've
categorized each item into three stages based on priority, the necessary process for
resolution, and the estimated time required to address them. Tasks with higher priority
and shorter completion times are listed first.
# HOW TO PROCEED?
STAGE - 1:
- .env file update
- Enable gzip compression
- Handle Unknown API Request
- API Security
- Upgrade Mongoose to 8.x
STAGE - 2:
- API Documentation
- HTTP Response Status
- Response Schema / Structure
- API URLs Format
STAGE - 3:
- Caching
- Unit Testing
- Logging
[STAGE-1]
# .env file update
HIGH PRIORITY API OPTIMIZATION
A few variables related to Linode/AWS configuration still exist in the config/config.js file
and need to be moved into the .env file.
# Enable gzip compression
HIGH PRIORITY API OPTIMIZATION
Gzip compressing can greatly decrease the size of the response body and hence increase
the speed of a web app. Ref
# Handle Unknown API Request
HIGH PRIORITY API OPTIMIZATION
Add the option to send back a 404 error for any unknown API request
# API Security
HIGH PRIORITY API SECURITY
[X] Set Security HTTP Headers
Need to set security HTTP headers. We could enable or set helmet.js for this which is
suggested by express.js itself [Ref]
[X] Sanitize MongoDB Data
Have to sanitize user-supplied data to prevent MongoDB operator injection.
Middleware: express-mongo-sanitize
[X] Reduce Fingerprint - Instructions / Ref
[X] Rate limit
express-rate-limit is needed to protect the API from being overwhelmed by too many
requests in a short period of time, which could potentially cause the server to crash or
become unresponsive. By limiting the number of requests a single IP address can make
within a certain timeframe, this middleware helps prevent abuse and ensures fair usage of
API.
[X] Request Data Validation
Need to include/implement a request validator to validate the request data before
executing any operation in the database. There a few good options are available from
where we can choose one. Here I have listed a few rich validators we can use.
Express-validator / joi / Vinejs
# Upgrade Mongoose to 8.x
HIGH PRIORITY PACKAGE UPGRADE
Our existing Mongoose version is 6.x, whereas the recent stable version is 8.x. We should
adopt the latest version as soon as possible since we are two major versions behind. It's
crucial for us to leverage new features like enhanced query capabilities, performance
improvements, and updated schema types. The latest version includes optimizations,
security patches, and bug fixes, ensuring better compatibility with recent Node.js and
MongoDB versions.
Process: We have to go through two migration processes
1# Migrating from 6.x to 7.x
2# Migrating from 7.x to 8.x
[STAGE-2]
# API Documentation
MEDIUM PRIORITY API TESTING
Already, we have the Swagger setup for API documentation. We need to restructure and
organize the documentation to ensure that all the APIs are properly documented.
# HTTP Response Status
HIGH PRIORITY API OPTIMIZATION
Standard HTTP status codes will be used instead of using manual code in the response
(code = 0/1 as used in the current API response to handle the response in frontend). The
response will be handled in the front end based on the HTTP status code. The following
status codes will be returned from the API:
200 OK The request was successful, and the server returned the requested
resource.
201 OK The request was successful, a new document was created and the
server returned the created document.
400 Bad Request The server could not understand the request due to invalid syntax.
401 Unauthorized Authentication is required, and the client must authenticate itself to
get the requested response.
403 Forbidden The client does not have access rights to the content.
404 Not Found The server cannot find the requested resource.
500 Internal The server encountered a situation it doesn't know how to handle.
Server Error
# Response Schema / Structure
HIGH PRIORITY API OPTIMIZATION
All the API response formats should be consistent, and the structure of the response
schema should be the same. Here is the plan or proposed structure/schema for all the
response scenarios
[HTTP Methods for Actions]
- GET for retrieving resources.
- POST for creating new resources.
- PUT for updating resources.
- PATCH for partially updating resources.
- DELETE for deleting resources.
1# Retrieve list (HTTP - METHOD: GET, STATUS: 200 OK)
Will return status=success, data=[item list], and pagination = {pagination information}
2# Retrieve a single document (HTTP - METHOD: GET, HTTP STATUS: 200 OK)
Will return status=success and data={item information}
3# Created a document (HTTP - METHOD: POST, HTTP STATUS: 201 OK)
Will return status=success and data={item information}
4# Update/Replace document (HTTP - METHOD: PUT, HTTP STATUS: 200 OK)
Will return status=success and data={item information}
5# Update partial document (HTTP - METHOD: PATCH, HTTP STATUS: 200 OK)
Will return status=success and data={item information}
6# Delete document (HTTP - METHOD: DELETE, HTTP STATUS: 200 OK)
Will return status=success and data=null
7# Error responses
[X] 400 Bad Request
Scenario: Invalid input data
[X] 401 Unauthorized
Scenario: Authentication Required
[X] 403 Forbidden
Scenario: Do not have access to the specific resources
[X] 404 Not found
Scenario: URL not found / IF any resource requested by ID is not found
[X] 500 Internal server error
Scenario: If anything happened in the server
# API URL’s Format
HIGH PRIORITY API OPTIMIZATION
We need to make sure that API URLs are consistent. Consistent API URLs are essential
for ease of use, reducing errors, and improving maintainability. They help developers
quickly understand and predict the API structure, facilitating efficient collaboration and
smoother updates. Consistency ensures a scalable and intuitive design as the API grows.
Proposed suggestions are:
- All the URLs must be in the kebab-case format
- Admin API: All the admin APIs should be prefixed with ‘/admin’
Example: /admin/clients
- List: /items [GET] []
- Create Single Item: /items [POST]
- Single Item: /items/{id} [GET/DELETE/PUT/PATCH]
- Hierarchical Structure: /items/{id}/sub-items
[STAGE-3]
# Chaching
MEDIUM PRIORITY API TESTING
I would recommend adding caching to the API project to boost performance and
efficiency. Caching reduces server load by minimizing database queries and enhances user
experience with quicker response times.
There are various caching mechanisms available, such as In-Memory Caching with Redis,
Database Query Caching, HTTP Caching, and Application-Level Caching with Node
Cache, among others. However, explaining and comparing each type here would make the
documentation too lengthy. Therefore, I'm skipping that part for now. Before
implementing caching, we can provide detailed descriptions of each caching type.
# Unit Testing
MEDIUM PRIORITY API TESTING
We need to add unit testing to our project as soon as possible to ensure that every part of
our project functions correctly as planned. Unit testing is crucial, especially when updating
portions of the code, as it helps to ensure that no other parts of the code are broken. By
running tests before pushing updates, developers can verify that all components are
functioning correctly. It will also help developers to catch bugs early, make code easier to
maintain, and provide a safety net for refactoring.
Jest is the good one for the purpose!
# Logging
MEDIUM PRIORITY API TESTING
I highly recommend adding a robust logging system to our API project. Logging is essential,
especially in production environments, as it helps us track and diagnose internal issues
more effectively than using console logs. This will enhance our ability to monitor
application behavior, identify and fix errors quickly, and ensure the overall reliability and
performance of the system.
For this purpose, I suggest using Winston as our logging package. Winston is an
open-source library available under the MIT License. It supports multiple logging levels,
various output formats, and different destinations, such as console and files. Its
flexibility and powerful features make it an excellent choice for capturing and managing
logs efficiently.
