Case Study #1

Memi Lavi
www.memilavi.com
A Real World Application
Application Introduction

Defining Requirements

Components Mapping

Technology Stack Selection

Architecture Design
Dunderly
Your Paper Source
Dunderly

• Sells Paper Supplies

• Printer paper, Envelopes, etc.

• Needs a new HR system

• Managing employees,
salaries, vacations, payments
Dunderly
Requirements

Functional Non-Functional
What the system should do What the system should deal
with
1. Web Based
2. Perform CRUD operations on employees
3. Manage Salaries:
1. Allow manager to ask for employee’s
salary change
2. Allow HR manager to approve / reject
request
4. Manage vacation days
5. Use external payment system
Dunderly
NFR - What We Know

1. Classic Information System

2. Not a lot of users

3. Not a lot of data

4. Interface to external system

Dunderly
NFR - What We Ask

1. “How many expected concurrent users?” 10

2. “How many employees?” 250

3. “What do we know about the external

Payment system?”
Dunderly
Payment System

• Legacy system, written in C++

• Hosted in the company’s servers farm

• Input – only files 

• File received once a month

Dunderly
Data Volume

• 1 Employee = ~1MB in data

• Each employee has ~10 scanned documents (contract, reviews etc.)

• 1 Scanned Document =~5MB

• Total storage for 1 employee = ~51MB

Dunderly
Data Volume – Cont.

• Company expects to grow to 500 employees in 5 years

• Total storage: 51MB X 500 employees = 25.5GB

• Not a lot, but:

• Need to consider document storage

Dunderly
SLA

4. “How critical is the system?” Not Very Critical

Dunderly
Requirements

Functional Non-Functional
What the system should do What the system should deal
with
1. Web Based
1. 10 Concurrent users
2. Perform CRUD operations on employees
2. Manages 500 users
3. Manage Salaries:
3. Data volume forecast: 25.5GB
1. Allow manager to ask for employee’s
1. Relational & Unstructured
salary change
4. Not mission critical
2. Allow HR manager to approve / reject
5. File-based interface
request
4. Manage vacation days
5. Use external payment system
Dunderly
Components

Employees Salary Vacation View

Based on requirements:
Service Service Service Service
1. Entities: Employees, Vacation, Salary
Performs Returns static
2. Interface to the Payment System CRUD Salary approval
Employee’s
Vacation files to the
Operations on workflow browser
Management
Employees (HTML, CSS,
JS)
Payment Payment Q: Single or Per
System Interface Service Data
Store?
Sends payment Data Store A: Data is
data to payment
system shared between
services, so a
Single Data
Logging Store is better
Dunderly
Messaging
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Logging Service

- Very Important

- Other services use it

Dunderly
Logging - Questions

1. Is there an existing logging mechanism in

No
the company?

2. Develop our own or use 3rd party?

Dunderly
Logging - Alternative
Dunderly
Logging - Alternative

ELK:

• Powerful data store (Elastic)

• Import log from many sources (Logstash)

• Great viewer with filter capabilities (Kibana)

Dunderly
Logging - Alternative

But:

No
• Requires maintenance
G o
• Quite complicated to install and setup

• Suitable mainly for large, data-intensive systems

Dunderly
Logging Service

Steps:

- Decide on Application Type

- Decide on Technology Stack

- Design the Architecture

Dunderly
Application Type

What it does:

- Read log records from queue

- Validate the records

- Store in data store

Dunderly
Application Type

What it does:

- Read log records from queue

- Handle the records

- Save in data store

 Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
 Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
 Technology Stack

For:

• Component’s Code

• Data Store
 Technology Stack

Code Should:

• Access Queue’s API

We’re familiar with Microsoft stack, so we are
• Validate the data expert in .NET and SQL Server

• Store the data

Technology Stack
 Architecture

User Interface / Service

Interface

Business Logic

Data Access

Data Store
 Architecture

User Interface / Service

Interface

Business Logic

Data Access

Data Store
 Logging Service

Polls the Queue every few

Polling seconds for log records

Dependency Injection
Business Logic Validates the records
using
Microsoft.Extensions.DependencyInjection

Saves the records in the

Uses Entity Framework Data Access data store

Data Store
Dunderly
Logging Service Redundancy

X
Logging Service
Dunderly
Logging Service Redundancy

Logging Service - Active / Active

Is Alive - Avoid duplicate

Logging Service reads?

Dunderly
Logging Service Redundancy

X
Logging Service

Is Alive

Logging Service
Dunderly
Logging Service Redundancy

X
Logging Service

Is Alive? No!

Logging Service
Dunderly
Logging Service Redundancy

X
Logging Service

Is Alive? No!

Logging Service
Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
View Service

What it does:

- Get requests from the end users’ browsers

- Returns static files (HTML / CSS / JS)

Dunderly
Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
Dunderly
Technology Stack

.NET Core has a great support for Web Apps

So…
Dunderly
Technology Stack
 Architecture

User Interface / Service

Interface

Business Logic

Data Access

Data Store
Dunderly
Architecture

User Interface

Business Logic

Data Access

Data Store
Dunderly
View Service Redundancy

Load Balancer

View Service View Service

Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Employees Service

What it does:

- Allows end users to query employees’ data

- Allows performing actions on data (CUD)

What it doesn’t:

- Displays the data

Dunderly
Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
Dunderly
Technology Stack – Dev Platform
Dunderly
Technology Stack – Database

Employee Data (Relational) Documents

?
Dunderly
Technology Stack – Database

Document (BLOB) Storage Alternatives

Relational Database

File System

Object Store

Cloud Storage
Dunderly
Document (BLOB) Storage Alternatives
Alternative Description Examples Pros Cons
Relational Store the document in a SQL Server’s Part of the app Clunky syntax,
specialized column type FILESTREAM, transaction Limited size
Database designed for BLOBs Oracle’s BLOB type Part of the DB’s
backup / DR
Dunderly
Document (BLOB) Storage Alternatives
Alternative Description Examples Pros Cons
Relational Store the document in a SQL Server’s Part of the app Clunky syntax,
specialized column type FILESTREAM, transaction Limited size
Database designed for BLOBs Oracle’s BLOB type Part of the DB’s
backup / DR

File System Store the document in a File System (duh…) Unlimited size Not part of
file, and hold a pointer Easy to execute transaction,
to it in the DB Unmanageable
Dunderly
Document (BLOB) Storage Alternatives
Alternative Description Examples Pros Cons
Relational Store the document in a SQL Server’s Part of the app Clunky syntax,
specialized column type FILESTREAM, transaction Limited size
Database designed for BLOBs Oracle’s BLOB type Part of the DB’s
backup / DR

File System Store the document in a File System (duh…) Unlimited size Not part of
file, and hold a pointer Easy to execute transaction,
to it in the DB Unmanageable

Object Store Use special type of store CEPH Great scale Complex setup
mechanism that Unlimited size Dedicated knowledge
specializes in BLOBs New product in the
mix
Dunderly
Document (BLOB) Storage Alternatives
Alternative Description Examples Pros Cons
Relational Store the document in a SQL Server’s Part of the app Clunky syntax,
specialized column type FILESTREAM, transaction Limited size
Database designed for BLOBs Oracle’s BLOB type Part of the DB’s
backup / DR

File System Store the document in a File System (duh…) Unlimited size Not part of
file, and hold a pointer Easy to execute transaction,
to it in the DB Unmanageable

Object Store Use special type of store CEPH Great scale Complex setup
mechanism that Unlimited size Dedicated knowledge
specializes in BLOBs New product in the
mix

Cloud Storage Store the documents in Azure’s Storage Great scale Requires internet
one of the public cloud Account Easy to execute connection
storage mechanisms AWS’s S3 Cost
Dunderly
Technology Stack – Database

Employee Data (Relational) Documents

?
Dunderly
Technology Stack – Database

Employee Data (Relational) Documents

- Documents are
small (~1MB)
- Already exists
- Part of the app
Dunderly
Architecture

Service Interface

Business Logic

Data Access

Data Store
Dunderly
API

• Get full employee details by ID

• List of employees by parameters

• Add employee

• Update employee details

• Remove employee Not physical delete!

Dunderly
API – Cont.

Q: Do we need a separate
• Add document
Document Handler service?

• Remove document
A: Since only the
• Get document Employee entity requires
docs, then no.
• Retrieve documents by parameters
Dunderly
API

Functionality Path Return Codes

Get employee details by ID GET /api/v1/employee/{id} 200 OK

404 Not Found

List employees by parameters GET /api/v1/employees?name=…&birthdate=… 200 OK

400 Bad Request

Add employee POST /api/v1/employee 201 Created

400 Bad Request

Update employee details PUT /api/v1/employee/{id} 200 OK

400 Bad Request
404 Not Found

Remove employee DELETE /api/v1/employee/{id} 200 OK

404 Not Found
Dunderly
API

Functionality Path Return Codes

Add document POST /api/v1/employee/{id}/document 201 Created

404 Not Found

Remove document DELETE 200 OK

/api/v1/employees/{id}/document/{docid} 404 Not Found

Get document GET /api/v1/employees/{id}/document/{docid} 200 OK

404 Not Found

Retrieve documents for employee GET /api/v1/employees/{id}/documents 200 OK

404 Not Found
Dunderly
Employee Service Redundancy

Load Balancer

Employee Service Employee Service

Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Salary Service

What it does:

- Allows managers to ask for an employee’s salary change

- Allows HR representative to approve / reject the request

Dunderly
Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
Dunderly
Technology Stack
Dunderly
Architecture

Service Interface

Business Logic

Data Access

Data Store
Dunderly
API

• Add salary request

• Remove salary request

• Get salary requests

• Approve salary request

• Reject salary request

Dunderly
API

Functionality Path Return Codes

Add salary request POST /api/v1/salaryRequest/ 200 OK

400 Bad Request

Remove salary request DELETE /api/v1/salaryRequest/{id} 200 OK

404 Not Found

Get salary requests GET /api/v1/salaryRequests 200 OK

Approve salary request POST /api/v1/salaryRequest/{id}/approval 200 OK

404 Not Found

Reject salary request POST /api/v1/salaryRequest/{id}/rejection 200 OK

404 Not Found
Dunderly
Salary Service Redundancy

Load Balancer

Salary Service Salary Service

Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Vacation Service

What it does:

- Allows employees to manage their vacation days

- Allows HR to set available vacation days for employees

Dunderly
Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
Dunderly
Technology Stack
Dunderly
Architecture

Service Interface

Business Logic

Data Access

Data Store
Dunderly
API

• Set available vacation days (by HR)

• Get available vacation days

• Reduce vacation days (by employees)

Dunderly
API

Functionality Path Return Codes

Set available vacation days PUT /api/v1/vacation/{empid} 200 OK

404 Not Found

Get available vacation days GET /api/v1/vacation/{empid} 200 OK

404 Not Found

Reduce vacation days POST /api/v1/vacation/{empid}/reduction 200 OK

Dunderly
Vacation Service Redundancy

Load Balancer

Vacation Service Vacation Service

Dunderly
Components
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Payment Interface

What it does:

- Queries the database once a month for salary data

- Passes payment data to the external payment system

Dunderly
Application Type

• Web App & Web API

• Mobile App

• Console

• Service

• Desktop App
Dunderly
Technology Stack
Dunderly
Architecture

Timer

Business Logic

Data Access

Data Store
Dunderly
Payment Interface Redundancy

Payment Interface

Is Alive

Payment Interface
Dunderly
Messaging
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Technology Stack – Queue

Queue Alternatives:
Self Developed

RabbitMQ

Kafka
Dunderly

Alternative Description Pros

Rabbit MQ General purpose message-broker Easy to setup
engine Easy to use
Dunderly

Alternative Description Pros

Rabbit MQ General purpose message-broker Easy to setup
engine Easy to use
Apache Kafka Stream processing platform Perfect for data intensive
scenarios
Dunderly
Technology Stack – Queue

Queue Alternatives:
Self Developed

- No data stream involved

RabbitMQ - Easy to use

Kafka
Dunderly
Logic Diagram
Users’ Browser

HTML Page
REST API / HTTP

Employees Salary Vacation View

Service Service Service Service

Queue

Payment Payment
Data Store Logging
System File Interface
Dunderly
Technical Diagram

Employees Salary Vacation View

Service Service Service Service
.NET Core .NET Core .NET Core .NET Core

Queue
RabbitMQ
Payment Payment
Data Store Logging
System File Interface
.NET Core .NET Core

SQL Server (data

+ BLOBs)
Dunderly
Physical Diagram

LB LB LB LB
Employees Salary Vacation View
Employees Salary Vacation View
Service Service Service Service
Service Service Service Service

Queue

Payment Payment
Payment Data Store Logging
System File Interface Logging
Interface