Scribd
Upload
34 views11 pages

W7D3 - Functional Redis Integration in Project (19june)

Uploaded by

rahulbadhe630
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
34 views11 pages

W7D3 - Functional Redis Integration in Project (19june)

Uploaded by

rahulbadhe630
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 11

Live Diagram:

Functionally integrating Redis in project.

Funtional requirement.

Validation service.

List of rules, comma seperated rules


we execute those rules.

Payment Attempt Threshold


60mins, only 5payments are allowed.
60 & 5 are configurable.

120mins, 20payment

--

Changing these parameters on-the-fly,


- In next production release.
1-2 weeks
- HotFix
cut branch from master
make changes
test
code review
merge to master
build & deployment in prod
ok done

50 business validations.
10 rules we want to inactive. it should not run.

We want only the fly immediate change in these values:


1. Anytime we inactive a rule.. realtime rule should stop working.
2. As soon as we change the parameter of rule, immediately it should
impact.
1. External property
2. AWS secretManager aws is costing money,only for sensitive info,not functional
data holding

3. Store this data in DB.

Client - even without developers


involment.
Reporting/Backoffice
Feature

what if clients want to directly


modify this data without tech team's
involment?

For every request


we need to get latest list
we need to get latest configured parameters

this mess is problem is caching  we are


going to store data in DB and even if the
client needs to change the values that
cannot be like daily requirements of
client is changing it could be in weeks or
months Cache
-We would keep the configuration data in DB.
(backoffice updates)
- Introduce a Cache to hold the data.
- Make application changes to work with
Cache & DB.

Sure, based on the image you sent, it appears to be a diagram illustrating a cache validation
mechanism used to improve data retrieval performance in a web application. Here's a breakdown
of the components involved:
Validation Cache:
 This is a temporary storage layer located in-memory (RAM) closer to the application
server.
 It stores a copy of frequently accessed data retrieved from the database.
 The validation cache also stores metadata associated with the cached data, such as
timestamps or version numbers.
Database:
 This is the primary data store where all the original data resides.
 The database is responsible for maintaining data integrity and consistency.
Process:
1. Client Request: The user interacts with the web application, triggering a request for
specific data.
2. Cache Lookup: The application first checks the validation cache to see if the requested
data is present.
3. Cache Hit (Valid Data):
o If the data is found in the cache and the validation metadata (timestamp or version
number) indicates it's current, the cached data is retrieved and served directly to
the client. This is the most efficient scenario as it avoids a potentially slower
database access.
4. Cache Miss or Invalid Data:
o If the requested data is not found in the cache, or the cached data is considered
invalid (outdated), the application proceeds to the database.
5. Database Access: The application retrieves the latest data from the database.
6. Cache Update: The application typically updates the validation cache with the fresh data
retrieved from the database, along with the corresponding updated metadata. This ensures
subsequent requests for the same data can be served faster from the cache.
7. Data Delivery: The application delivers the data (either from cache or database) to the
client.
Benefits of Validation Cache:
 Improved Performance: By serving data from the faster in-memory cache, the
application reduces response times and improves overall user experience.
 Reduced Database Load: By offloading frequently accessed data from the database, the
cache can significantly decrease the load on the database server.
Cache Validation:
 The validation mechanism ensures that the cached data remains consistent with the actual
data in the database. This is typically achieved using timestamps or version numbers
stored alongside the cached data.
 If the cached data's validation metadata indicates it's outdated, the cache is bypassed, and
the application retrieves fresh data from the database.
Additional Notes:
 The specific cache invalidation strategies and cache update mechanisms can vary
depending on the chosen caching solution and application requirements.
 Some caching solutions offer write-through or write-behind mechanisms to keep the
cache in sync with the database during data updates.
I hope this explanation clarifies the caching mechanism with validation cache and database. If
you have any questions or would like to know more about specific aspects of cache validation,
feel free to ask!
===
1. Db table design

CREATE TABLE validations.`validation_rules` (


`id` int NOT NULL AUTO_INCREMENT,
`validatorName` varchar(50) NOT NULL,
`isActive` BOOLEAN NOT NULL,
`priority` SMALLINT NOT NULL,
`creationDate` timestamp(2) NOT NULL DEFAULT
CURRENT_TIMESTAMP(2),
PRIMARY KEY (`id`),
UNIQUE KEY (`validatorName`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

select:
select validatorName from validations.`validation_rules`
where isActive = true order by priority

List<String>

Redis
LIST
listOperations.rightPush("PAYMENT_VALIDATORS", value);
List<String> myvalues = null;
listOperations.rightPushAll(key, myvalues);

0 - All Field Validations

5-

10 - All Business Validations.


10 - RULE1
10 - RULE2

Any rule, can have any number of parameters.. and any type of parameters...

CREATE TABLE validations.`validation_rules_params` (


`id` int NOT NULL AUTO_INCREMENT,
`validatorName` varchar(50) NOT NULL,
`paramName` varchar(200) NOT NULL,
`paramValue` varchar(200) NOT NULL,
`creationDate` timestamp(2) NOT NULL DEFAULT
CURRENT_TIMESTAMP(2),
PRIMARY KEY (`id`),
FOREIGN KEY (`validatorName`) REFERENCES `validation_rules`
(`validatorName`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

In lat 60mins, 5 payments allowed.

1. PAYMENT_ATTEMPT_VALIDATION_RULE `paramName`
paymentAttemptThreshold `paramName` 5
2. PAYMENT_ATTEMPT_VALIDATION_RULE `paramName` timeInMins
`paramName` 60
..
..
..
.

PAYMENT_ATTEMPT_VALIDATION_RULE
Map<String, String>

RULE1
Map<String, String>

Redis Hashes
HMSET PAYMENT_ATTEMPT_VALIDATION_RULE paymentAttemptThreshold 5
timeInMins 60 ...:
RestTemplate
HashOperations<String, String, String> hashOps

2. Redis cache data structure


LIST
Hash
3. Spring JDBC to query data from DB (Jdbctemplate)
@Autowired
private NamedParameterJdbcTemplate namedParameterJdbcTemplate;

4. Redis implementation to save & read data from Redis (RedisTemplate)

5. On service startup, checking & loading data from DB to redis cache.


where to write this code???

@PostConstruct
public void processInit() {
//
/*
* Check value in redis
* if not there
* then use DAO class read value.
* Store in Redi
*/
}

Spring bean - scope


(default scope singleton)

6. We need to write new APIs to refresh data in cache.


- Use this from postman
(load all active validation rules)
(load parameters for given validation rule)
- Plugin to Backoffice

7. Explore & use Transaction block.


delete existing &
load new

public void executeTransaction() {


List<Object> txResults = redisTemplate.execute(new
SessionCallback<List<Object>>() {
@Override
public List<Object> execute(RedisOperations operations) throws
DataAccessException {
operations.multi();
operations.opsForValue().set("key1", "value1");
operations.opsForValue().set("key2", "value2");
return operations.exec();
}
});

1. Cache is empty in start..


1. When 1st request come, it should load the data & put in cache.
always 1st request would be slow
from 2nd request it is ok.

2. When service comes up, then automatically check if data is in redis. If not
there, then load the data.
=> Power of simple IF check
on every restart it would try to read DB & store in redis.
every autoscaled ec2 instance.
duplication is possible
IF is mandatory.

Cache
Eviction/expiry - removal from cache
LRU, expiry time.

YES No eviction

10 yes
5 No (I support)
10 AM load fresh values from DB.
10:05 - modifies params.
2hrs
12

To reflect latest data in cache.

When values are mofidied in DB??


How will your cache know it?

Some APIs to reload cache.

1. LIST OF ValidationRules
DB: RULE1, RULE2, RULE3
Redis: RULE1, RULE2, RULE3

RULE3 - INACTIVE

DB: RULE1, RULE2


Redis: RULE1, RULE2, RULE3

1.1: deleted all existing value./


1.2: Insert all new values.

2. refresh the Parameters


10validation rules
3 rules: have configuration params
RULE1: 5 params
3 value

inputs: RULE_NAME
refresh the entire data:

RULE_NAME, parameter
RULE_NAME, LIST<parameter>

Calls the refresh cache API


1.1: deleted all existing value.
1.2: Insert all new values.

Parallel many live customer are trying to make payment...

100 people

If there is no rule configured.. fail the current


payment process.
You failed 100 payment.,even you fail the request of 100 people but
let fraud didn’t happen

=> running multiple rules as transaction block.


1.1: deleted all existing value.
1.2: Insert all new values.

=> redis Single threaded.

YOU WILL NOT GET JOB, based of whether you WANT IT.
YOU WILL GET JOB, if you DESERVE

You might also like