Scribd
Upload
86 views12 pages

Mongodb AWS Cloud Migration

The document discusses migrating a MongoDB database from an on-premise data center to AWS cloud without any downtime or code changes. It proposes using a lift and shift approach to migrate the 6TB database by setting up a MongoDB replica set in AWS that initially syncs from the on-premise primary node, then promotes the AWS nodes to primary once migration is complete. Bandwidth allocation and potential brief unavailability during the primary switchover are identified as challenges that require mitigation.

Uploaded by

2k.pravin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
86 views12 pages

Mongodb AWS Cloud Migration

The document discusses migrating a MongoDB database from an on-premise data center to AWS cloud without any downtime or code changes. It proposes using a lift and shift approach to migrate the 6TB database by setting up a MongoDB replica set in AWS that initially syncs from the on-premise primary node, then promotes the AWS nodes to primary once migration is complete. Bandwidth allocation and potential brief unavailability during the primary switchover are identified as challenges that require mitigation.

Uploaded by

2k.pravin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Mongodb AWS Cloud

Migration
Lift & Shift Online AWS Cloud Migration

8/19/2021 Pravin Dwiwedi 1


What is MongoDB?
Mongodb is an Open Source, JSON document oriented cross platform NoSQL(Not
only SQL) database. The main features of Mongodb are Flexible Schama, High
Availability & Horizontal Scalability.
Mongodb is a developer friendly database. The structure of the information
is under the control of the developer. Developers adjust and reformat the database
as the application evolves.

8/19/2021 Pravin Dwiwedi 2


How Does Mongodb Work?
• As a database Mongodb stores data in form of JSON documents in the
collections. Collections are like RDBMS tables and documents are like
rows.
• High availability is achieved using replica set which is a group of
MongoDB servers(called replica nodes) that maintain the up-to-date
data set across the servers thus providing automatic failover, data
redundancy, and availability.
• Horizontal scalability is achieved using partitioning and distribution of
the data( based on the certain key data) across a cluster of MongoDB
servers. This technique is called sharding.

8/19/2021 Pravin Dwiwedi 3


Why Should Mongodb Be Used?
• Mongodb stores data in JSON format, so if you have the unstructured
or semi structured diverse data, you should think about Mongodb.
Use case like Product Catalogue, Logs, Session/Cookies data etc are
suitable for Mongodb.
• Mongodb supports horizontal scaling(using data distribution) so if you
have to deal with large amount of data for real time usage then you
should think about it. IOT data, system log data are best use cases for
Mongodb. Also, if your application servers are geographically
deployed then Mongodb would be a great choice.
• If your application requires Hybrid and multi-cloud environment then
Mongodb would be great choice.

8/19/2021 Pravin Dwiwedi 4


Migration Requirement Statements
• We are migrating our all of the applications to AWS without any code
change.
• We have highly available Mongodb 3 nodes cluster running on our single
data Centre. Our application runs 7X24 so Mongodb cluster is operational
7X24. We can not tolerate any downtime.
• We have direct connect to AWS.
• Our servers are running on RHEL with Community Edition of Mongodb 4.4
with 7TB of storage, 60 cores CPU and 512G of RAM.
• Right now we have 6 TB of data. Our growth rate forecast is 3% every year.
• We need to migrate highly available Mongodb cluster to the AWS without
any downtime. We want to migrate all the 6TB data.

8/19/2021 Pravin Dwiwedi 5


Migration Strategy
As per the requirements, all the applications are migrating to AWS
cloud as it is so Mongodb needs migrate to the cloud without any
changes. We are going to adopt here Lift & Shift approach.
Growth rate is 5% each year so we are going to plan an infra which can
support 5 years load and capacity.

8/19/2021 Pravin Dwiwedi 6


High Level Solution Design

8/19/2021 Pravin Dwiwedi 7


AWS Mongodb Infra Setup
• VPC: Dedicated database Virtual private network to create Mongodb
cluster into.
• Subnet:
• Private Subnet for Mongodb servers
• Public instance for Nat Gateway
• Nat Gateway: Connectivity between EC2 instances & Internet Gateway.
• Internet Gateway: Internet connectivity to the EC2 instances for software
upgrades.
• Router: Traffic routing between Private EC2 instances and Internet.
• Direct Connect: An AWS cloud service solution, dedicated private high
speed network connection from on premises to AWS.
• EC2 Instances : i3en.24xlarge; 96 Core CPU/768G RAM/15TB SSD VVMe
8/19/2021 Pravin Dwiwedi 8
• Proposed Design

8/19/2021 Pravin Dwiwedi 9


Migration Process
There are two popular approaches-

• Mongodb will sync all the data from the on-prem primary(or secondary) node to
the newly created secondary node in the AWS cloud using built in feature
IntialSync. This approach involves less manual intervention.
• Manually copy the data files from the primary node and paste it to the newly
created secondary node in the AWS cloud using S3. Then start the secondary
node. Mongodb will sync the delta data between the on-prem primary node and
secondary node using native replication process.
Because we have direct connectivity between data center & AWS using Direct
Connect and have only 6TB of data to migrate so there will not be any data transfer
performance issue. Also, requirement is to perform online migration. So, we are
going to adopt first approach.

8/19/2021 Pravin Dwiwedi 10


• Migration Process Design

8/19/2021 Pravin Dwiwedi 11


Challenges & Risks
• Data Transfer can consume most of the bandwidth during migration so
need to configure bandwidth allocation properly as per the business
requirements.
• During data migration process, secondary node in the cloud will not be
available. So, only two nodes will be available.
• Mongodb cluster will not be available for 2-3 seconds during stepping
down on-prem primary node and promoting cloud nodes as primary node.
Either this can be done during application cut over to the cloud or can be
done during off/non-peak business hours.
• Monitoring & alerting system set up will have to be started from scratch.
• New backup process will have to be set up.

8/19/2021 Pravin Dwiwedi 12

You might also like