MMTC Backup Approaches

As part of Data archival, we have tried following approaches which includes default options
available in MySQL Community. We have tried out following approaches:

1. Live Replication Approach

2. Incremental Backup-Restore
3. Point in Time Recovery
4. Data Archival Softwares

1.Live Replication Approach

Figure 1: Database Live Replication

Overview

It facilitates live data replication from master node MySQL database server to one or more
slave nodes of MySQL database.

Replication Scenarios

Consider employee database on both master database server and slave database server and also
both them are in sync. Below are the scenarios considered in replication.

Scenario 1:

In below diagram at any time T1, master Database and slave database having same numbers of
records.

R1 R1
R2 R2
R3 R3
R4 R4
R5 R5
 Master Slave

Scenario 2: In below diagram at any time T2, inserting 1 new record, updating 1 existing record,
and deleting 1 existing record in master node then the same records would get reflected in slave
node as expected.

R1 R1
R2 R2
R3 R3
R4_Update R4_Update
R5_Update R5_Update

Master Slave

Observation:

Replication approach maintains same state of data across master and slave, so it would not be
possible to use it for MMTC data archival requirements.

2. Incremental Backup / Restore Approach

Figure 2: Incremental Backup/Restore Diagram.

Overview

An incremental backup is a type of backup that only copies records that have changed since the
previous backup.

For example, if a full backup was performed on Monday, Tuesday's incremental will back up all
changed records since Monday's backup. However, Wednesday's incremental will only back up
records that have changed since Tuesday's incremental backup and so on until another full
backup is performed.
Scenarios

Scenario 1: Considering below diagram 2,1, at time T1(End of March), full back up is taken on
primary database and it is restored on secondary database. So now both databases on primary
and secondary database are in sync.

Full Back-up Full Back-up Restore

January January

February February

March March

Primary Database Secondary Database

Figure 2.1 Full Backup restore

Scenario 2:

Requirement: Assume that we have to keep latest 3 months of data on primary Database.

Consider below diagram 2.2, incremental backup is taken at time T2 (End of June), we should
have data from January to June on Primary Database. Now when we take the incremental backup
(which contains all the data including all inserts, updates, and delete since April to June) since
the previous full backup.

Now when we restore this backup on secondary Database, will restore data from April till June
and hence it will contain actual data from January till June on secondary Database.

Once latest incremental backup restored successfully on secondary Database, will delete data
from January to March, in order to fulfill our requirement.

As we have Data from January to June on secondary Database, and on Primary Database we data
from April to June. It is working as expected.

Incremental Backup
April January
Restore
Incremental Backup
May February

June March

April
Newly inserted data
in first incremental After restoring first May

backup incremental backup June

Primary Database Secondary Database

Figure 2.2 First Incremental Backup and restoration
Scenario 3:

Requirement: Assume that we have to keep latest 3 months of data on primary Database.

Consider below diagram 2.3, incremental backup is taken at time T3 (End of September), we
should have data from April to September on Primary Database. Now when we take the
incremental backup (which contains all the data including all inserts, updates, and delete
including January to March- as part of delete at time of T2) since the last incremental backup.

Now when we restore this backup on secondary Database, will restore data from July till
September but it will delete data from January to March on secondary Database. Hence it fails to
maintain record from January till September.

Incremental Backup

Deleting the records After restoring

from January till second incremental
March. backup. Records will
get deleted from
April April
database.
May
Incremental Backup May
Restore
June June

July
Newly inserted data After restoring July
in second second incremental
August August
incremental backup. backup new data
er gets added. September

Primary Database Secondary Database

Diagram 2.3 Second Incremental Backup and restoration

Observation

Using Incremental backup-restore approach it would not be possible to use it for MMTC data
archival requirements.