Академический Документы
Профессиональный Документы
Культура Документы
Contents
Data Migration Procedure .............................................................................................................. 1
Contents ................................................................................................................................... 1
Summary .................................................................................................................................. 2
I. Define Data Migration strategy ............................................................................................... 3
Tasks to be performed during this phase................................................................................. 3
II. Extract (download/collect) data ............................................................................................. 7
Tasks to be performed during this phase................................................................................. 7
III. Process data ....................................................................................................................... 9
Tasks to be performed during this phase................................................................................. 9
IV. Transfer (upload) data ....................................................................................................... 11
Tasks to be performed during this phase............................................................................... 11
V. Reconcile data .................................................................................................................... 13
Tasks to be performed during this phase............................................................................... 13
Rulmenti roles and responsibilities................................................................................................ 14
Appendices ................................................................................................................................. 15
Appendix 1 – Data Mapping Form [DMF] template .................................................................... 15
Appendix 2 – Transfer methods identified for RULMENTI data objects ........................................ 16
Appendix 3 – Data Migration Log [DML] Template ..................................................................... 17
Appendix 4 – Data Object Dependencies in SAP ........................................................................ 18
Appendix 5 – File Naming Convention Proposal ......................................................................... 19
Page 1 of 19
Summary
Data Migration [DM] is the relocation of data from existing (legacy) systems into a new solution. It is
important to note that DM projects do not happen independently. Rather, they are spawned from
other development efforts such as the implementation of a new ERP and are performed as part of the
implementation of the new solution.
The DM is an important aspect of any new solution implementation; however it is also often ignored
till the very last moment. It is quite common to review a project plan for a new solution and discover
that DM is merely listed as a single task item, if at all. In hindsight, in most cases a separate project
entirely devoted to the DM is required to ensure overall implementation success.
Extract Transfer
Define DM Process Reconcile
(download/ (upload)
strategy data data
collect) ... data
Page 2 of 19
I. Define Data Migration strategy
During the DM strategy definition phase, the question ‘What we are trying to migrate?’ must be
answered keeping in mind that:
• Only the required data is migrated.
• The most appropriate method of getting the required data into SAP is achieved.
• All data to be migrated is migrated accurately.
• The integrity of the migrated data is maintained.
• Disruption is minimized during transition.
While analyzing legacy data processing systems, we should consider the different types of data to
be migrated:
• Reference (master) data;
• Historical (transaction) data;
• Data in non-core systems;
• Data available in other formats (electronic, paper based, etc)
• Data held external to the organization.
Also, if there is data that is not going to be migrated and will no longer be available following
implementation then such exceptions need to be included in DMC along with reason of not
including them in DM.
1
Data object: A group of data elements which together constitute meaningful information, such as a
customer master record.
2
Data element: A single value in a data object which only makes sense when taken together with
other elements in the set, such as the city of the customer’s address.
Page 3 of 19
2. Examine (at high level) the actual data we plan to migrate
At this point in the project, we might have no idea if the data is even of enough quality to
consider migrating. In order to get a better sense, it is helpful to obtain reports that can provide
row counts, column counts and other statistics pertaining to source data. This kind of information
can give us a rough idea of just how much data there is to migrate. We must then update DMC
with this information.
A popular misconception about data mapping is that it can be performed against logical data
models. Unfortunately logical data models represent attributes that arise through relationships
without defining the attributes in the child entity. This essentially means that you cannot map any
of the connections between data structures while you are working with the logical design. This is
why, it is necessary to perform data mapping to the actual physical data model (list of required
and optional SAP fields). With the physical data structures in place, we can begin the mapping
process. Mapping is generally conducted by a team of at least three to four people (data owner,
functional consultant, technical consultant and optionally the process owner) per business area.
Data Mapping Forms [DMFs] (see Appendix 1) should be prepared for each data object during
this task. These forms should demonstrate clearly which fields in SAP will be filled using which of
the legacy data sources. Once draft DMFs are available, better decisions concerning how data can
be extracted from legacy systems can also be taken. It should be noted that we are not trying to
define all data processing rules at this step, but rather assess the quantity and quality of data we
can obtain from legacy systems and more importantly, identify as quickly as possible missing
(unavailable) information. As DMFs are prepared, DMC also needs to be updated concerning their
status.
As part of the SAP implementation methodology, for each SAP object to be migrated in SAP there
is a formal data migration specification. The specification describes the format of the files as
accepted by the SAP migration tools. The mapping represents the definition of the conversion
rules between legacy data and the migration specification formats.
Page 4 of 19
The mapping is done:
• Automatically, when the rule of conversion between the legacy attribute and the SAP
specification attribute can be logically defined and dealt with in a conversion program. Such
conversion applies mainly for the long term assets, e.g. for determining the cost center,
company code, other asset attributes. The prerequisite for using this method is that legacy
data exists in electronic format and the conversion rule is logically consistent and can be
expressed in a conversion program.
• Manually, when the legacy data does not exist in electronic form, or when the conversion rule
can not be defined in enough detail to be put in a conversion program. This method applies
for example when determining the vendor’s bank accounts (no electronic legacy data exists,
just legacy invoices and contracts on paper documents). Also this would apply to certain
accounts that can not be split easily between Distribution, Supply and Support (e.g.
receivables from employees for travel advances).
Sometimes a combination of these methods can turn out to be the optimal strategy (For a list of
transfer methods / applications for RULMENTI data objects see Appendix 2).
DMC and relevant DMFs should be updated as transfer method decisions are taken and
responsibilities are assigned to functional and technical consultants (Depending on identified
requirements, functional consultants and data owners might need to prepare technical
specification documents for technical consultants to write custom programs).
It is imperative that for each data element a decision is taken regarding the approach and a rule
is set as early as possible and these decisions are reflected in the relevant DMFs. These decisions
must then be reviewed to ensure that data integrity will be maintained. Only after this review can
the DMFs be considered complete and signed-off by all related parties (RULMENTI data owners,
migration team members and project managers).
Page 5 of 19
7. Convert DMC into a project plan
The final step in defining the DM strategy phase is to convert DMC into an executable plan. This
does not mean discarding DMC; it should actually remain as a living document and the most
important work product of DM efforts. But we also need to identify other DM deliverables and the
tasks to produce them, estimate the tasks and assign resources to each; always with the time
and dependency constraints in mind (For example, vendor open items can not be loaded unless
vendors have been created in the system first). Those dependencies are reflected in the SAP
deliverable called “cutover plan”. Since the planning of the DM is usually tight, if one object fails
during the process, it will delay all dependent objects in the cutover plan that depend on its
existence in the SAP system. So, if in our example vendor loading fails, the vendor open items will
have to be postponed until the preceding object (vendors) exists free of errors in SAP.
The cutover plan also shows the moments when legacy transactions stop and SAP corresponding
transactions start. The time gap between the shutdown of legacy and the start of operations in
SAP is called “cutover”. It is desirable that the cutover period is as short as possible. Different
legacy applications will shut down at different times (e.g. materials transactions will be first shut
down, later other objects and so on). The planning of legacy shutdown dates and SAP start-up
dates can also be found in the cutover plan.
Our cutover plan should make it easy for everybody to clearly understand what needs to be done,
in which order and when. DM is an iterative process – it does not happen in a single sitting –
therefore a number of practice runs should be planned to be performed before actual DM takes
place. It is essential that the overall management of data migration observes the sequence of
operations, dependencies and the shutdown/startup dates. This ensures the consistency and
orderly management of the data transfer process.
The plan should also allow us to track additional considerations such as business continuity
impacts3, risks4 and purchases5 from one central location. It is also a good idea to integrate the
DM project plan with the overall implementation project plan.
Progress monitoring is essential for meeting the DM deadlines and enables the early issue
identification and proactive management of risks. Progress should be monitored for every data
object for all steps of the data migration process. Progress monitoring should also include
qualitative checking and assessment to ensure that the data quality is not compromised. Typical
assessment criteria are:
• Relevance
• Completeness
• Consistency
• Duplicity
• Accuracy
• Compliance
3
Assess how the data migration schedule will impact on current business activities, and determine
ways to minimize this impact.
4
Conduct a risk assessment for DM and document the risks, identifying the strategy for managing
these risks.
5
Identify any purchases required for DM and list these with details of estimated costs. Integrate
these costs into the project budget as well.
Page 6 of 19
II. Extract (download/collect) data
During the data extraction phase, the required data must be obtained in an accurate and integral
manner, without disrupting the daily work routine. As early as possible in this phase, someone should
be appointed to port the legacy data from its current environment into a staging area6.
Then one of the most important work products for DM should be prepared / produced for each
data object: Data Migration Logs [DMLs] (See Appendix 3). The DMLs contain vital information
such as exact number of records, date and time of extraction, responsible person, etc. The DML
contents should be agreed by all related parties before data extraction applications are developed
(see step 3) since the DMLs should be prepared – as much as possible – automatically by these
applications.
In case there are any changes (either to extraction assumptions or to processing rules), DMC and
relevant DMFs should be revised and re-signed by all parties before being communicated to the
rest of the implementation team. In case of major changes, the project/cutover plan should also
be updated (see step 8).
Such applications will allow fast and reliable extraction of the final data to be transferred, and
furthermore they can be used across different locations for similar / same data sources, reducing
DM effort significantly.
Through workshops, legacy system technical experts should be explained in detail (using actual
DMFs and DML design templates) what they are required to extract and how to export it to the
staging area where the data will be further processed. DMC and relevant DMFs then should be
updated.
6
A staging area is merely a database account that contains tables that essentially replicate the data
structures of the legacy system.
Page 7 of 19
4. Extract the data
At this point, we should have:
• More precise estimates of data volumes in our DMC;
• Up-to-date DMFs containing the final mapping and the processing rules;
• Finalized templates for DMLs;
• A designated target staging area; and
• Tested data extraction applications.
Data extraction can be done in a wave approach, meaning that as soon as certain extraction
applications or resources are ready; they can be used to transfer data to the staging area – of
course always keeping the dependencies in mind.
In case application development is not feasible (too few records to be transferred, too
complicated extraction rules requiring decision making, etc.) necessary resources should be
informed, trained and allocated to carry out data extraction manually from legacy systems into
the staging area. It is very important for these resources to maintain the DMLs properly, which
should also be part of the training. DMC and relevant DMFs should be updated and such changes
to resource planning should also be updated in the project plan (see step 8).
Such errors should be dealt with as soon as possible and noted in the DMLs according to the
predefined format. In case there are changes to the data objects or elements, DMC and relevant
DMFs should be updated.
6. Validate methodology
There are multiple iterations of the data extraction, conversion and loading procedure. SAP
implementation methodology recommends that the extraction/conversion/data loading tools are
initially validated on a small set of representative data. This has been performed for all of the
financials SAP objects to be migrated.
A second iteration is done with the figures of a closed month of the legacy. This iteration was
performed for the most complicate SAP objects in the financials area (long term assets, vendors,
customers). For G/L accounts balances and open items the test is not yet completed.
The third iteration will be done in the SAP quality assurance system with the real migration data
as on December 31st, 2006. The results of this iteration will be formally approved by the data
owners before the load of those objects in the productive system.
It is essential to validate all extraction methods (applications) and document them both on DMC
and the DMFs for traceability of the DM efforts.
7. Data back-up
Starting from data extraction phase, necessary measures should be implemented to back-up data
(especially in the staging area) regularly. Data back-up provides both history (and the
traceability) and disaster recovery (in case of data loss). DMLs should also be updated with back-
up information such as date and volume.
Page 8 of 19
III. Process data
During this phase, the extracted data in the staging area is processed and prepared till it is error free
and ready to be uploaded into SAP. This is an iterative process, with many test runs and real runs on
the data.
In case there are any changes (either to the assumptions, processing rules or transfer methods),
DMC and relevant DMFs should be revised and re-signed by all parties before being
communicated to the rest of the implementation team. In case of major changes, the
project/cutover plan should also be updated (see step 8).
Such applications will allow fast and reliable processing of the final data to be transferred, and
furthermore they can be used across different locations for similar / same data sources, reducing
DM effort significantly. Applications are usually good for cleansing, amalgamating/splitting and
unbundling where a rule can be defined upfront and data can be treated en masse by the
application later on following the rule.
Through workshops, technical experts should be explained in detail (using actual DMFs and DML
design templates) what they are required to process and how to report it in the DMLs. DMC and
relevant DMFs then should be updated.
In case application development is not feasible (too few records to be processed, or too
complicated rules requiring decision making, etc.) necessary resources should be informed,
trained and allocated to carry out data processing manually in the staging area. It is very
important for these resources to maintain the DMLs properly, which should also be part of the
training. DMC and relevant DMFs should be updated and such changes to resource planning
should also be updated in the project plan (see step 8).
Manual processing is usually good for filtering/matching and enriching where rules need to be
defined depending on the individual situation.
Page 9 of 19
5. Analyze and correct errors
As data is processed in the staging area, the DMLs should be revised for possible errors or
missing information (in terms individual data elements or entire records).
Such errors should be dealt with as soon as possible and noted in the DMLs according to the
predefined format. In case there are changes to the data objects or elements, DMC and relevant
DMFs should be updated.
6. Validate methodology
There are multiple iterations of the data extraction, conversion and loading procedure. SAP
implementation methodology recommends that the extraction/conversion/data loading tools are
initially validated on a small set of representative data. This has been performed for all of the
financials SAP objects to be migrated.
A second iteration is done with the figures of a closed month of the legacy. This iteration was
performed for the most complicate SAP objects in the financials area (long term assets, vendors,
customers). For G/L accounts balances and open items the test is not yet completed.
The third iteration will be done in the SAP quality assurance system with the real migration data
as on December 31st, 2006. The results of this iteration will be formally approved by the data
owners before the load of those objects in the productive system.
It is essential to validate all processing methods (applications) and document them both on DMC
and the DMFs for traceability of the DM efforts.
7. Data back-up
As data is being processed, necessary measures should be implemented to back-up data in the
staging area regularly. Data back-up now provides not only history (and the traceability) and
disaster recovery (in case of data loss) but also room for error correction (in case a faulty process
is carried out, the data can be “rolled back” to the previous state using the back-ups). DMLs
should also be updated with back-up information such as date and volume.
Page 10 of 19
IV. Transfer (upload) data
During this phase, the processed data in the staging area is loaded in SAP system in a controlled
manner (Development – Quality – Production). There should be no more modifications made to the
data itself during this phase.
In case there are any changes (either to the assumptions or transfer methods), DMC and relevant
DMFs should be revised and re-signed by all parties before being communicated to the rest of the
implementation team. In case of major changes, the project/cutover plan should also be updated
(see step 8).
Such tools / applications will allow fast and reliable transfer of the final data to SAP, reducing DM
effort significantly (For a list of transfer methods / applications for RULMENTI data objects see
Appendix 2).
DMC and relevant DMFs should be updated if transfer method decisions are changed and new
responsibilities are assigned to functional and technical consultants. In case of major changes,
the project/cutover plan should also be updated (see step 8).
In case using standard SAP tools or custom developed applications is not feasible necessary
resources should be informed, trained and allocated to carry out data transfer manually in SAP. It
is very important for these resources to maintain the DMLs properly, which should also be part of
the training. DMC and relevant DMFs should be updated and such changes to resource planning
should also be updated in the project plan (see step 8).
Manual transfer is usually done as part of customizing for very specific data objects.
Page 11 of 19
Such errors should be dealt with as soon as possible and noted in the DMLs according to the
predefined format. In case there are changes to the data objects or elements, DMC and relevant
DMFs should be updated.
The objective is to reach the status where data transferred to Quality Assurance System [QAS]
can be formally approved by the data owner of each data object for being transferred to the
Productive System – the final run in QAS should be absolutely error free.
6. Validate methodology
There are multiple iterations of the data extraction, conversion and loading procedure. SAP
implementation methodology recommends that the extraction/conversion/data loading tools are
initially validated on a small set of representative data. This has been performed for all of the
financials SAP objects to be migrated.
A second iteration is done with the figures of a closed month of the legacy. This iteration was
performed for the most complicate SAP objects in the financials area (long term assets, vendors,
customers). For G/L accounts balances and open items the test is not yet completed.
The third iteration will be done in the SAP quality assurance system with the real migration data
as on December 31st, 2006. The results of this iteration will be formally approved by the data
owners before the load of those objects in the productive system.
It is essential to validate all transfer methods (tools / applications) and document them both on
DMC and the DMFs for traceability of the DM efforts. After final validation all objects, tools and
applications should be frozen (protected against any changes).
7. Data back-up
As data is being transferred, necessary measures should be implemented to back-up data in the
staging area and SAP regularly. Data back-up now provides:
• History (and the traceability);
• Disaster recovery (in case of data loss); and
• Error correction (in case a faulty process is carried out, the data can be “rolled back” to the
previous state using the back-ups).
DMLs should also be updated with back-up information such as date and volume.
Page 12 of 19
V. Reconcile data
Although reconciliation takes place in a smaller scale in all of the previous phases (especially in
analysis and correction of errors), during this phase, by reconciliation we imply the final validation by
the business side on data transferred to SAP. The closing and approval of the DM should be
documented in a formal manner.
3. Data back-up
Once SAP has been set to “Productive”; regular back-ups should be made of all data (including
migrated data) according to determined procedures by appointed RULMENTI responsible. Data
back-up from this point on will provide:
• History; and
• Disaster recovery.
Page 13 of 19
Rulmenti roles and responsibilities
RULMENTI Project Manager is responsible for managing the co-ordination of all the partners and
working groups engaged in the data migration project by:
• Developing and maintaining a detailed project plan;
• Effectively resourcing the project;
• Managing their project team personnel on a day-to-day basis;
• Managing project deliverables in line with the project plan;
• Resolving cross-functional issues at project level;
• Managing project scope and change control and escalating issues where necessary;
• Monitoring project progress and performance;
• Reporting progress to the ABS on a regular basis and at least weekly;
• Providing status reports to other related parties;
• Liaising with, and updating progress to, project steering committee;
• Work closely with related parties and consultants to enable them to gather, analyze and “map”
the necessary legacy data; and
• Support in audit documentation preparation (if necessary).
Page 14 of 19
Appendices
Technical name
Data element 1
Characteristics
Characteristics
(if applicable)
No processing required
number of digits)
Brief explanation
(alpha/numeric,
Technical name
Data element 2
Characteristics
Characteristics
(if applicable)
No processing required
number of digits)
Brief explanation
(alpha/numeric,
Technical name
Data element 3
Characteristics
Characteristics
(if applicable)
Page 15 of 19
Appendix 2 – Transfer methods identified for RULMENTI data objects
The following SAP objects and transfer methods have been defined for the transfer of the migration
specification files to SAP:
Page 16 of 19
Appendix 3 – Data Migration Log [DML] Template
Reconciled by Reconciled on
Reconciled by Reconciled on
Reconciled by Reconciled on
Page 17 of 19
Appendix 4 – Data Object Dependencies in SAP
SAMPLE
Page 18 of 19
Appendix 5 – File Naming Convention Proposal
01-project management
01-project plan
02-status reports
03-work products
02-deliverables
00-templates
01-DMO data migration object 1 (i.e. COA: chart of accounts)
~
29-data migration object 29
99-other
DMO_template_v9
YYYYMMDD_DMO_SUC_v9
YYYYMMDD_report/log_name_v9
Recommended naming convention for work products/other files (meeting minutes, memos, etc)
YYYYMMDD_file_name_v9
Where:
YYYY is year (always four digits), MM is month (always two digits) and DD is day (always two digits)
of preparation of the file;
V9 is the version.
Page 19 of 19