Data migration

As technologies evolved, it is common that software companies discontinue legacy systems, discontinuing support. Organisations may also undergo a restructuring that may call for new software systems that better support their business processes. Either way, organisations may find themselves faced with the daunting task of migrating years and even decades of old data to a brand new system.

Think of it like moving house

Data migration can be likened to that of moving house. In parts of Europe and America, some homeowners would pick their entire house from its foundation, tow it behind a truck and setting it down in another spot.

While this might seems like the fastest and easiest way to move houses, there could be a higher possibility of damaged furniture during transportation. And unwanted items that should long be discarded, are brought along for the ride. 

An alternative to that would be, moving your belongings in batches and only bring with you the most essential items; the “need-to-have’s”.  This would ease the burden of also moving the “good-to-have’s” that may otherwise complicate the migration process.

Challenges Involved

Data Extraction

Data extraction can be complicated when the sets of data can originate from different parts of a system. For example, in an ERP system, they can come from the procurement, finance or master data. Further complications may arise if different entities are currently sharing the same server to store their data (e.g. An entity splitting into two due to restructuring).

Volume of Data

Business leaders have to be certain of the scope of data that are to be migrated right from the beginning. This would concern the overall volume of data and thereby determining the duration required for the migration project. Longer project durations would dictate the time needed for business process owners involvement for data validation. This would inherently take away the time they have for working on their everyday tasks. It is crucial to keep user involvement during project implementation to a minimum to not disrupt the day to day operation of an organization.

Data integrity & consistency

When shifting data from one system to a brand new one, they are often incompatible and is especially so if the systems are from different software providers (e.g. SAP and Oracle). Data are often required to be mapped in order to make them compatible with the new system. This mapping process must be done right as any inconsistency would render these data as unreliable and could be potentially more cumbersome to rectify the issues down the road.

Scope of Data

The scope of data to be migrated into the new system can go out of hand. Historical data does not get access all the time in the new system. we had clients before who looked at moving all their data into a new system and end up just moving the open items and balances as activities in data migration became overwhelming for them.

Project Duration

Activities such as extracting the data, data validation and data loading may require a substantial amount of time during a data migration project. The longer project duration would translate to a higher cost as only incur additional project-related expenses.

Key Takeaways

To reduce the cost and effort of the project implementation while maximizing the return of investment:

• Migrating only the “need-to-have” while archiving older data. This would help in reducing the duration of the project which would thereby reduce the project costs.
• Scoping of the project. Assign resources and form a reasonable project timeline to allocate tasks more efficiently and provide employees with a common direction and goal.
• Align users expectations. Ensure that stakeholders understand the outcome of the migration. These data are mostly accessed by business users and they would give their inputs on what needs to be migrated.
• Access the compatibility of the new system. Take advantage of the new functionality or extensions that the new system may provide in the future, such as connections to a CRM system.