A Look at Data Migration – Volume 1: Intro to Data Migration

This article was co-authored by Lynsie Grover and Anthony Imanlihen Jr.

Welcome to Data Migration

It is within human nature to look at the overall and overlook the minor details; and while accomplishing the bigger picture is always the end goal, small steps are what are used to obtain this. The Microsoft Dynamics 365 ERP environment offers a wide functionality; however, we often forget of the everyday tasks that it is ultimately comprised of. Business and users first thoughts when entering the system is not initially how the information became available, but they can use it. Inside data migration, those minor details which revolve around migration data elements scope, ownership, timeline, risk, data cleansing, data transformation, data reconciliation, data validation, and environment settings is part of the detail of what it takes to get us to the initial implementation, upgrade or consolidation of any ERP system.

What is it All About?

Data migration is the movement of data from one system to another. This is not to be confused with data conversion or data integration which is the transferring of data from one format to another, and consolidation of data to be viewed in a single capacity or user interface, respectively. The main aspects of data migration include extracting, transferring and finally loading data from one place to another. The activity creates an opportunity to cleanse, transform the system of old, incorrect and unnecessary information, allowing for a fresh start. It is a critical component of any ERP system change; however, with focus being on the end solution and often learning of a new system, the true complexity is often not considered prior to commencing the initial implementation, upgrade or consolidation within a system.

A Key Factor

At the end of the day, business users are responsible for deciding how the system will look like post implementation. Every organization will have different needs and strategies for managing data depending on industry and in which way the environment is intended to be utilized. When considering data migration, it is important to understand that whatever is eventually loaded will ultimately be what the system is made up of. The information will be available in operation and set the stage for future transactions. This activity is what is also used to validate the system functions as expected to meet user needs through various stages of testing. Data migration must align the project phases throughout the entirety of the implementation.

Down time must be considered in the planning process to prepare for the roll up of new environments. This involves the cleansing and loading of data, as well as time to investigate and correct any issues there may be with files. More time must be allocated for issue resolution once the data is made available in an environment and users are able to run transaction tests. Making mistakes is the best way to learn as data migration is an iterative process. During the project implementation multiple cycles of data migration will be performed to fine tune the process and address gaps and issues identified in the early migration cycles. It’s important to keep adequate documentation of all the minor tasks involved such as the data elements scope, data cleansing details, data transformation, data validation, data reconciliation approach and tools, order of data loads and their dependencies, common errors found within data extracts, etc. Properly timing when information will arrive is important in providing the proper information for importing. Incorrect timing can result in failed imports and exports. Issues deriving from poor planning, lack of documentation, un-validated data and considering information needs for all stakeholders involved are common and can be easily mitigated. Future editions will discuss the risks within data migration in more depth.

Getting Ready For Change

Change is the only constant in our world, and it is inevitable – things are no different when stepping into the world of Microsoft Dynamics. Changes which aid with data migration involve cleansing data as well as having a fresh and configured environment. The upgrade, consolidation, or implementation of an environment presents an opportunity to refresh system information which may no longer be valid. Data must be formatted, and any outdated or duplicated records should be removed to prepare for migration activities. Cross-functions must work closely together to ensure not only the right type of data is brought over, but a satisfactory amount of information is available to suit all business needs without going overboard.

When bringing over information for the first time, a pristine environment is strongly suggested. There should be no prior transactions or previous data. It is also extremely important to keep a back-up of the environment for when a refresh is required between phases of testing. Understanding environment needs as well as Microsoft’s point-in-time restore will be future topics for this series. In order to prepare data and facilitate that it is loaded properly, certain items must be considered. Master data must be loaded first in preparation for static. Configurations must be staged. Some may need to be temporarily changed to aid data loads. Some examples include number sequence manual entries, account structure activations, or dimension requirements. These changes seem obvious, yet they can be overlooked and change the direction or duration of a project. Look for more on this topic in detail with the A Look at Data Migration – Types of Load series.

What to Expect Next

While very few can say they are a master with data, with our following series no one will remain static. Stay tuned for next edition as we take our first step to dive deeper into data migration with A Look at Data Migration – Volume 2: Master vs. Static Data.