For any web site / web application big or small, data is one important thing. The amount of data you may have can differ but we know you can never underestimate the value of those. Also for complex systems, like e-commerce, bookings, ticketing to get going with a newly rolled out system you need to have all your old content migrated in to the new systems and they have to be ready to use. This involves a lot of planning, thinking, and testing. For instance an e-commerce system you will always want to have the old product data, and the orders to be in the system. No matter how old they are sometimes, you need to have these in your reports etc.
When building websites this is one of the things we offer to our clients. What we do first is to analyze the structure of the databases. And then we write migrator scripts to validate, convert, migrate data from the old database to the newly laid out structure. Also we make logs for all the migrator scripts we do, so we can go back and see whether everything was smooth on the migration. Its not just the data you will have in the databases which needs to be migrators, PDFs, images etc needs to be counted in. And sometimes you may want to convert assets like these, we once had to write a migrator where it gets all the images from a server and to put them on Amazon S3 cloud. Also if the information architecture changes with a new system, this is where we do convert contents to support the new architecture, we use XML files, JSON objects, plain text files to store rules and check all the records before we write them to the new software.
We have done many migrators, for e-commerce applications, website's which has tens of thousands of pages, bookings system etc. Most of the times we do database to database sorts of migrators, where we reads one database and write them to another. But some times we've had to write scrappers to scrape content from an old site and then convert it and write it again to the new structure.
If you are looking to build a new website, and worried about migrating data,talk to us, we can help.