Nitish Agarwal is the Principal Engineer with GoDaddy. In this article, he discusses the System Migration Lifecycle.
Business Lifecycle
Every company and business goes through a cycle where you have an introduction to the market. Then, the business goes through finding a product market fit. If they find it, they hit a hockey stick part of the curve, which is growing and growing rapidly with more competition in the picture. Over time, maturing, streamlining the processes, minimizing costs, and essentially making the business more efficient and profit-making.
As part of the cycle, though, companies do not want to go there, and that is the decline stage, where your competition outpaces you, or you fade away. So that’s where companies start looking for a lifecycle extension. And this is where they look at mergers or acquisitions.
In acquisitions, a smaller company is being acquired by a bigger company to give a larger company, where you are provided with more services from a single company. For example, Salesforce acquired Tableau.
In mergers, similar smaller companies may decide to merge to form a larger company.
What happens after acquisition?
In an acquisition, you can either consolidate bits and pieces, keeping the useful bits, or letting go of the pieces that are not useful. You can also do big bang consolidations, where you acquire everything. These are the fastest, but can be more painful or can make or break parts of the business. The third type is where you can run both businesses as parallel businesses or separate entities.
Let us look at the Big Bang Consolidation. You take teams from your parent company and the company you plan to merge with and tell them about the merger. So, you are taking two tech stacks and either merging them or letting one tech stack take over. This is where the consolidation process starts to happen. Right.
So, the first task is prioritization. You have to identify the core features of your company. Based on that, you identify the stacks that are required. You decide on what to keep and what to dump. Then, deep-dive into parallel stacks with large data sets, both historical and ongoing.
Issues with Big Bang Consolidation
The main issue with big bang consolidation is the data volume being migrated or transferred. This may not be conducive to existing APIs. The existing company might have a more transactional-based system, not lending itself to bulk data inputs or historical data imports in functions. There may be different naming conventions, etc. All these things have to be considered. Teams have to work on existing features as well as migration. It is like a moving train and people trying to get onto it.
There will be no single point of contact or system. This is because we have multiple microservices talking to each other. There will be multiple interfaces to study. If there is an API gateway, it may further complicate things. Authentication mechanisms may not support service-to-service interaction. So, a lot of steps may be repeated. This can lead to custom development.
Big Bang Consolidations without pain
Creating dedicated migration and consolidation teams.
These are responsible for all infrastructure or shim layers for API, which help in these migration or consolidation activities. This will also ensure a single point of contact for the team. This point or team can help identify the appropriate APIs, their features, etc.
The downside is that the team can be overwhelmed by requests for change and become a bottleneck.
Shared responsibility model for teams
Each team is responsible for their APIs and serves clients related to their APIs.
On the downside, multiple teams may lead to issues with coordination. Fresh consolidation will require rallying up those teams and starting to discuss consolidation. This can also lead to consistency issues.
The Solution
My experience says that dedicated teams are a better solution. Having a dedicated team with a diverse skill set, like a DevOps team. The proposed architecture model needs to be consistent and ensure standardization around connectors. The API Interfaces should be consistent.
Backchannel APIs
Backchannel APIs are internal APIs specifically designed for migration use cases that provide standards and connectors for a team to onboard new businesses onto existing companies. These APIs have to have a set purpose. These teams are small and can be easily overwhelmed. So, start with a core set of must-have features, and the consolidation will fail in their absence. You start making API extensions or standalone APIs to handle such features. Then, we standardize interfaces or connectors. Next comes the mapping of the APIs. This continues in a cyclic manner by versioning, deprecating, creating, etc.
System structuring
Based on your organization size, we can choose how the functionality for APIs is split. The function of our backchannel APIs starts to look like an API Gateway.
A single controller is an ideal scenario where you have a dedicated team developing an API facade for you for all the different business logic, services, teams, or whatever you have and then providing this for our business.
If every company has their own interface, we need to stitch them together. This requires a different team structure. So, the API façade has much more functionality to cater to migration use cases.
The third approach is more for a mature company use case. In this case, migration use cases have already been considered. It is not just a single lane of data. In those cases, the job becomes a lot easier. You have a gateway, which is routing requests to the right party.
To conclude, there is no silver bullet to all of this. Having backchannel APIs has helped me with more than ten migrations.
