Major Information Systems have been renovated in the last decade in order to develop digital applications on front end layer and on top of existing backends applications, using integration solutions like ESB. SOA has structured IS in 4 main Layers: Front End Presentation (ex: Web, Mobile), Front End Business Logic (ex: CRM), Application Integration (ex: ESB), Backend Business Services (ex: Contract Management).
In this model, as an Application Integration Layer, ESB manages 3 roles:
- Normalization: manages heterogeneous connectivity (SOAP, CTG) with Backend applications and exposes Backend Services as normalized and reusable Business Services.
- Adaptation: manages business logic and integration logic to meet applications needs, relying on Backend services and forming applicative services.
- Exposition: exposes applicative services to Front-End Application.
But this architecture model, enterprise centric, is not enough to meet the challenges of a new economy, customer and ecosystem centric, requiring more interaction, agility and openness.
- ESB has not been designed to support REST API interactions and lacks capabilities like traffic management, quota management, API authentication and authorization, traceability, API lifecycle management, API promotion, API analytics.
- ESB implies a central governance approach of Design and Build, impacting deadline of projects.
API Management, Service Mesh and Cloud are new building blocks that transform the way to design integration.
> API Management brings capabilities and agile approach: to fluidize and secure interactions both outside and inside enterprises, to build new digital products that are customer and ecosystem centric and to promote a catalog of existing APIs (SOAP, REST).
However, API Management is not responsible for implementing resources (endpoints) that are exposed on API Gateway as APIs (ex: implementation of operations of a resource /clients).
We are often divided between 2 approaches:
- Bottom-Up/Asset Valorization: expose existing APIs on API Gateway, with the risk of not being used because not RESTFul, too detailed, too far from needs, too complex.
- Top-Down/Usage First: define APIs from needs, with the risk of not being able to dock on existing services or APIs (ex: mapping of /clients/123/accounts/1 with 2 existing web services WS_Client, WS_Account)
The fact is that an adaptation is often required to map API RESTFul resource model with existing Backend Business Services, even if these last ones are exposed as REST APIs. ESB can effectively support this application adaptation.
> APIzation of Backend application is a strong trend. Pressed by the need for openness, more and more backend applications expose their Business Services as REST APIs. Data Platforms expose their data as REST APIs, Saas solutions too. These APIs can be exposed on API Management solutions. But, since their data model is inherited from the underlying application data model, there is a risk of exposing heterogeneous and inconsistent APIs in terms of syntax and semantics.
That’s why APIs should be exposed with an enterprise canonical “pivot format”. ESB can adapt Backend APIs to this pivot format, but is there an alternative to ESB?
> A Microservice approach can be used to improve agility and modularity, especially by restructuring a monolithic application in several fine-grained services, that are functionally integrated, autonomous, composable, minimal, scalable and resilient. But renovating an existing monolithic backend in microservices is a huge and expensive task.
We can also use Microservice technology, as containers (ex: Kubernetes), and framework (ex : play, spring boot) to build a Business “Facade”. This facade can expose Backend Business Services as REST APIs, helping APIzation of Backend applications, for instance on a data platform or over monolithic applications. We could use this Backend Business Facade to implement APIs with an enterprise canonical format and expose these “Backend APIs” on API Management solutions.
Microservice technology can also be used to build integration services in order to implement APIs docking on Backend Business Services. The idea is to replace the ESB solution by “Integration Microservices” relying on a microservice architecture. We can use Apache Camel Framework, or Ballerina WSO2 Solution to implement integration Microservices. So, we can expose on API Gateway, APIs that are directly relying on Backend Services.
In this way, an API Product Owner using API Swaggers on an API gateway and Microservices containers, can deploy APIs in full autonomy without depending on a centralized team (especially ESB), becoming more agile.
By default, a microservice can be deployed on any server. However, containers have become the best way to run Microservices because they encapsulate a lightweight runtime, presenting a consistent software environment that can follow the Microservices from the developer’s desktop, to testing and to final production deployment. Additionally, containers can run on physical or virtual machines and are multi-cloud ready.
Of course, as soon as several applications get split into Microservices, the Information System has to cope with a lot of Microservices. This requires a specific resource and routing management.
> Service Mesh is the way to control how different services of an application can share data with one another. Unlike other systems used to manage such communications, a service mesh is a dedicated infrastructure layer built right into the applications.
A Service Mesh is oriented toward service-to-service communication (East-West) while API Management is focused on North-South interactions. A Service Mesh proposes to offload the communication logic from Microservices, by relying on a network of proxies (sidecars). A sidecar is a component that can be onboarded in a microservice in order to manage http communication layer in coordination with the Service Mesh.
A Service Mesh can then provide capabilities to route requests between micro-services with resilience (circuit breaking, retries and timeouts, fault injection, load balancing, failover), and observability functions to locate dysfunctions (metrics, monitoring, distributed logging and tracing).
However, a Service Mesh is not a distributed ESB. A Service Mesh doesn’t implement the integration logic between Microservices (ex: connectivity, pub/sub, orchestration). Consequently, each microservice has to be designed in order to implement the integration logic, becoming a “Smart Endpoint”. It may also call messaging services playing the role of “Dumb Pipes” for asynchronous (pub/sub) use cases.
Conclusion
Companies deploying API Management can see that the historical integration layer of their Information System may ask themselves how to answer future agility and scalability stakes. The APIzation of their backends leads to a lot of Microservices. The ESB on which they invested a lot in the past is not agile enough. They could consider using the mix of capabilities offered by a Service Mesh with integration framework and message bus to develop their Microservices. The issue in this context is to choose the best solution mix and deployment patterns according to one’s context, as well as the most efficient granularity and role for each service.
