With APIs having evolved into critical business assets, assuring their quality is of paramount importance. In this article, we present a number of recommendations to help you put the right focus on your API testing to ensure an optimal result. In summary:
-
- Consider API testing versus API product testing.
- Identify testing activities and the stakeholders that should be involved with any.
- Consider continuous API testing.
- Be aware of specific aspects that must addressed under API product testing.
- Consider continuous monitoring for APIs in production.
Looking back, it is easy to see what profound impact technology has had on business models. On the one hand, companies needed to adapt to new trends and challenges that sometimes materialized almost over-night. On the other, no one will dispute the opportunities for growth and innovation it has brought.
APIs have always been placed in the ‘opportunities’ category, but it has taken some years for their true potential to become apparent.
For quite some time now, APIs have been acknowledged as indispensable building-blocks for creating digital channels rapidly and successfully. They are enabling developers to create user experiences that are both more integrated and more connected, thus contributing to business success.
It is a significant step further, however, to acknowledge the true power of APIs in terms of revenue generation, which is recognizing the API itself as a money-making asset. With businesses increasingly shifting to digital channels as the principal means to reach and service their customers, they have not only ‘digitized’ their front-door; they are also able to receive direct feedback in terms of customer behavior, preferences, etc., which has value in and of its own. Businesses can now decide to capitalize on these data – and just as for their other products, they have the technical means to make them available.
These days, APIs are the undisputed mechanism by which data are shared. In cases where a business decides to provide access through direct digital means, it is the API that becomes the product on offering.
In other words, APIs then become products in their own right.
Consequences of APIs As Products
Once APIs get productized, they become subject to the same demands as they pertain to other products that the business delivers; essentially, you want to guarantee the same level of customer satisfaction as with your other products. Consequently, API products are likely to be subject to rigorous quality assurance.
A related aspect that can easily be overlooked is that positioning APIs as products will involve additional business stakeholders next to IT. Sales & marketing, product management and legal & compliance can be expected to take on distinct responsibilities. In other words, productizing APIs is not just a technical matter, it is also an organizational one (though addressing this in detail is beyond the scope of this article).
Essentially, the goal is to deliver a high-quality API product, that can be trusted to satisfy demands in terms of:
- Expected/desired functionality.
- Overall quality (e.g. stability, availability, quick resolution of any defects).
- Security: Adhere to security standards to prevent security breaches and protect sensitive data.
- Performance: Ensure the API performs under different conditions.
- Usability /UX: Easy to understand and effectively implement the API.
To make sure that API products are (and continue to be) adhering to the standards you may expect from them (and they may be formalized in contracts or SLAs), their quality needs to be assured continuously. In the remainder of this article, we’ll focus on the ways and means available for this purpose.
API Testing and API Product Testing
API testing is at the heart of API quality assurance. In general, API testing is no different from other software testing, focusing on aspects like listed above. Once considered from an API product perspective, however, some distinct aspects should also be considered.
First, it is important to understand the distinction between API testing and API product testing. When considering API testing, the intuitive approach is to focus on the technical aspects of the API, in particular around functionality, security, and performance.
However, we need to be aware of the fact that ensuring the quality of an API product goes beyond testing the API from just the technical perspective. Obviously, testing the technical aspects provides the unconditional foundation for API product testing. Yet, API product testing is driven more explicitly from a usability/UX perspective. In order to be successful, the product needs to be found and, once found, be sufficiently ‘attractive’.
API product testing can help to ensure that the API is easy to use and understand for developers and end-users. In turn, this will contribute to increased adoption and satisfaction with the product.
API product testing will typically involve different stakeholders, like API product managers and representatives from both marketing and sales. The focus is on different API aspects, different target audiences and different business objectives as compared to testing the technical aspects.
The interdependency of these two is best illustrated by looking at it from an API (product) lifecycle perspective. First, you need to ensure that you deliver a high-quality product; essentially, testing activities are executed on non-production ‘versions’ of the respective assets; initially, focusing on the technical aspects (by the technical team) and then, once API code gets ‘productized’, on usability/UX aspects (by the API product team).
Once the API product becomes available in its production environment, the focus again shifts to the technical team (operations), as it now becomes critical to monitor whether the API is delivering as expected (availability, performance, security, etc.).
Continuous Testing
High and consistent API quality nowadays is often achieved through continuous API testing. Testing is done in a highly automated manner, touching every stage of the API (product) lifecycle, from development to production. It is a key aspect of a CI/CD-practice, enabling developers to detect and resolve issues in an early stage.
Once APIs get productized, continuous testing should be extended to include API product testing, though this may be delegated to different groups within the business.
API Product Testing in Parallel
An alternative approach is to test API products distinctly from the underlying API code. APIs are often created from their functional design, captured in a description document like OAS or Swagger. This document can be used to generate a ‘mock service’ that temporarily replaces the actual API endpoint and responds with a mock response to any incoming request. Extensive functional testing against the mock endpoint is very efficient, in particular when the test cases can be captured for automation. Once the actual API endpoint becomes available, the tests simply need to be re-executed for final confirmation of expected quality.
End-to-End Mocking to Test API Security
Such testing efforts can be extended beyond mere functional aspects. Security policies associated with the API can also be validated using a mock approach. In this case, sample requests are created that are meant to elicit a ‘success’ or ‘failure’ response – effectively creating an end-to-end mock scenario. API security requirements can often be complex, and the number of cases to test them quickly expands. Generating these cases to touch every aspect covered by the policies is immensely beneficial (and can hardly be replaced by a manual testing effort).
What’s more, such test cases (assuming they have been automated) can also be applied to production endpoints (synthetic monitoring) to ensure that the API continues to respond to both valid and invalid requests in the appropriate manner. This often allows for issue detection before an actual customer does. Automated monitoring is particularly applicable to API upgrade scenarios (is all still working as expected?). Last but not least, it can be extended to cover testing under load (validate behavior under anticipated or non-anticipated traffic volume spikes).
Mock services can also temporarily replace endpoints that are part of an orchestrated back-end, as they may not always or not yet be available for testing purposes.
Obviously, these approaches can also be applied to the API product. Here, however, the focus should lie on aspects like ‘discoverability’ (can the API product easily be found?), description (explaining the value of the API), documentation (technical interface details, supported use cases, security constraints that the developer has to be aware of) and ease of implementation.
In summary, the tasks involved with comprehensive API product testing are numerous and can only be expected to increase over time. The benefits of executing them well, however, cannot be denied, as they help to ensure the quality of your API products. With APIs evolving into revenue-generating assets, API quality becomes a critical factor. A sound API (product) testing strategy should take both technical and product aspects into account and ensure continuous testing across the entire API (product) lifecycle.
API quality is a critical factor in determining whether an API can be successfully monetized. Poor quality APIs can lead to a range of issues, including reduced adoption rates, increased development costs, and lower revenue streams.
