Despite the popularity of REST as a buzzword, the API community has been struggling for years to raise the understanding of proper “RESTful” design. One reason for that is insufficient tooling which would facilitate implementations which satisfy the REST constraints as outlined by Roy T. Fielding.
I would like to propose a functional API testing tool which fills that gap by closely relying on the tenets of REST. By using this approach, API publishers will be able to avoid some of the common mistakes in REST API design.
What is REST?
While this has been covered many, many times before, for the purpose of this post I’d like to characterise REST by a few short bullet points:
- clients only know a single entry URL, either as documented by the server or bookmarked.
- clients are not hard-code against any other URI/patterns, HTTP methods, etc.
- servers include links and forms so that clients can issue further requests
- clients only perform requests advertised by the server in the resource representations in form of said links and forms
- clients and servers can support multiple resource formats (representations) and use content-negotiation to decide on which one to use
Software testing
Wikipedia describes software testing as a process, which involves the execution of a software component or system component to evaluate one or more properties of interest.
When speaking about unit tests, these properties of interest are mainly correctness of individual units of composition.
Integration tests ensure that smaller parts work well together, satisfying the functional and non-functional design requirements.
Finally, different kinds of high level or end-to-end tests ensure that the exercised system functions correctly as a whole, deployed to a live environment. These kinds of tests include manual tests, UI automation or A/B tests.
State of the art of testing REST APIs
It would seem that most popular API testing tools have a number of design choices which limit their applicability to REST principles.
- Focusing on specifics of HTTP messages
- Unit-test approach of individual resources
- Binding the tests to specific representation format
- Ignoring hypermedia controls
Here’s an example of a REST Assured test scenario:
First problem if relying on concrete URL and HTTP methods, which should be treated as an implementation detail of APIs. Yet most, if not all, API testing tools require the test scenarios to be coded against concrete resource identifiers (URLs), HTTP methods, etc.
Such a test will start failing when the URL changes or the API would be changed to use another HTTP method.
Second, the typical use case of testing tools is to interrogate “random” resources directly, disregarding their relation with other resources of the API (or other APIs!). The example above starts off by posting to the /greet resource “out of the blue”. In a real application, be it implemented as a machine-to-machine fashion or executed by a human, such a request would likely be one a sequence of requests, the first one always being a `GET` request. This is similar to how humans need to first open a web page to be able to fill in a submit a form. It is not possible the other way round.
Third, while the example above does show it possible to imperatively code around the representation to extract a link and follow it, that approach bears some problems. Such test requires that the API responds with JSON. And, more importantly, that it keeps responding with JSON in the future. If any at any time the implementation changes to favor XML, or YAML, or some other media type, the test will fail but for the wrong reason.
Finally, related to the previous paragraph, API tests do not treat hypermedia controls, that is links and forms, as first-class citizens. Apparently it is not easily possible to verify that a link exists or that a specific form, as identified by the API contract, is available for submission.
Hypermedia-focused testing
Dissatisfied with the functionalities offered by available API testing tools, I decided to build a new one which would close the gaps I outlined above. Here are the goals I have for the new tool:
- Use a simple text DSL so that test scenarios can be source-controlled
- Implement tests in a fashion which matches real interactions of the API’s client
- Never hard-code URLs and HTTP methods
- Include features to work with links and forms
- Be media-type agnostic (where possible)
Here’s an example of a slightly more complex test scenario declared in an early version of the testing DSL (Hydra flavour):
Having tests which closely resemble the “right way” of consuming REST APIs should also help improve the overall design, putting stress on what’s important for their longevity and evolvability.
If you’d like to learn more, be sure to head to https://testing.hypermedia.app/.
