Last month, APIdays and Platformable released the Market Scan: API Serverless Architecture report. The report — sponsored by Restlet — is the first in a four-part series looking at the new tools that are enabling application development and automated workflow construction completely in cloud environments, without developers having to manage backend servers or application architecture.
We have been wanting to dig in to the emerging serverless trend since late last year, when we heard increasing discussions amongst our APIdays community about the advantages of the approach to create value for end customers, drive agility and reduce complexity. Recently, there has been a spate of articles on Medium discussing how startups are using serverless architecture to test business models and identify the best product market fit: Benoit Hediard from AgoraPulse and Sam Kroonenburg from A Cloud Guru have both discussed how they used serverless environments to quickly release products to the market and gain end user traction. An article by CK Oliver on The New Stackat the end of last year also pointed to how Daniel Ilkovich built the startup Dexter by linking together services from different APIs.
All three examples are startups, talking about how they are using AWS infrastructure like API Gateway, Lambda and DynamoDB to build serverless applications to test a market opportunity and grow a (paying) customer base.
But we wanted to step back and get a sense of who (beyond just startups) are building server-less APIs first (and how, and why) before we dig in to serverless application development.
We knew from our discussions with enterprise, government, IoT makers and startups on a growth path that many have an interest in serverless. Given the technical and operational trends we regularly discuss at APIdays events, we know that two of the key factors driving cloud and API environments today are the need for agility, and the desire to reduce complexity. Serverless is igniting the interest of many because it can respond to both of those factors. To get there, business must first understand what value they are opening up by creating an API, and then serverless can help them open and refine that value faster, and cheaper. In an economy that can scale — like APIs— learning faster than the market is the key to success, and serverless architecture can let API providers quickly enter the market, and iterate towards creating real value.
Enterprise, startups, government and non-profits must first understand what their end users and partners need. In a digital economy, businesses need to understand what data assets they own, and how those data assets can add value to their ecosystem partners and to their end customers. Services, business functionalities and business capabilities can also be opened up as a value asset in a similar way. This is the YCombinator adage: make something that people want. Often, business partners will come to an enterprise asking for a particular data or service to be made available to them digitally so that they can reduce and automate the time they need to transact with the enterprise. Perhaps internal or external colleagues repeatedly request the same set of information from within the business.
A list of the business’ offices or product catalog are easy ones to think about, but digging deeper into what value they can provide can highlight additional data endpoints that are useful to make available. Perhaps colleagues need to know which offices are the busiest for walk-in business, which offices provide particular customer services, which staff members are located at which office, or which offices have meeting rooms and a booking system. Perhaps customers want to know what products are in stock that can be shipped immediately, or what complementary products other customers buy when they buy a product. This is where there is real value in the data for the user. Understanding this can help prioritize what data to open up via API, and which data fields need to be included for that data to be of real value to those using it.
Jérôme Louvel, Chief Geek and Founder of Restlet, says he is seeing interest in Restlet’s API Spark product — which helps users build APIs in the cloud without having to manage backend resources — as coming from enterprise and governments who want “custom APIs, which are exposing the exact resources needed. This is a differentiator from more general backends,” he says. “It is resonating with a user market where developers want to consume APIs. They need to have them very quickly without worrying about the long lead times with IT and external suppliers.” Louvel says it is not really intended for API designers who many want to control every stage of the lifecycle process, but more for others in an enterprise who need to prototype an idea quickly. Louvel’s insight matches similar comments made by Jaime Ryan from CA Technologies’ Live API Creator, Aaron Allsbrook from ClearBlade’s Novi Platform, and Richard Mendis from JustAPIs.
The speed at which APIs can be released also allows a faster feedback loop to be put in place to measure the value that users are receiving from the API availability. As API serverless architecture products include some form of user analytics (see the dark purple area in our diagram above), API providers can begin to measure the usage and value the API is creating and can more quickly release more data endpoints that match what is needed by end users.
Bill Appleton, CEO of open source API serverless platform DreamFactory says APIs and backend complexity are “a big problem” that enterprise are looking to serverless to solve. He explains:
It used to be that all the attention was on the server. Now in the enterprise, there are two teams: client and backend, but a back-and-forth negotiation between the two can take months to play out and mobile projects can fail if they don’t get their act together.
But it is almost a bigger problem if they do get their act together: we have seen this again and again. In an enterprise, they start with one API, then they don’t want to rebuild that first API, so they build a second API. That development is outsourced, and over time, they dig this hole of complexity that can’t scale, that can’t port anywhere.
Vijay Pullur, CEO of WaveMaker, and Dmitry Sotnikov, VP of Cloud at WSO2 talked about how some businesses wanted a middle ground between the simplicity of a cloud-hosted API and the feature-richness of API management providers. These users may have a few APIs they need to build and manage, but don’t want the complexity of a full-service API management solution. Products that allow them to create and manage serverless APIs were seen as a potential middle ground that enabled them to use APIs to drive innovation, build application prototypes, and automate workflows.
Categorizing the Market
Simplify, our four-part research series that discusses how agility is being enabled and complexity diminished in serverless environments, starts with a look at the tools that can be used to create and manage APIs completely in the cloud.
To better map the market landscape, we have categorized tools and products using the following groupings:
- Products that create and manage APIs without needing a backend
- Tools that make a simple API that can Create/Read/Update/Delete data at its source
- Tools that funnel multiple data sources or APIs into a single, serverless API
- Tools and frameworks targeted at developers who want to build apps in a serverless environment.
API Serverless Products
- Our definition: API Serverless Architecture Providers offer end users a product where they can design and create an API, attach the API to a cloud-based data store, add business logic to how the API functions, publish the API, and manage API consumer usage. These products all have the functionalities shown in the dark purple row of the diagram shown at the start of this article. In cloud-speak, these products could be called Platform-as-a-Service for APIs.
- Users: Users need a fully functioning API that has the same level of strength as building an API with an internal development team, but with all the data and functionality to be connected residing in cloud servers and with some API management capabilities to monitor API usage included.
- Key distinctions: The products in this category are often no code environments or offer a visual interface for creation of the APIs (with some additional code scripting able to be added). Our report digs into what are the common, additional and specific features offered by the current 12 providers in the space (although another two products are expected to be released in the coming months). As Mark O’Neill from Gartner pointed out to us, while we have used the term API Serverless, perhaps it would have been more appropriate to call this Application Server-less, “because you can deploy an API in front of a data source without an application server (including that you don’t have to write code to deploy in the application server)”. Several of these products also do have the capability of creating server-less APIs that are drawing data and services from a legacy and server-based backend.
- Our definition: Products that can help users transform unstructured and structured datasets into simple APIs that can Create, Read, Update and Delete data, in order to ease third-party consumption and interaction with the underlying data source
- Users: Users need to be able to scrape web sites or online data sources and instantly make that data machine-readable and accessible by adding an API
- Key distinctions: Several of these products are not the ideal way to consume data in large volumes via API, but instead aim to make an API available as a demonstration of the potential of making use of the underlying dataset. For example, Socrata, OpenDataSoft and CKAN have automatic tooling that makes any open dataset on their platform consumable as an API. This is great for building visualizations or testing ideas, but for production use cases where a heavy end-user base is expected, the developer may need to go to the open data owner and request that the original dataset being available via an API directly. We will be doing a further scan of products that fit this category for the fourth part of our Simplify series, to make sure we catch new additions like civic tech leader Mark Headd’s recent open-source CSV-to-API creation tool .
- Our definition: Products that can help a user to consume an API through an abstraction layer that funnels data and services into a single API. In cloud-speak, these products could be called middleware for API integration.
- Users: Users need to be able to combine data coming from a variety of hardware sensors, SaaS tools, and databases, and have that available via a single API
- Key distinctions: Our original research gave an overview of example products in this category and we are now trying to catalog a more thorough listing. We are excited at the extended list of existing APIs that offer a funnel-like feature. For example, the Pitney Bowes API aggregates a range of postal and mapping services into their shipping/delivery and location services APIs respectively. So they are an API Funnel that draws in other APIs and data sources to make them available in the one API. SumAllis a social media aggregator service that also offers a partner API that aggregates all social media APIs and data sources into the one API so their end customers can use the one API to display realtime social media metrics. Both of these products were left off our initial list and have now been added. We are actively researching a more comprehensive list of API Funnels to share in the second and third part of our Simplify series.
Serverless API Creation Tools and Frameworks
- Our definition: Tooling and frameworks that help developers create APIs in the cloud and link APIs together to create new serverless applications. For now we have grouped a wide variety of products which a much larger degree of openness to them here, than is evident in the API Platform-as-a-Service type products listed under API Serverless Products. The main difference with our API Serverless products listed above is that these products often do not have a visual interface for building the APIs, require more coding, and need to be coupled with API management tools, whereas our API serverless products have all of those features baked in. This grouping can be further broken down into subsets, which we will revisit in parts two and three of our Simplify series.
- Users: Developers who do not want to be locked in to an server-less API product; who want to create their own APIs in their own preferred language frameworks; or who want to build within an existing larger cloud platform environment (like IBM, Microsoft Azure, Amazon Web Services and Google Cloud Platform).
- Key distinctions: This is a difficult category to define in a standard way. We were keen to explicitly mention Serverless, Amazon’s serverless framework but as Michael Hart pointed out to us over Twitter, technically we should mention AWS Lambda and Gateway as that is akin to Google Cloud Functions, and Amazon’s Serverless sits on top of that. While this is true, we didn’t want to not mention Serverless at all in the market landscape. Serverless API Creation Tools and Frameworks is also a fast growing space with new entrants IBM OpenWhisk, Webtask.io, IOpipe recently becoming available. Also, a number of serverless app development tools like Stamplay, Zapier and Built.io Flow should be listed here as they often allow the creation of serverless APIs to be added alongside custom code and API workflow integrations. They will be explored in more detail in parts two and three of our Simplify series.
Our categories do not always create an exact fit. Some tools can be considered as both API serverless products and as creation tools/frameworks. Restlet and JustAPIs easily fit into both categories.
However, this is part of the difficulty in mapping a fast-moving and growth-blossoming sector within the API and application-building industries. As we release each part of the Simplify series, we expect this landscape to further evolve and consolidate slightly. For now, we are using this research as an opportunity to answer key questions:
- Part One answers Why? and Who? Why is serverless growing as a trend, who are the users of API serverless products and why are they using a serverless approach?
- Part Two starts answering How? How are developers building serverless applications? How can you learn from their approach and implement in your use case?
- Part Three finishes answering How? How are business users automating workflows in a serverless environment? How can you optimize business processes by leveraging serverless?
- And Part Four asks When? When should you take up the serverless opportunity? When should you partner and participate in the serverless ecosystem? And when you should you move on to a server-based architecture and infrastructure environment?
Join us on this journey as APIdays explores Serverless.
Written by Mark Boyd and Mehdi Medjaoui.