DX, API Design & Documentation

Polling vs web sockets vs other approaches?

43views

As part of co-operation with apidays conferences, and apidays Helsinki 2020, a joint online event was held in collaboration with Joint Research Centre of the European Commission on public and private sector co-design on API development.

After presentations by public sector organizations from around Europe, a panel discussion summarized and aimed to answer the questions coming from these public sector organizations.

Private sector seasoned API experts Alan Glickenhouse (IBM) and Marjukka Niinioja (Osaango, panel chairperson), as well as Lorenzino Vaccari (European Commission, JRC consultant) discussed API-related questions presented by the public sector speakers in part II as well as topics brought on by the audience.

Panel concentrated on questions related to lifecycle management, discoverability, API design and security.

Part 1 : How should you handle API change management?

Part 2 : What is the impact of distributed architecture to API lifecycle?

Part 3 : How to discover public and private-sector APIs? 

Part 4 : Can we have APIs with different data models in the same marketplace?

Polling vs web sockets vs other approaches?

Questions for the panel: 

  • What should be designed as part of a generic REST API profile to multiple communication patterns in a business sense (e.g., request/response, publish/subscribe, broadcast, collect)?
  •  AMQP and MQTT are often quoted as alternatives to REST. Could you share any experience you have with these protocols? 
  •  Is there a recommended/predefined set of metadata that could be included in a generic REST API profile in order to facilitate interoperability with third-party message or data exchange systems?

Glickenhouse: Historically we thought about APIs as REST APIs, maybe SOAP. Now we are seeing more need for asynchronous handling, like AMQP, MQTT, etc. There isn’t one size fits all technology. We need different technologies for different purposes. Right now events are the hot topic, but even they don’t solve all needs. We should think more about consumable assets and then choose the right way to access them.  

Niinioja: Some years ago Zapier introduced the “RESThooks” with a slogan “Let’s stop the polling madness”. There are definitely occasions where polling is bad and pushing is good. You still need to consider that you might miss some data with the “push” model and still have the need to poll, just in case. In large scale big data scenarios one might agree that MQTT is a very strong candidate, and might be faster to process for example by IoT devices, because of the smaller payload. Some of these technologies, like gRPC and MQTT rely a lot on trusted servers on both ends and are not therefore best candidates for public or partner APIs. These might still be good for X-Road or e-Delivery use cases we heard by previous speakers.

Vaccari: Smart cities, evolution of the networks like 5G etc. bring the need to think about common technologies and especially the asynchronous technologies because of the large amount of data produced by the devices. There is currently a large set of technologies to explore and their number is growing. Regarding specifically metadata the AsyncAPI specs, derived from the OAS specifications, are raising lots of interest. 

Niinioja: I think the conclusion we should come to today is that there is no one size fits all, but there are definitely some principles to use when choosing technologies. If you have a “normal” network, public internet and you are building a digital application and using the APIs to communicate from front-end to backend, then REST APIs, or GraphQL would be a good choice. If you need to transfer lots of data fast between known servers you might be better off with gRPC, but you need the protobuf definition files on both ends with the ability to use gRPC and you need to trust each party. And there isn’t any API management solution, yet, to really help with that. 

Glickenhouse: We see this as a wider integration need and we combine for example APIs, Kafka and files under the same integration solution. In the future this is going to evolve and in the future it’s going to cover a larger set of tools. There might also be a developer portal to cover all the different styles. 

Niinioja: Looking back to what we discussed earlier about hiding the technical implementation of under the API design is also important when considering what technologies to use and how. For example, many GraphQL engines help generate the API directly from the database structure, which is really bad for change management. Didn’t we learn the lessons already at the database design levels? It’s a security issue, but it also creates unnecessary dependencies and ties consumers to your database schema.

Part 6 : API Security – How to send signed/encrypted payloads?

Marjukka Niinioja

Marjukka Niinioja

Founding Partner at Osaango.com
Niinoja is a co-author of API Economy 101 book. She is also the creator of the free "Introduction to API Economy" course with Tampere University. Niinioja is the “Mother” of the lean, open and business-oriented APIOps Cycles method. Niinioja works as API business consultant, architect, and trainer for companies and public-sector. She has 10+ years of experience with API Economy from retail, energy, ICT, construction and traffic industries, among others. Her team at Osaango Academy work together with universities, public sector and companies. They create courses on how to use APIs, Platforms and Data in business to grow thriving ecosystems.

APIdays | Events | News | Intelligence

Attend APIdays conferences

The Worlds leading API Conferences
with 9 Conferences in 2019:

Singapore, Zurich, Helsinki, Amsterdam, San Francisco, Sydney, Barcelona, London, Paris.

Get the API Landscape

The essential 450 companies

Get the API Landscape
Industry Reports

Download our free reports

The State Of Api Documentation: 2017 Edition
  • State of API Documentation
  • The State of Banking APIs
  • GraphQL: all your queries answered
  • APIE Serverless Architecture