No REST for us: New API Architectures

 

BACKGROUND

Speaking straightly, REST architectures are simply not enough for the challenges to meet by new APIs in the market. These architectures were designed in 2000 by Roy Thomas Fielding as an amazing and intelligent response to Service Oriented Architectures.



They (you'll remember) were heavy, slow, difficult to maintain and not flexible. Mainly based on SOAP Web Services the inclusion of more and more heavier and expensive (in terms of resources and money) software like the ESB's family made REST architectures more interesting, flexible and advanced.

REST architecture is what the world wide web is based on right now but this is changing rapidly. It is as relevant to that context now, as it was when HTTP was designed. Programs or other components could now easily talk to each other efficiently through firewalls over HTTP; just like SOAP but much more efficiently. More so the versioning in WSDL, or the lack of it, and the problems it causes when modifying, or slightly extending a published interface, all but vanished. Though some old APIs still use SOAP Web Services based, most of new developments and especially software systems became internally composed of REST based services.

However, the  HTTP protocol constraint is already a handicap. We need to do many other new amazing things with our APIs and the implementation of HTTP bridges for everything is not an option anymore! This post is just a quick overview on the current complexity that modern APIs must handle.

 

CHALLENGES

We'll have a fast overview about the main challenges and new factors that we should consider for the construction of any modern API.

Multiple Types of Devices

Modern APIs has multiple types of devices as consumers. From the device point of view:

• IoT devices


• Smartphones


• Mobile devices (tablets, 2 in 1)


• Laptop and Desktop computers.


• ATMs and similar Automatic Devices

Multiple Types of Consumers

• Mobile applications


• Web Browsers


• Programmatic consumers


• Message producers/consumers

Non Blocking APIs!

During last 5 years (at least) the software industry has tried to figure out how to make non-blocking HTTP based APIs with a reasonable success. However, it implies a change in the programming paradigm, software architecture and practices that absolutely has not been extended to the most of the software industry that keeps in most of cases synchronous blocking not scalable APIs.





 

Multiple Protocols

HTTP, the old good protocol that everybody knows about.

HTTP2, the new version to replace v1.1 integrating multiplexed streams, server push, headers compression and binary formats.

WS. For example, a Websocket client to view messages pushed from the consumer device to the browser or publish messages from the browser to the device. This is ideal for testing and debugging since the consumer can display live information from a device or sensor in real time.

MQTT (Message Queue Telemetry Transport) is a very lightweight messaging protocol, created with embedded systems, sensors and mobile applications in mind.MQTT has very strong delivery guarantees. There are three modes; fire-and-forget, at-least-one and exactly-once. A multi-protocol API should support them all. Besides, Bridges are essentially a way for one MQTT broker to connect to another MQTT broker. This is a very useful feature and enables the API to setup High Availability features for MQTT.

AMQP (Advanced Message Queuing Protocol). Based on the same principles of messaging but applied to API consumers (not Message Broker consumers) the idea is to provide broking services throughout API endpoints. Again, incompatible with REST.



gRPC. Protobuffer was the technology Google was using for long, along with gRPC , in its software ecosystem — one of the largest micro service ecosystem in the world. Google open sourced gRPC in 2015. Though still few companies will operate at this scale, this is a technology that is very relevant in most enterprise software with globally distributed teams, working over different time-zones, languages and cultures.

gRPC is part of the Cloud Native Computing Foundation. It has a strong focus on HTTP2 and mobile/IoT first and supports more than 10 programming languages.



Multiple Data Formats

We need to handle many data formats from the different types of protocols, consumers and functions in the API:

• XML


• JSON


• Text


• Binary


• Big Data formats


• and more..

Multiple API Roles

• Resource Set


• Publisher


• Consumer Message Broker


• Device Message Broker

The Influence of Technologies

Containers and Container Orchestration. The Linux containers were a disruptive innovation in IT provision and the introduction and evolution to Container Orchestration (to provide out of the box Discovery Service, Network, High Availability, etc) has been even more disruptive, making immediately obsolete the Virtual Machine based approaches. For example, in terms of software development frameworks the Spring Cloud approach was totally unnecessary when compared to services like DCOS and Kubernetes. Container Orchestration makes possible to stand up new components along the same IT, providing High Availability out of the box and therefore making more difficult basic approaches to server push strategies. We need new Container Orchestration compatible strategies: streaming.

Streaming. Many use cases require streaming. Keeping in sync data structures, flow of files, messaging, MQTT publishing/subscription, etc. Again, REST principles are not covering streaming at all.

Polygot Programming
The mantra of microservices requires polyglotism. Besides, gRPC and Protobuffer are available in popular languages like C++, Java, NodeJS (Typescript), Go among others. This gives tremendous flexibility in building a system. In practical terms, composing a software system with the flexibility to use the right programming language is a pretty good advantage.

IoT. By using MQTT we use messaging and to act as a MQTT broker for usage and telemetries the REST HTTP only APIs do not fit.

Messaging. Request/Response based communications do not fit in many cases in modern APIs. Message-based consumers send messages asynchronously to the API that acts as the message broker. In fact, a total decoupling of consumer/API happens in these cases obtaining full non-blocking behavior.

Big Data/AI. Well, HTTP only resource based APIs will not work pretty well with Big Data/AI operations unless you offer only a simple prediction or a small set of data. Again, these systems are consuming and using massive resources and amounts of data and the performance is not predictable. So, they usually require to APIs other approaches based on streaming but most frequently asynchronous messaging.

 

CONCLUSIONS

1. New technologies, functions and usages of modern APIs can not be limited to a single protocol, HTTP. Old HTTP APIs were designed considering only one class of consumer, the Web Browser.


2. REST architectural principles were the guidelines for HTTP APIs (although it is difficult to find RESTful APIs) but REST does not cover most of modern requirements,


3. Other protocols have to be supported by modern APIs to integrate heterogeneous consumers and heterogeneous types of information.


4. New technologies (IoT, HTTP2, gRPC, etc) allow the integration of heterogeneous consumers and types of information.


5. Modern APIs are smarter, more capable, fully scalable and versatile, offering much better products in the Market since they can be consumed by a variety of devices, protocols and consumer applications, avoiding the web browser-only constraint.