Hypermedia-driven RESTful web APIs
Table of contents
Recently, I started working on a project, which involved the development of a RESTful API, so I decided to first do a research to better understand exactly what it means to be REST compliant (RESTful). It didn't take long to notice that many people out there were arguing about whether their APIs should be RESTful or not and I thought it had to be something either very important or very confusing, since a lot of people could not get to a definite agreement on whether a RESTful API really matters.
First I found a document, written by Dr. Roy Fielding, who introduced the REST to the world. He wrote a doctoral dissertation in which he explained in detail what REST is and why it is important, so I'll try to summarize it here.
So, what is REST? And what is it used for?
REST is an architectural style, used to create scalable, performant and maintainable distributed web services. In simple words, REST gives you a good platform to build a web service that will last for a long time. All these features (scalability, performance, user-perceived performance, maintainability,etc.) are something that Dr. Roy Fielding considered important for the evolution of the world-wide-web services. But that doesn't mean REST is a silver bullet or the one-and-only solution to any web service you need to develop.
It simply means that, for a given set of requirements (features, goals), Dr. Fielding has found and documented all the constraints that need to be satisfied in order to achieve these goals. Nothing more, nothing less. If your goals match with his, then you should also consider using REST to build your web services, but if your goals are different, REST will not help you much. It can even make things worse because each of those constraints introduces certain trade-offs, which might collide with some important features of your web service. Shortly, don't use REST for just any web service you want to build, because everybody is talking about REST. It's simply wrong.
REST is an architectural style, used to create scalable, performant and maintainable distributed web services. In simple words, REST gives you a good platform to build a web service that will last for a long time. All these features (scalability, performance, user-perceived performance, maintainability,etc.) are something that Dr. Roy Fielding considered important for the evolution of the world-wide-web services. But that doesn't mean REST is a silver bullet or the one-and-only solution to any web service you need to develop.
It simply means that, for a given set of requirements (features, goals), Dr. Fielding has found and documented all the constraints that need to be satisfied in order to achieve these goals. Nothing more, nothing less. If your goals match with his, then you should also consider using REST to build your web services, but if your goals are different, REST will not help you much. It can even make things worse because each of those constraints introduces certain trade-offs, which might collide with some important features of your web service. Shortly, don't use REST for just any web service you want to build, because everybody is talking about REST. It's simply wrong.
That being said, let's see what are all the goals, their constraints and trade-offs introduced in Dr. Fielding's REST architecture. The following table briefly summarizes that:
| Constraint | Goals | Trade-offs |
|---|---|---|
| Client-Server | Client portability and server scalability, through the separation of concerns (user interface vs data storage). This separation also allows the components to evolve independently. | No peer-to-peer communication (client to client directly). An imposed need for a server can be a single point of failure for the web service. This is mitigated with scalability goal. |
| Stateless | Visibility, reliability, and scalability (there is no stored context on the server which is used across multiple client requests). | Increase of the repetitive data (per-interaction overhead). |
| Cache (client-side) | Scalability and improved network efficiency (user-perceived performance caused by the reduced average latency of requests). | Decreased reliability (stale data within the cache). |
|
Uniform interface:
|
Implementations are decoupled from the services they provide, which encourages independent evolvability. | Degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application's needs. |
| Layered system | Reduced system complexity. | Reduced user-perceived performance due to the overhead and latency to the processing of data. |
| Code-On-Demand (optional) | Extends client functionality by providing downloadable and executable code in the form of applets or scripts (Java, Flash, Silverlight). This simplifies clients by reducing the number of features required to be pre-implemented. | Reduced visibility. |
The important constraint here is the one named "Uniform interface" and the most important of those 4 interface constraints is the one named "hypermedia as the engine of application state" (aka HATEOAS). This constraint is the main reason behind those widespread disagreements between people. Since the majority of the web APIs, that exist these days, don't implement this constraint, they can't really be characterized as being RESTful. Maybe "RESTish" at most.
And why is that HATEOAS constraint so important?
The HATEOAS constraint is important because it provides an important feature called "independent evolvability". This means that the development of the client and server code is decoupled and that different teams can develop the client and the server separately. Take a typical web browser and a web server, for example. The Firefox/Chrome development team needs to know nothing about the Apache/IIS development in order to successfully improve their product. Their product only needs to conform to that "uniform interface" constraint which guarantees those products can continue to communicate successfully, even though they are developed by different teams.
Now, take a look at a typical web API client implemented these days. It is usually very tightly coupled with its server through the extensive knowledge of server's internals. In the client source code there is usually a list of all the server endpoints (resource URIs with RPC-style web calls), plus it contains the business-level logic (like workflows) and it usually generates the URIs for subsequent requests to the API server.
$('#poster').html("Loading...");$.getJSON("http://api.example.org/2.1/foobar.search/en/json/31866/" + film + "?callback=?", function(json) { if (json != "Nothing found.") { $('#poster').html('We found you a poster: <img src=' + json[0].posters[0].image.url + ' />'); } else { $.getJSON("http://api.example.org/2.1/foobar.search/en/json/31866/goonies?callback=?", function(json) { console.log(json); $('#poster').html('Nothing was found. Perhaps you were looking for The Goonies: <img src=' + json[0].posters[0].image.url + ' />'); }); }});
Of course, it also doesn't implement the HATEOAS constraint, which means it doesn't use hypermedia formats (like XML, XHTML, HTML, ATOM, etc.) in its client-server communication. Mostly a JSON (JavaScript object notation) media format is used, but it's not a hypermedia format at its core and it doesn't have a standard way to represent basic hypermedia elements like links, embedded resources and relations. Instead, JSON is a format created primarily for the transmission of data objects in a human-readable format and all those JSON extensions (like Hal or Siren) are just trying to adapt that "object notation" format to a hypermedia-like format. On the other hand, using native hypermedia formats like XML, ATOM, XHTML or just HTML already gives you a basic set of hypermedia features out of the box.
If those API clients were using the hypermedia for client-server communication, the server could embed links into a response so the client would just have to follow links, without the need to keep the list of all the resource URIs. The server would be able to provide the list of possible actions/links a client can perform at a given state, following those links. That also means the server could manage the workflow through the use of hypermedia, so the client wouldn't have to worry about it.
The lack of hypermedia makes decoupling of the client/server code a bit difficult which eventually prevents the "independent evolvability". What would really help here is if we could move all the business logic from the client side to the server side, because we could then reuse our client for other web APIs, too. Or even better, just like a standard web browser, a standard API client would be able to emerge, handling just any kind of a hypermedia API. Wouldn't it be awesome if you wouldn't have to write an API client at all, each time you create a new hypermedia API?
Imagine now how would it look like if Facebook would provide its own web browser, that only knows how to browse the Facebook's website, then imagine Twitter, Microsoft, Google and all the other popular websites doing the same thing. That's what would have happened if today's world-wide-web was implemented as RESTish instead of RESTful.
Why so many people ignored this constraint and considered their APIs RESTful anyway?
After reading about this HATEOAS constraint, it was clear to me why it was an important feature of a RESTful API. But, to answer this question, I had to read a lot of discussions online in order to understand the reasoning of each side. So, my first impression was that other people maybe didn't care or didn't have time to read Dr. Fielding's dissertation (or at least a Wikipedia article on it), basically blaming it on the people's ignorance. But once I tried to improve one of my RESTish APIs to be truly RESTful, I realized the horrible truth. The amount of work that needs to be done to upgrade your RESTish API to be RESTful is just huge. It is probably easier to start everything from scratch.
"The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction." - Dr. Roy Fielding
That being said, it is clear that the hypermedia APIs are very convenient in cases where we need an ease of maintenance, rapid development and scalable deployment. On the other hand, hypermedia APIs will perform poorly if we need to build a time-critical API or a network-efficient service, without too much client/server chatter. These services should be implemented using some other, probably binary-based, form of communication, without caching and layering, to improve the response time and the information accuracy, etc. Obviously, we first need to get our requirements on the table and compare them with the REST requirements, before we decide if the RESTful API service is what we really need. It doesn't mean that, if our API is RESTful, that it's automagically the best possible choice for any use case.
Isn't it strange to use HTML for the web API?
Well, it depends on your goals. As previously said, if your goals match those of the REST then no, it's not strange. You should consider using HTML for your REST API (also XML, ATOM, etc.), because it will give you all the hypermedia elements out of the box. If you are worried about the overhead of the HTML communication (even though HTML can be gzipped with a very high compression ratio) and you want to use something that is smaller and faster, then you should also consider creating a stateful web service (take a look at the table of constraints to see the trade-offs for the "stateless" constraint). What I'm trying to say here is that you probably don't have the same set of goals as the REST web services do. In other words, are you sure you really need a RESTful web service? To properly answer this question, take your requirements (goals) and match them with those of REST. If they don't match, then REST might not be the best thing for you and you are probably better off using some other software architecture style for your web service.
Any interesting links on Hypermedia APIs?
Yes, there are many interesting videos and articles on this topic, but I'll only recommend a few for now, because these authors really understand the bigger picture and you can learn a lot from them. So, here they are:
Yes, there are many interesting videos and articles on this topic, but I'll only recommend a few for now, because these authors really understand the bigger picture and you can learn a lot from them. So, here they are:
A great video lecture by Mike Amundsen, recorded at Google in Mountain View, 22 April 2014:
REST, Hypermedia, and the Semantic Gap
Amundsen has authored numerous books and papers on programming over the last 15 years. His most recent book is a collaboration with Leonard Richardson titled "RESTful Web APIs" published in 2013. His 2011 book, "Building Hypermedia APIs with HTML5 and Node", is an oft-cited reference on building adaptable Web applications.
A great video lecture by Jon Moore, recorded at ØREDEV Developer Conference, Malmö, Sweden, 10 Nov 2010:
Session about Hypermedia APIs
Dr. Moore is the Chief Engineer at Comcast Interactive Media (CIM), a division of Comcast Corporation dedicated to developing and operating online and cross-platform entertainment and media businesses. He guides technical choices that allow CIM to bring innovative products to our customers ever more quickly.
An article about Roy Fielding on " Versioning, Hypermedia, and REST"
An article by Martin Fowler on "Richardson Maturity Model (Glory of REST)"
