Kate @kalabro and I @kkomelin were pleased to give a presentation at DrupalCamp Baltics 2015 in Tallinn, Estonia on August 28th.
Here are the results of our thorough and deep research of RESTful solutions in Drupal, which we believe nobody has made before. We hope you'll like the presentation and our speech transcript, and leave your opinion in comments ;)
When we were preparing to the presentation we got excited about the topic. It's so interesting and large-scale that it deserves a separate conference. We've got a lot to say about protocols and APIs but today we'll primarily focus on REST.
Intro
Some time ago if I needed to create API in Drupal, I would simply use the Services module because it was the only well-known solution. Today it's not so easy and quick to choose from all options available in Drupal. Kate and I decided to compare them and share the results. So, let's have some REST! ;)
Buzzwords
In the Drupal world you have probably heard such marketing terms as Decoupled, Headless and API-First. They all have about the same meaning - you put the Drupal core on backend, enable API on top of it and create a separate frontend app, which consumes that API, so that the standard Drupal representation layer is not used any more. By the way, REST has also become a marketing term that people sometimes use to call any HTTP-based API.
What is API for?
There are many areas where APIs are useful. Here are a few examples.
Mobile apps
First of all, mobile apps. They need an external API for storing data, processing payments, almost for everything.
Microservices
Imagine that each function of your system is a separate service and the only way to use it is through API. These are microservices.
Frontend apps
It's trendy now to split backend and frontend of the application to two separate parts that communicate through API. At the end of our presentation we'll show you an example of the Angular app that consumes Drupal 8 RESTful Web Services. Stay tuned!
What is REST?
The REST term was coined by Roy Fielding in his Ph.D. dissertation. Representational State Transfer is an architectural style for implementing Web systems. in fact, it's just a set of constraints that define how a well-designed Web application should function. REST itself is not a standard but it prescribes the use of standards such as HTTP, URI and some others.
REST Basics
In REST, we have resources with URI identifiers and representations of the resources in the formats of HTML, XML, JSON, etc. A Client sends a request to the Server using one of the HTTP methods and the Server responds with a representation of the resource. The table illustrates resources and methods.
The important thing here is that all messages between the Client and the Server are self-descriptive, which means that they contain enough information to be processed. This information is transferred in the URI, query parameters, body or headers.
RESTful or RESTless
Sometimes designers of the REST API don't follow REST architectural style precisely. That can happen because of a natural tendency to simplify things, make them easier to use. Let's think of it as an evolution instead of raising void holy wars. Alright... now let's move to the the most interesting part - comparison of the existing solutions. (Kate, please)
Comparison
From theory to practice
Now it's time to move to the more practical part. In the next section, we are going to compare existing Drupal solutions for API building and input the results...
Results will be in the table. Intro.
...into this big table. Let's start with a quick introduction of all solutions listed in the heading of this table. The first one is Services module. Actually, how popular is it? Please raise your hand if you have used it. Services Yes, Services is a very popular mature module with a lot of features inside. It promotes itself as a standardized solution for building APIs in Drupal. We gave Services 5 stars for popularity. RestWS The next player is the Restful Web Services module, also known as RestWS. It is 1 year older that Services but much less popular. RestWS has very clean architecture based on Entity API. Out of the box, it provides API endpoints for every entity on the site. It doesn't have any settings except permissions. According to Drupal.org, it is used on more than 4 thousand sites. So 3 stars for popularity. RESTful Our next competitor has a pretty similar name: RESTful. RESTful is a framework for developers. It is built with modern REST API concepts in mind. It has some settings in the UI but most of the work has to be done in code. This module is still new but very promising. We gave it 2 stars, but its popularity is growing. JS & Endpoint Let's move to more exotic solutions. Imagine you have built your custom API callback in hook_menu. And it works. But as your site becomes more popular you will notice that your simple code is rather slow. It is because of Drupal's full bootstrap process. To avoid it, you can use JS module or its successor for APIs: Endpoint module. They both provide some infrastructure to run your code at lower bootstrap levels. This approach isn't very popular as you can't use all the goods of Drupal before full bootstrap is done. 1 star. Drupal 8 REST Finally, let's look into the future. In Drupal 8, we have REST module in the core. It's full name: RESTful Web Services. The idea is the same as RestWS: every entity has its endpoint. Additionally, there is REST UI contrib module to configure REST resources from the UI. This solution will definitely become very popular when Drupal 8 will enter the market. 2 stars at the moment.
From Popularity to Documentation
Now, we're done with Popularity criteria. Next...
Documentation
We are interested in how easy it is to start using these modules. If you wonder how we have calculated these points, we have some additional data under the hood. Documentation Details I'll skip it now but all the data will be available on SlideShare after the session.
From Documentation to Extensibility
In short, there is no module with 100% documentation coverage. But most of these modules provide some decent docs to start with. So, imagine we succeed in the initial configuration of API in Drupal.
Drupalism output
Our node's json output will look like this. In the real world, you will probably want to customize this output: add or remove some data, rename fields and so on.
Extensibility & Hooks
So, we need some hooks!
Extensibility
After we have calculated the extensibility score we recognised a strong correlation between project size and extensibility. Code Statistics So, we decided to include this data into the presentation to give you some idea of how big these modules are. Surprisedly, RESTful has won by code size. But let's keep the order and start with Services. Services Services has custom plugin architecture, that's why it provides 18 own hooks. The most popular is hook_service_resorces(). It should return an array with custom keys which have some meaning for Services. The real work is done in your callback functions. RestWS It uses all the goods of Entity API and has 7 hooks. The most popular is hook_restws_resource_info() where you declare your resources. Instead of callbacks you should create controller class based on RestWS interface. You are not limited by entities in your custom resources. RESTful This module has only one hook. It is because it heavily uses Ctools plugins. To create your custom resource you need to implement Ctools plugin directory hook and create a controller which implements RestfulInterface. Endpoint Endpoint isn't really a module. It just provides a custom bootstrap function. To start using Endpoint you should create an api.php file in the root of your site and call endpoint_route with array of your API endpoint. Drupal 8 REST REST module uses Drupal 8 framework with all those Plugins, Routes, Annotations and so on. To declare your custom resource you need to create a class which extends ResourceBase and to place it in specially named folder in your site. To enable this new resource you can place the rest.settings.yml file in your module or enable it from REST UI.
From Extensibility to Authentication
So, the larger modules have more features and more ways to extend them, but all the modules provide programming API to create your custom resources. Next, what about security?
Security & Authentication
Again, we did some research. I'll skip the details and do quick review. X-CSRF-Token Let's start with CSRF-Token. It's not authentication method. It's a way to protect the site from cross-site request forgery attacks when you are doing writing requests like POST or PUT. It is supported and enabled in most of modules. Cookie Auth Cookie authentication is built in Drupal and supported by every module in the list. To use it client has to send header with Drupal session cookie on every request. HTTP Basic Auth Another widely supported authentication method is HTTP Basic Auth. With this method you have to send username and password on every request. Very simple but pretty insecure. Token Auth This is modification of Cookie Auth for APIs specially. Instead of session, you are using string token that has some additional data and settings. Potentially, this method can be implemented in any module, but only RESTful has it build-in. OAuth & OAuth2 Drupal has contrib implementations of OAuth and OAuth2. Services supports both. RESTful has support of OAuth2 server. Drupal 8 development in progress. If you want to configure authentication flow with OAuth please be prepared to spend a lot of time on it because it's usually a very complex task.
From Authentication to Performance
I guess you all are waiting for our next section, right? And now Konstantin will share with you results of our benchmarks.
Performance. How did we count?
I've seen a bunch of presentations about performance. Many of them measured HTTP GET. However, GET can be cached by clients and it is not so critical measure, so we've decided to count POST.
- Ubuntu 14.04, Nginx 1.8.0, Mariadb 10.0.20, PHP 5.5.9 with php5-fpm, 1GB RAM
- Minimal Drupal Profile
- Node with just Title and Body
- Disabled Drupal cache
- Anonymous requests
- HTTP POST to create entities
- Apache Benchmark (ab)
- Clean database after each ab run
Performance. Benchmark w/o Endpoint
Here are the results:
RestWS (orange) is the first here with 82 rps. RESTful (grey) and Services (yellow) are close to each other, they served less than 40 rps. Drupal 8 (blue) has a great potential but from our point of view it's not yet ready for a real load. Actually, we lost 3-7% of requests due to 500 Internal Server Error. You may wonder where Endpoint is.
Performance. Benchmark w/ Endpoint
And here it is:
As you see, Endpoint (green) is the fastest but Its tests are quite synthetic. The problem with Endpoint is that it's not so easy to replicate node_save call having just a configuration level of bootstrap. So we calculated the time of one node_save call beforehand and added the delay to our Endpoint callback. So, we give Endpoint 5 stars and Drupal 8 gets 1 star. restws - 4, restful 3 и services 2.
Comparison (continue)
Auto API Docs
Now, you may think it's already enough information but it's not! An API Is Only as good as its Documentation.
An API Is Only as good as its Documentation.
This is very good slogan from apiary.io and we totally agree with it. API should be understandable to both human and machine. In perfect world REST API is self-documented, but in real Drupal world we still need to spend some effort on writing documentation. Services Documentation API For Services, there is Services Documentation API to write docs in code and UI. It generates initial docs automatically, but from our experience it's not clear enough for those who unfamiliar to Drupal. Self Documenting REST API (7.x) RestWS has similar module too. rest_api_doc module was initially developed for Drupal 8 and backported to Drupal 7 later. It generates documentation from registered RestWS Controllers and Entity API metadata. RESTful RESTful implemented automatic documentation in more RESTful way. Any resource returns its documentation on OPTIONS request. GET request to root of your API returns list of all available endpoints. Pretty handy. Self Documenting REST API (8.x) As I already said, rest_api_doc was initially developed for Drupal 8. Thus, it plays nicely with core REST module.
From Auto API Docs to Versioning
There is one more topic we can't skip: versioning. In the future REST principles should reduce our need in versioning. But today it is in fact standard for public APIs. Services In Services, you might noticed that some of resources have Version dropdown. You can implement the same, but versioning API is rather complicated. A more common way is to use multiple endpoints with version number in path. RESTful Restful has built-in versioning for resources. You just need to follow naming convention.
Versioning Conclusion
For other modules in the table you have to invent your own versioning mechanisms. We simply gave 5 stars if versioning is supported and one — if it's not. And...
Table finished
It seems we did it! Congratulations to all of modules. Good job!
Slides online
Of course, all these data will be available online.
Conclusion. Big table
So, we've just learned the theory of REST, compared several available solutions by different criteria, such as performance, popularity and others. We hope this information will save you a few hours when you need to create a RESTful API in Drupal.
As we can see, evolution creates new solutions. We believe Drupal 8 with its built-in! RESTful Web Services is definitely a result of evolution too. And we are confident it will make a difference soon.
Conclusion
In conclusion, we'd like to encourage you, as a community, to join forces instead of creating yet another RESTful module or your custom implementation of API.
"Better to make a difference together than make it different alone."
Now it's time to test our demo.
Leave feedback
The source code of the demo app is on Github.
We would appreciate your live feedback and questions.
