REST API Conventions
REST API Conventions
The key abstraction of information in REST is a resource. Any information that can
be named can be a resource: a document or image, a temporal service (e.g.
"today's weather in Los Angeles"), a collection of other resources, a non-virtual
object (e.g. a person), and so on. In other words, any concept that might be the
target of an author's hypertext reference must fit within the definition of a resource.
A resource is a concept ual mapping to a set of entities, not the entity that
corresponds to the mapping at any particular point in time.” - Roy Fielding’s
dissertation.
What to expect?
RESTful design is an architectural pattern and NOT a protocol. Hence there are no specific
thumb-rules which will guide you to create good REST compliant URIs and APIs.
My goal is to talk about core concepts of REST which one should keep in mind before creating
REST URIs and APIs. One of the key point to understand is that there is an infinite URI space
that is available to you, but with great power comes great responsibilities - it is good to avoid
resource proliferation that may add confusion to the API design. As long as there is a genuine
need for the resources with clear consumer “intent” which fits in the overall API design, a
particular URI space vector can be instantiated for consumption.
You need to have basic concepts on REST Architecture and nothing advanced to understand
this article.
Core Concepts
Idempotent methods
An idempotent HTTP method is a HTTP method that can be called many times without different
outcomes. It would not matter if the method is called only once, or ten times over. The result
should be the same.
x = 4;
x++;
The first example is idempotent: no matter how many times we execute this statement, 'x' will
always be 4. The second example is not idempotent. Executing this n times will result in
different outcomes.
OPTIONS YES
GET YES
HEAD YES
PUT YES
POST NO
DELETE YES
PATCH NO
Reification
Dictionary meaning of Reification is:
make (something abstract) more concrete or real
In other words, reification makes something abstract (e.g. a concept) more concrete/real. The
idea is to not focus on the entity itself, but on the intent. For example, let's take an upcoming
feature of capturing a lineage in Bedrock from internal and external client:
@RequestMapping(value = "/lineage", method = RequestMethod.POST)
public @ResponseBody JSONResponse saveLineage( @RequestBody LineageVO lineageVO) {
do execute security checks
do execute data validity checks
The above pseudo-code is about creating a new lineage detail for an entity and as expected it
goes through multiple functional steps. As you can see, "lineage" is an abstract idea and what it
primarily represents is entities and connections between them.
Note: This could have been modeled with entities like /baseuri/entity/lineage but
lineage operation is an aggregated business process by itself. A good URI will be
/baseuri/lineage
Reification plays a classic role in this as it helps to define a clear resource state for lineage. If
you had modeled it with "entity" then you are not clear on your "intent", as entity URI should be
operating on entities only and not creating relationships between them. However a reified idea
of "lineage" has a clear "intent" of resource state which is of relationships (DAG - directed
acyclic graph) of entities and its related properties. I hope you are able to see the differentiation
between the TWO "intended" states.
Also, you could have not created an URI /lineage and let the consumer call appropriate
smaller APIs to accomplish a task. That would have been worse as you are now migrating
business logic to the API consumer/client which is not desired at all. Further, by doing so we
make the application brittle and error prone. For example, let's assume the consumer forgets to
run security or data validity checks - implications can be catastrophic and make the application
unstable. There’s no reason for the client to have to do all this itself. The API provider needs to
pick one service to handle the coordination responsibility. In object oriented methodology you
can call it encapsulation. You will also see below this concept reinforced via Coarse Grained
versus Fine Grained Services.
Parts of Speech
There has been argument on what part of speech the REST URIs should be named i.e. whether
they should be nouns, verbs, adjectives and so on. However, we will focus on just TWO of them
which are NOUNs and VERBs. Generally, the resource names should be NOUNs and the
VERBs (action) are portrayed by HTTP verbs like GET, PUT, POST etc. Hence if you are trying
to create an entity, then you would name the URI as /base_uri/entity (entity being the
noun) with a VERB of HTTP POST.
But there are instances where business processes are better depicted as VERBs. For example,
let's take this Twitter API
https://api.twitter.com/1.1/direct_messages/sent.json. Here "Sent" is a
VERB but if you see the definition - its "intended" use is on messages - "Returns the 20 most
recent direct messages sent by the authenticating user." So one could argue that the resource
is not "sent", but rather "direct messages sent".
● If a process needs a resource state on its own then it can be part of the URI. As in the
above example - process "sent" has a STATE and it's INTENT(ed) use is on
RESOURCE "messages", hence it can be modeled in the URI.
● Also remember a NOUN can be converted to VERB and vice-versa. Let's take an
example of workflow registration. The URI can be /baseuri/workflow/register
with HTTP POST - although "register" is a VERB but it has a STATE on RESOURCE
"workflow", OR the URI can be /baseuri/workflow/registration with HTTP
POST, where "registration" is NOUN form of "register". We would recommend the
second option.
A guideline to follow here might be that a service shouldn't violate its "intent(ed) r esource".
Please pay close attention to the word "intent(ed) resource", what I mean is that a service
should abide by its responsibility for the resource it depicts. For example, if a process just needs
to create an entity resource then define an URI for that like /baseuri/entity with HTTP
POST. But if you want to define a workflow on entities then the need might be create an entity,
do additional steps and then create a workflow, in which case a URI like
/baseuri/entity/workflow with HTTP POST will be a bad REST URI as we are diluting
out "intent". A good REST URI will be /baseuri/workflow/registration as now
"workflow" represents a RESOURCE and everything related to workflow registration goes via
this service.
CRUD Operations
One of the common mistakes we make while designing REST interfaces is thinking in terms of
CRUD (Create, Read, Update, Delete) operations. This concept works very well from a data
access point of view but not from a REST perspective. If you take the above example of lineage,
thinking it from a CRUD perspective of entity domain model would have led to a wrong REST
design as we will either put a lot of lineage domain specific logic into the client or we overload
the entity with a resource state (like the relationship via DAG) it's not intended to represent.
Hence think from a point of service graininess, how low level or aggregated you
need so that you have correct representation of resource state. Applying SOA
(Service Oriented Architecture) principle - service that hosts the resource is the only
agent that can directly change its state.
Intent
Literal meaning of "intent" means "intention or purpose". Intent helps us to identify the
underlying resource and it's state representation. As per Roy Fielding's definition - "Any
information that can be named can be a resource: a document or image, a temporal service
(e.g. "today's weather in Los Angeles"), a collection of other resources, a non-virtual object (e.g.
a person), and so on." - a resource can be any abstract concept or concrete thing. Hence while
defining a REST API you need to be clear on intent of resource representation. The boundary
should be very crisp and clear with the service operating on a well defined state of a resource.
Each service (URI) then just POSTs their 'intents' or publishes the resource states they
themselves own.
Use HTTP Verbs to Mean Something
API consumers are capable of sending GET, POST, PUT, and DELETE verbs, and these verbs
greatly enhance the clarity of what a given request does. Generally, the four primary HTTP
verbs are used as follows:
GET
Read a representation of a specific resource (by an identifier) or a collection of
resources.
PUT
Update a representation of a specific resource (by an identifier) or a collection of
resources.
Use PUT to create new resources only when clients can decide URIs of resources. For
example, in Ingestion API, if the client can control the URI identifier of the fileSystem
type then use a PUT to create it else use POST.
DELETE
Remove/delete a specific resource by an identifier.
POST
Create a new resource or make various other changes to a resource. Also a catch-all
verb for operations that don't fit into the other categories. In POST the server decides the
URIs of the resource.
Suggested usages for the "Top 10" HTTP Response Status Codes are as follows:
200 OK
General success status code. This is the most common code. Used to indicate success.
201 CREATED
Successful creation occurred (via either POST or PUT). Set the Location header to
contain a link to the newly-created resource (on POST). Response body content may or
may not be present.
204 NO CONTENT
Indicates success but nothing is in the response body, often used for DELETE and PUT
operations.
Bedrock Handled Exception Types - ContentNotFoundException
401 UNAUTHORIZED
Return this when the client is not authenticated to access the resource. If your server will
not let the client access the resource even after authentication, then return 403 instead.
403 FORBIDDEN
Use this when your server will not let the client gain access to the resource and
authentication will not help. For instance, you can return this when the user is already
authenticated but is not allowed to request a resource, for reasons such as not having
the right access privileges.
In addition, the response message shouldn't have any HTTP Error Codes in them. Error codes
should be sent back part of the HTTP message body as per its specification. Please take a look
@ GitHub section on the same - https://developer.github.com/v3/#client-errors. The application
response message should contain error messages and its details.
Example (GitHub)
There are several examples of REST APIs out there and I have quoted some from Twitter
above. However I think the best one I have seen so far are from public GitHub API - it is very
well defined with an appropriate resource granularity.
You will see that all the above concepts are being applied to its resource naming. For example
GitHub has reified concepts of "Forks" and "Merges", which are both abstract concepts but
made concrete via merges and forks sub-collection. At the same time it also has very fine
grained resources like "Tag" as an API. Hence as you can see in practical implementation of
APIs, you will need both coarse grained aggregate "business processes" plus low level “nouns”
resources.
Summarizing
As you can see there is no single approach that will lead to a compliant REST URI. A fact I
mentioned before REST is an architecture and not a protocol. However if you follow the above
guiding principles, you will come up with a pretty well defined API. Here is a summary of what
we have seen so far:
● Idempotent Methods
○ The very first thing to keep in mind is which methods are idempotent and which
ones aren't. This will help in selecting the right HTTP Verb for the REST API.
○ There is another concept of "safe methods" which I haven't touched as I don't
think it plays a big role. However please feel free to read on that @
https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Safe_methods
● Reification
○ You will come across scenarios where you are trying to model a business
process or an aggregate process which cannot be represented using low level
nouns. In those scenarios think from an elevated angle and try to structure an
API which has a crisp "intent" and distinct representation of resource state.
● Noun versus Verb
○ Structure the URI's as NOUNs and if a process needs to be structured as VERB
then ensure it has a resource state on it's own.
○ HTTP Verb should complement as the action for the URI.
● Granularity of services
○ REST URI definitions should also be driven by the granularity of services i.e.
whether low level nouns suffice or needs to be an aggregated service.
○ Ensure business logic doesn't spill into client and all coordination is encapsulated
into a service
○ Don't think from a CRUD perspective, whereas always think from a resource
point of view.
○ URI needs to have a clear and crisp "intent" of the underlying resource state it is
representing.
○ Apply SOA design principles as almost most of the times they help in devising a
good REST URI.
● HTTP Verbs
○ Abide by the core definition of each of the HTTP verbs. They are part of the
REST URI and together they should indicate the "intent" of the URI.
● HTTP Response Codes
○ API responses should NOT contain HTTP error codes in their message body
whereas they should be in HTTP body. However API responses can contain
application specific error codes.
○ API responses should contain error details which helps the client display
meaningful messages to the user.
Cookbook
Feature Flag Operation
● Intent: Need to indicate to the user that this API is unavailable because the feature Flag
is turned off
● What is the action - respond with http 501 error code. Optionally respond with a
"Feature "FLAG_NAME" unavailable." message
Date Time
● Use the forward-slash separator (/) in the path portion of the URI to indicate a
hierarchical relationship between resources. For example, all the following URIs convey
a hierarchical association between path segments:
○ /ingestion/filesystems/{fileSystemId}/connections/{fileSys
temConnectionId}
○ /ingestion/filesystems/{fileSystemId}/connections
● Use the comma (,) and semicolon (;) to indicate nonhierarchical elements in the path
portion of the URI. The semicolon convention is used to identify matrix parameters:
○ I am not able to find a good example in Bedrock but what I mean is this -
/coordinates;w=39.001409,z=-84.578201
○ Note: not all code libraries recognize the comma and semicolon as separators
but Spring MVC does in the form of MatrixVariable.
● Use the ampersand (&) to separate parameters in the query portion of the URI. For
example:
○ /ingestion/filesystems/connections/search?sort=ascending
Save/Create Operation
● Intent: Need an API for Ingestion to save a "Connection" for a given "Filesystem"
● What is the action - SAVE which indicates a new connection object will be created and
identifier generated by the service, hence "Idempotent" property is NOT needed. POST
HTTP Verb is what we need .
● What is/are the resource(s) - "filesystem" and "connection" and hence each will have
an association with respect to each other.
● "Plural" OR "Singular" - Plural, as the resources are collections. Filesystem will have
an identifier in the URI as we are creating a connection for a filesystem and the
connection identifier will be generated by API.
● What is the URI - /ingestion/filesystems/{fileSystemId}/connections
Get/Read Operation
● Intent: Need API for Ingestion to get a "Connection" from a list of connections in a
filesystem.
● What is the action - GET which indicates an existing connection object will be retrieved
by the service, hence "Idempotent" property is needed. GET HTTP Verb is what we
need .
● What is/are the resource(s) - "filesystem" and "connection" .
● "Plural" OR "Singular" - Plural, as the resources are collections. As we can directly get
a connection from a list of connections, we are going to get our resource via a
connection identifier.
● What is the URI -
/ingestion/filesystems/{fileSystemId}/connections/{fileSystemCo
nnectionId}
Delete/Delete Operation
● Intent: Need API for Ingestion to delete a "Connection" from a list of connections in a
filesystem.
● What is the action - DELETE which indicates an existing connection object will be
deleted by the service, hence "Idempotent" property is needed. DELETE HTTP Verb is
what we need.
● What is/are the resource(s) - "filesystem" and "connection" .
● "Plural" OR "Singular" - Plural, as the resources are collections. As we can directly
delete a connection from a list of connections, we are going to delete our resource via a
connection identifier.
● What is the URI -
/ingestion/filesystems/{fileSystemId}/connections/{fileSystemCo
nnectionId}
● Note: As you can see the REST URI for GET and DELETE is identical but the difference
is in the intent/behavior which is depicted by the HTTP VERB (GET vs DELETE)
Search/Post Operation
● Intent: Need API for Ingestion to search from a list of connections in a filesystem.
● What is the action - This is a classic use case of 'reification' which means search is an
abstract concept and hence we need to make it concrete in REST resource parlance. As
it's a complex request object, hence we use POST HTTP Verb.
● What is/are the resource(s) - "filesystem" and "connection" .
● "Plural" OR "Singular" - It doesn't apply over here.
● What is the URI - /ingestion/filesystems/connections/search
● Note: It cannot be a GET operation as the API is not retrieving a resource on a single
resource property but a combination of resource properties.
Put/Create Operation
Put/Update Operation
Batch Operations
Asynchronous Operation
Question - Recently I got an issue of using HTTP method DELETE. Any API having DELETE
as request method can not return any value if invoked using java.net.HttpURLConnection.
(java.net.ProtocolException: HTTP method DELETE doesn't support output). This is a bug
solved in jdk-8. (http://bugs.java.com/view_bug.do?bug_id=7157360). Is that something we
should take care while making an API as our clients might not be having jdk-8?
Answer - Although Oracle portal mentions it is a bug fix but I don’t think it’s a bug and just an
enhancement to support non-defined HTTP RFC Requirement. If you look at this -
https://tools.ietf.org/html/rfc7231#page-29 there is a paragraph "A payload within a DELETE
request message has no defined semantics; sending a payload body on a DELETE request
might cause some existing implementations to reject the request.” Hence it’s not required
technically to send back a body, however it doesn’t hurt and is a good to have feature. Anyways
most of the popular AJAX APIs support so it’s not a problem there. But what we should do is not
to have a body for DELETE and let the client react on the HTTP status code like 200 etc.
However you might say that our REST responses have a JSON property to depict the status
code and message and for that I would say that’s something we don’t do correctly and should
be amended. HTTP error codes are self descriptive and it should be part of HTTP response
message not part of application response message. The below java portal link mentions what
doesn’t work.
However here is the code that does work and it is a perfect code in my opinion. There is no
response body but there is response code of HTTP which describes the state of the resource
operation perfectly.
URL url;
try {
url = new
URL("http://javafx-jira.kenai.com/rest/api/1.0/issues/47181/watchers"
);
HttpURLConnection connection = (HttpURLConnection)
url.openConnection();
connection.setRequestMethod("DELETE");
int responseCode = connection.getResponseCode();
System.out.println("Delete Method Response Code: " +
responseCode);
} catch (MalformedURLException e) {
// TODO handle Exception
} catch (IOException e) {
// TODO handle Exception
}
References
● Roy Fielding's REST dissertation (Section 4, 5 and 6 primarily) -
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
● Reification - https://en.wikipedia.org/wiki/Reification_(computer_science)
● Twitter REST APIs - https://dev.twitter.com/rest/public
● HTTP Status Codes - http://www.restapitutorial.com/httpstatuscodes.html
● GitHub AP - https://developer.github.com/v3/
● Richardson Maturity Model of REST services -
https://martinfowler.com/articles/richardsonMaturityModel.html
● Different kinds of REST API
Authentication.https://docs.google.com/spreadsheets/d/1tAX5ZJzluilhoYKjra-uHbMCZra
aQkqIHl3RIQ8mVkM/edit#gid=0