How to Debug Web API Problems in 4 Steps

killalldefects.com/2021/08/30/how-to-debug-web-api-problems-in-4-steps

August 30, 2021

This article is a step-by-step diagnostic guide for working through common issues with
web services. The article assumes that you have a web client and a server and are trying to
get the two of these to communicate but are having issues.

This article was written for Tech Elevator students and graduates who typically work with
Java, C#, and JavaScript technologies, but the contents of this article are language
agnostic.

Web Services and Complexity

Code is complicated and many things can go wrong. However, when you add web
communications to the mix, a much larger amount of issues can occur beyond compiler
and logic errors.

Some things that can go wrong include:

Network issues prevent you from talking to the other server

The other server might be down for maintenance
New code on the server’s side may require you to change how you call their API
Your request data may not be in a valid form the server can accept (for example,
trying to book an appointment on a day that the business is closed)
Users may not have filled out required pieces of data
Your login information may be missing or may have expired requiring you to log in
again

All of the issues above are things that can break an already working API, so as you can
imagine, getting a new API working for the first time can be very hard.

However, if you follow a logical progression you can diagnose and resolve most web
service issues.

Diagnosing API Issues

Here are four steps I’ve learned to follow when diagnosing issues:

1. Get Information on the Request and Response

2. Identify the Problem Area
3. Divide and Conquer
4. Debug and Resolve

1/12
We’ll look at each one of these in turn.

Step 1: Get Information on the Web Request

The first thing to look at when diagnosing web communication issues is to get information
about the request being sent and the response coming back.

If you are making a simple GET request that doesn’t require authentication, you can
typically do this in the browser with your browser’s developer tools open and the network
tab selected. See Chrome’s official tutorial for more information about how to access this
feature.

Chrome’s network tools (pictured above) allow you to select a specific request and get
details on a number of things on the Headers tab, including:

The Request URL

The Request Method (or verb)

2/12
 The resulting Status Code
Headers associated with the request and the response
The Request Payload (or body)

The Preview and Response tabs will also provide more information about the returned
data as pictured below:

Sometimes you need to diagnose non-GET requests or requests that require

authentication tokens. In these cases I like to test from the wonderful (and free) Postman
application.

With Postman, you can specify the method/verb you want to use, the URL to use, set any
headers you need (such as an authentication bearer token), and provide a body to
accompany your message (make sure you set the content to raw with an application JSON
content type).

3/12
Once that’s done, clicking Send will make your request and display the results in the lower
pane, allowing you to observe the status code, response body, and response headers.

Alternatively, if your application is making calls from code, you can use a debugger to get
details on the request and response based on whatever library you are using to make web
requests. However, be very careful if you are doing this because it is very easy for
developers to assume details about the requests they are making that are incorrect.
Postman removes this concern and allows you to verify the behavior of the web service
independent of your client code.

See Postman’s wonderful learning center for more information on how to get started with
that application.

Step 2: Identify the Problem Area

Now that you have information on the actual request and response that you are
encountering, you can start to determine what’s going wrong.

In my experience, the most helpful thing to do is to determine where the error is

occurring – if the client is sending a bad request or if the server is sending back a bad
response. Once you can focus on one of the two areas, the problem becomes a lot less
daunting.

As you’re looking over the information about the request and response it’s important to
verify that the request looks correct.

Here are some of the things I commonly check:

1. Is the request using the appropriate URL?

2. Is it using the right method/verb? E.G. is it doing a GET when it should be doing a
POST?
3. If the request is a POST or a PUT, is it including a body?
4. Does that body look correct? Even little things like sending a string “1” instead of a
number 1 can make a difference.

4/12
If you are certain that the request looks correct in all aspects, look at the response. The
response can tell you a lot about how the server handled your request. There are two very
important pieces of information associated with the response: the response status and the
response body.

Using the Response Status

The response status is a code ranging from 100 to 599. As a developer, the status codes
you’re most likely to see are in the 200-299 range, the 400-499 range, and the 500-599
range.

Each of these major ranges of status codes represents something different. The 200 series
of status codes is typically used for things going without issues. On the other hand, the
400 and 500 series status codes all indicate something went wrong or is invalid. 400
series status codes indicate that the server thinks there’s something wrong with the
request whereas 500 series status codes typically indicate things going wrong with the
server.

Although it can be intimidating, these status codes can quickly communicate what
happened to you as a developer. Because of this, they are important to learn. Typically if
the response status is not in the 200-299 range, something went wrong and you’ll need to
diagnose it if it is not immediately apparent.

We’ll talk more about some of the individual status codes people see most as developers
later on in step 4.

Using the Response Body

While the response status is a simple numerical code, the body of the response could be
anything from completely empty to a simple error message to a complex JSON object. The
contents of the response body depend on the server technology that you are working with
(some of them will automatically generate responses for invalid requests, for example)
and based on whatever response data your web service returns.

When you look at the response body, you want to check for two things:

1. Does the response body tell me anything interesting? For example is an error
message included in this response?

If it doesn’t, your server may have a bug. If it
2. Does the response body look correct?
does, your API may be working correctly but you may have a bug working with the
result of the API call.

If you suspect your server may not be returning the correct result based on the body you
got back, it’s probably a good idea to put debugging breakpoints on your server and step
through what happens when you send that request.

Step 3: Divide and Conquer

5/12
You should now have a reasonable idea of what message your client is sending to the
server and what response the server is sending back.

At this point your primary goal is to divide the problem in half.

Because software systems are very complex, it can be helpful to rule things out and focus
on the remaining factors. This principle is extremely helpful when it comes to working
with web services.

You should now have enough information to answer the following questions:

Is the client sending an appropriate message to the server?

Is the server responding with a successful status code and useful response body?

If you answered no to the first question, your problem at least starts with the client-side
code which is not sending a valid request.

If you answered no to the second question, your server is not correctly handling the
request from the client either due to an issue with the request (either content or
authentication) or due to an error or logic flaw on the server. See more details in step 4 for
diagnosing what’s going on, but typically if the status code is in the 400s you investigate
the client and if the status code is in the 500s you investigate the server.

In the case where the server is returning 200 status codes but invalid data, that typically
indicates a logic problem on the server-side and you should investigate the server.

In the rare case where the client is sending an appropriate request and the server is
replying with a good response but things still appear buggy in your application, you likely
have a problem on the client-side in handling the response to your data and should look at
that code.

Step 4: Debug and Resolve

At this point, you’ve focused on just the client or just the server. Congratulations, this
makes your job significantly easier. Your job now is to confirm the problem through
debugging and make an appropriate fix.

Unfortunately, given how many different problems there could be and how many different
status codes there are in web services, this makes my job in writing about debugging
difficult.

When things are hard, I like to use pictures, so let’s take a look at a flowchart I created.
This flowchart starts at the orange rounded rectangle at the top of the diagram and flows
from there based on decisions in diagrams. Have your status code handy and follow the
chart below:

6/12
You may need to click on the image to expand it or right-click it and open it in a new tab
for the best effect.

If you follow through this process, you’ll likely see a blue box containing the error you
encountered and at least one green box recommending solutions or additional steps you
can take.

I’ll outline some of the more common problems and the steps you take to address them in
the section below, but if your error isn’t present here you may need to refer to Mozilla’s
list of status codes or do some web searching. 

Common Issues

Let’s take a look at some of the common status codes and what you can do about them.

No Response / Connection Refused

In the case where your server doesn’t return a response at all, you’ll usually get some form
of “Could not connect to the server” message. This typically happens when your internet is
out or your server is not running.

Another reason this can happen is if the server is paused on a breakpoint it may not
handle subsequent requests.

Make sure you have a valid internet connection and your server is running and not stuck
on a breakpoint..

200-299 Status Codes

7/12
If you get back a 200 series status code, it means the server thinks everything is fine.
However, if you are debugging things and trying to figure out why something is working,
that means that one of two things is going on:

1. The server is returning a 200 series response but not doing the correct thing /
returning the correct response body
2. The server is returning a 200 series
response and a correct body, but a bug still
appears to the user

In case #1 you should debug the server and see how it goes about fulfilling the request
because some form of a logic error is occurring where your program is not doing exactly
what you expect it to do.

In case #2 you should debug the client’s code to handle the server’s response since it
seems the API is functioning correctly. You may need to take a step back and think about
how your application should work at a high level in order to spot the bug.

400 – Bad Request

A bad request is typically encountered when the client sends information that doesn’t
match what the server is expecting. This could be anything from a missing piece of data to
a value being in string form instead of a number to an invalid zip/postal code being sent
over.

Typically when a server returns a 400 bad request, the response body will tell you more
about what is invalid about the request. You can also take a look at the request body to
make sure that the request looks valid and all parameters are present and in the
appropriate type.

401 – Unauthorized

This error occurs when a client requests something that is secure but the server can’t
verify who they are. Typically this happens when the client isn’t including an
authentication token in its request headers, but it can also happen if an authentication
token has expired.

Inspect your request and make sure you are sending the appropriate authorization header
to the server.

If you are working with an external API, you may need to consult their documentation or
contact them to discover how they want you to authenticate.

403 – Forbidden

If a 401 unauthorized occurs when a server doesn’t know who you are and requires login
to get a specific resource, a 403 occurs when a server knows who you are but that identity
isn’t allowed to access a specific resource.

8/12
Photo byTinus Marte onUnsplash

Think of it this way: Your server is a security guard standing in front of a very important
doorway. A request walks up to the doorway, but the guard denies entrance. The request
then says “Do you have any idea who I am?”. If the guard responds “No”, that’s a 401
unauthorized. If the guard replies “Yes, but you’re not allowed back here”, that’s a 403
forbidden.

403 forbidden responses are not bugs at all. These are instances where your application’s
security is preventing unauthorized access to something. If you think you should have
access to something, double-check your user’s permissions and the permissions the server
enforces on the resource you requested.

404 – Not Found

This status code is the most commonly known one outside of development circles since it
shows up in many browsers when a link is invalid or a URL is not typed correctly.

The 404 Not Found response code occurs in one of two ways in API development:

1. You tried to call an API endpoint that isn’t present on the server
2. You called a valid API endpoint requesting a specific thing that isn’t there anymore

The first case is the most common, so let’s talk about that. If I have a server that supports
GET on a URL like /api/menu and you do a GET request for /api/menuitems, this is
when a server is going to respond 404 telling you that there is no supported endpoint that
handles a GET request to /api/menuitems. In this case, it’s best to compare the
request’s URL to the URLs the server supports.

9/12
The second case occurs when you are trying to get something specific that is not present.
For example, if I did a GET request to /api/products/5003 but the largest product
number in the database was 250, the server wouldn’t be able to find the product I’m
looking for and would justifiably return a 404 telling me that it wasn’t there.

In both cases, you should double-check your client’s logic and make sure that you are
requesting an appropriate URL. Using the network tab in your browser is helpful for
verifying that the requested URL is what you thought it was.

405 – Method Not Allowed

A 405 occurs when a server offers an endpoint that supports a specific method such as
GET, but you try to access it using a different method. For example, if a server offered an
/api/menu endpoint that supported a GET and I did a DELETE to that endpoint, I
would get back a 405 method not allowed.

Typically a 405 occurs when you are using an incorrect URL for an operation or when the
server has not yet implemented the method you’re trying to call. Double-check the API
and make sure your URL is correct.

415 – Unsupported Media Type

This error is one that is most likely to occur inside of Postman. In Postman if you forget to
change the content type of the request Body to JSON, you are likely to get a 415 response.

This is because the server doesn’t know how to handle the contents of your request. Make
sure you set the content type appropriately in Postman.

If you are encountering this from code, you are likely either not specifying the content-
type header of your request, or are setting it to something the server does not support.
However, this is less frequently an issue so I will not go into it in more detail here.

420 / 429 – Too Many Requests

Some servers support rate limiting which is a way to throttle incoming requests and make
sure that no one user overwhelms an API with too many requests in a specific time span.

10/12
Typically if you encounter a 420 or 429 response code it means you are calling the server
too frequently and either need to decrease the number of requests you send or need to
upgrade to a paid tier.

Consult your API’s documentation for more details.

Other 400 Series Status Codes

Sometimes you’ll get different responses from 400 – 499 that are not listed here. These
tend to be more situational errors when working with unusual APIs. Refer to Mozilla’s
documentation on the specific error code and consult the documentation of the API you
are trying to work with. If the API is run by someone else, you may need to reach out to
their support team.

500 – Internal Server Error

This typically happens when an exception is thrown in the server’s code while trying to
handle the request. Take a look at the response body and see if it tells you where the error
occurred. You can also put a debugger breakpoint on your server and step through until
the error occurs. 

Once you identify the offending line of code, make a change and test again.

Other 500 Series Status Codes

Sometimes you’ll get responses above 500. These typically refer to networking issues,
servers being temporarily offline, etc. and are much harder to diagnose. Refer to Mozilla’s
documentation on the specific error code and consult your network administrator as
needed.

Step 5: Rinse and Repeat

Yes, I know, I claimed it was a 4 step process. It’s lied because when one problem is
resolved, it may not be the end of your process.

After resolving a problem in web services, it is important to test again and go back to step
1 if a new problem is encountered.

Software is complex and many times there may be more than one problem preventing
something from working. Sometimes you need to work through two or three problems in
sequence before your application gets back into a working state.

Closing
As you can see, APIs are complex but powerful things. The learning curve is steep, but
these technologies are critical to modern web development.

Understanding APIs and API debugging is a core skill for developers of all types which is
why we put such an emphasis on it at Tech Elevator.

11/12
Beyond developers, technology professionals such as quality assurance professionals,
project/product managers, system administrators, and technical leadership all need to
have a firm understanding of the basics of APIs and the challenges they present to
development teams and development projects.

In short, learning APIs is vital and something you need to do. Hopefully this guide helps
you along your way.

12/12