How to build an automated API test framework - Part 1

For a while now I've wanted to do a series of posts on how to build an automated API / web service / integration testing framework and especially how to build one in a robust way.  There is a lot to go over so I plan to break this series up into three posts to focus on specific areas of the framework and in this one we'll focus on getting the framework up and running with a handful of automated checks.

These posts assume you have some working knowledge of how HTTP works.

The problem

We want to build a suite of automated checks against the web service restful-booker.  However, the service offers no UI so our checks will use HTTP to make requests and then assert against the response.  We also want to be able to run these checks in a continuous integration environment.

For this iteration, we will focus on building checks around the GET endpoints of restful-booker.

Getting started with an automated API test framework

To run a suite of GET checks we are going to need:

• A framework to run our checks
• Methods to create our HTTP requests and return a response
• Checks to trigger our API

Whilst the model we're designing can be applied using various languages we'll use Ruby for this tutorial and whilst I will attempt to explain everything I am doing it will help if you have prior experience of using Ruby.   In summary, we will be using the following:

• ruby-2.2.1 - A popular language with testers
• Rake - Allows us to create tasks to manage dependencies and run our checks
• Rspec - Framework to create and organise our checks
• Rest-client - Builds and triggers HTTP requests whilst handling responses to be used in our checks

Implement an RSpec framework

First of all, we want to get our framework up and running, so we'll start by pulling down the gems we require by using Bundler:

1. Create a new directory integration-framework
2. Inside the new directory create a new file named Gemfile with the following content

require 'spec_helper'

describe('Restful-booker') do

  it('/booking should return a 200') do
    puts 'Done'
  end

end

With the Gemfile created we can run bundle install to install all the dependencies we need to start building our framework together.  Next, we'll create the skeleton for RSpec and get a single check passing.

• Within our root folder, create a new folder named spec
• Inside the folder, we want to create two files integration_spec.rb and spec_helper.rb
• spec_helper.rbis responsible for tying together the framework, but for now, we just need to add in the following: require 'rspec'

Next, we want to create an initial check for the framework to run, so we add the following into integration_spec.rb

require 'spec_helper'

describe('Restful-booker') do

  it('/booking should return a 200') do
    puts 'Done'
  end

end

Of course, this test doesn't actually check anything yet, but we want this dummy test to pass to confirm that we have everything setup correctly.

Finally, we are now at the point of trying to run the framework which we do by creating a Rakefile in our root directory with the following:

require 'rspec/core/rake_task'

RSpec::Core::RakeTask.new(:spec)

task :default => :spec

The RakeTask will find all the checks in the spec folder and run them, feeding back to us whether it has passed or not.  Since our task :spec is tied to :default we can simply navigate to a folder via command line and run: rake

And we should receive something similar to this as an output:

Finished in 0.04645 seconds (files took 0.41312 seconds to load)
1 examples, 0 failures

If you did then that means our RSpec framework is in place and we are ready to get start with building our API library.  If not go back and check that your code is correct, your requires are correct and files are in the correct folders (Try comparing your work to my framework stored on GitHub)

Implement rest-client

With RSpec in place, we could simply begin writing our checks, dropping our HTTP requests into each request as we require.  However, ultimately this will lead code that is hard to read, maintain and will be brittle to changes in the API.  Therefore, much like how we use Page objects in a Webdriver based framework to create a library of objects/classes that reflect each page on a site, I like to create a library that reflects the various endpoints of the API I am testing.

So how do I structure this library?  Let's look at the structure as we create a library for our current framework.  To begin with we create a new folder in the root directory named api which will contain the API library.

Next, we want to create a file inside the api folder named booking.rb, but why are we creating this file?

If the service under test is a mature RESTful service then the API will be grouped based on the resources that it can serve, and that is how I group my objects (or in this instance modules as we are using Ruby).  For example, if the services' API has URIs that reference to resources such as apple, orange and pear (in the form of /apple, /orange,/pear) then I would create three different objects name apple.rb, orange.rb, and pear.rb which will contain a method for each URI related to that resource.

Restful-booker only has one resource which is a booking resource, so, therefore, I only need to create booking.rb and add the following inside the file:

require 'rest-client'

module Booking

  def all_bookings
    begin
      return RestClient.get 'http://localhost:3001/booking'
    rescue => e
      return e.response
    end
  end

  def specific_booking(id, accept)
    begin
      return RestClient.get 'http://localhost:3001/booking/' + id.to_s, :accept => accept
    rescue => e
      return e.response
    end
  end

end

What we have created is two separate functions mapped to two different endpoints in restful-booker using rest-client

• all_bookings will call the endpoint /booking
• specificbooking will call the endpoint /booking/{id} or /booking/1

Adding them to specific functions allows us to also parameterize aspects of our HTTP request such as which booking we want to request (id) and what headers we might want to use in the request (accept).  The calls themselves are wrapped in begin's to ensure whatever response, whether good or bad, is returned to assert within our checks, which we will do in the next step

Let's create some checks!

Now that we have an API library we can import the booking module into our checks and then design some tests by adding the following to spec_helper.rb

require './api/booking'

This will make the booking module available to our spec files which we will update to the following:

require 'spec_helper'

include Booking

describe('Restful-booker') do

  it('/booking should return a 200') do
    response = Booking.all_bookings

    expect(response.code).to be(200)
  end

  it('/booking/{id} should return a 200') do
    response = Booking.specific_booking(1, :json)

    expect(response.code).to be(200)
  end

  it('/booking/{id} should return a 418 when sent a bad accept header') do
    response = Booking.specific_booking(1, :text)

    expect(response.code).to be(418)
  end

end

We add the Booking module into the checks by using include Booking and then can use the Booking module to make the calls we need for our checks.  With the checks structured in this way if, say, for example, the specific_booking URI changed we would only need to change the URI in one place to propagate the change into all our checks.

So let's try running rake again in our command line. Hopefully we should get an output similar to this:

Finished in 0.04645 seconds (files took 0.41312 seconds to load)

3 examples, 0 failures

Conclusion

So to recap, we've set up RSpec to run using Rake, created a small automated API library that we then use to create our automated checks.  Hopefully, your framework should look similar to this in structure:

In the next post, we'll focus on checks around POST endpoints which will involve extending our API library and creating a payload library.