Test Example

What follows is a commented example of some tests in a single file demonstrating many of the Test Format features. See Loader, below, for the Python needed to integrate with a testing harness.

# Fixtures can be used to set any necessary configuration, such as a
# persistence layer, and establish sample data. They operate per
# file. They are context managers, each one wrapping the next in the
# sequence.

fixtures:
    - ConfigFixture
    - SampleDataFixture

# There is an included fixture named "SkipAllFixture" which can be
# used to declare that all the tests in the given file are to be
# skipped.

# Each test file can specify a set of defaults that will be used for
# every request. This is useful for always specifying a particular
# header or always requiring SSL. These values will be used on every
# test in the file unless overriden. Lists and dicts are merged one
# level deep, except for "data" which is copied verbatim whether it
# is a string, list or dict (it can be all three).

defaults:
    ssl: True
    request_headers:
        x-my-token: zoom

# The tests themselves are a list under a "tests" key. It's useful
# to use plenty of whitespace to help readability.

tests:

# Each request *must* have a name which is unique to the file. When it
# becomes a TestCase the name will be lowercased and spaces will
# become "_". Use that generated name when limiting test runs.

    - name: a test for root
      desc: Some explanatory text that could be used by other tooling

# The URL can either be relative to a host specified elsewhere or
# be a fully qualified "http" or "https" URL. *You* are responsible
# for url-encoding the URL.

      url: /

# If no status or method are provided they default to "200" and
# "GET".

# A single test can override settings in defaults (set above).

    - name: root without ssl redirects
      ssl: False
      url: /
      status: 302

# When evaluating response headers it is possible to use a regular
# expression to not have to test the whole value.

      response_headers:
          location: /https/

# By default redirects will not be followed. This can be changed.

    - name: follow root without ssl redirect
      ssl: False
      redirects: True
      url: /
      status: 200 # This is the response code after the redirect.

# URLs can express query parameters in two ways: either in the url
# value directly, or as query_parameters. If both are used then
# query_parameters are appended. In this example the resulting URL
# will be equivalient to
# /foo?section=news&article=1&article=2&date=yesterday
# but not necessarily in that order.

    - name: create a url with parameters
      url: /foo?section=news
      query_parameters:
          article:
              - 1
              - 2
          date: yesterday

# Request headers can be used to declare media-type choices and
# experiment with authorization handling (amongst other things).
# Response headers allow evaluating headers in the response. These
# two together form the core value of gabbi.

    - name: test accept
      url: /resource
      request_headers:
          accept: application/json
      response_headers:
          content-type: /application/json/

# If a header must not be present in a response at all that can be
# expressed in a test as follows.

    - name: test forbidden headers
      url: /resource
      response_forbidden_headers:
          - x-special-header

# All of the above requests have defaulted to a "GET" method. When
# using "POST", "PUT" or "PATCH", the "data" key provides the
# request body.

    - name: post some text
      url: /text_repo
      method: POST
      request_headers:
          content-type: text/plain
      data: "I'm storing this"
      status: 201

# If the data is not a string, it will be transformed into JSON.
# You must supply an appropriate content-type request header.

    - name: post some json
      url: /json_repo
      method: POST
      request_headers:
          content-type: application/json
      data:
          name: smith
          abode: castle
      status: 201

# If the data is a string prepended with "<@" the value will be
# treated as the name of a file in the same directory as the YAML
# file. Again, you must supply an appropriate content-type. If the
# content-type is one of several "text-like" types, the content will
# be assumed to be UTF-8 encoded.

    - name: post an image
      url: /image_repo
      method: POST
      request_headers:
          content-type: image/png
      data: <@kittens.png

# A single request can be marked to be skipped.

    - name: patch an image
      skip: patching images not yet implemented
      url: /image_repo/12d96fb8-e78c-11e4-8c03-685b35afa334
      method: PATCH

# Or a single request can be marked that it is expected to fail.

    - name: check allow headers
      desc: the framework doesn't do allow yet
      xfail: True
      url: /post_only_url
      method: PUT
      status: 405
      response_headers:
          allow: POST

# The body of a response can be evaluated with response handlers.
# The most simple checks for a set of strings anywhere in the
# response. Note that the strings are members of a list.

    - name: check for css file
      url: /blog/posts/12
      response_strings:
          - normalize.css

# For JSON responses, JSONPath rules can be used.

    - name: post some json get back json
      url: /json_repo
      method: POST
      request_headers:
          content-type: application/json
      data:
          name: smith
          abode: castle
      status: 201
      response_json_paths:
          $.name: smith
          $.abode: castle

# Requests run in sequence. One test can make reference to the test
# immediately prior using some special variables.
# "$LOCATION" contains the "location" header in the previous
# response.
# "$HEADERS" is a pseudo dictionary containing all the headers of
# the previous response.
# "$ENVIRON" is a pseudo dictionary providing access to the current
# environment.
# "$RESPONSE" provides access to the JSON in the prior response, via
# JSONPath. See http://jsonpath-rw.readthedocs.io/ for
# jsonpath-rw formatting.
# $SCHEME and $NETLOC provide access to the current protocol and
# location (host and port).

    - name: get the thing we just posted
      url: $LOCATION
      request_headers:
          x-magic-exchange: $HEADERS['x-magic-exchange']
          x-token: $ENVIRON['OS_TOKEN']
      response_json_paths:
          $.name: $RESPONSE['$.name']
          $.abode: $RESPONSE['$.abode']
      response_headers:
          content-location: /$SCHEME://$NETLOC/

# For APIs where resource creation is asynchronous it can be
# necessary to poll for the resulting resource. First we create the
# resource in one test. The next test uses the "poll" key to loop
# with a delay for a set number of times.

    - name: create asynch
      url: /async_creator
      method: POST
      request_headers:
          content-type: application/json
      data:
          name: jones
          abode: bungalow
      status: 202

    - name: poll for created resource
      url: $LOCATION
      poll:
          count: 10 # try up to ten times
          delay: .5 # wait .5 seconds between each try
      response_json_paths:
          $.name: $RESPONSE['$.name']
          $.abode: $RESPONSE['$.abode']

Loader

To run the tests they must be generated in some fashion and then run. This is accomplished by a test loader. Initially gabbi only supported those test harnesses that supported the load_tests protocol in UnitTest. It now possible to also build and run tests with pytest with some limitations described below.

UnitTest Style Loader

To run the tests with a load_tests style loader a test file containing a load_tests method is required. That will look a bit like:

"""A sample test module."""

# For pathname munging
import os

# The module that build_tests comes from.
from gabbi import driver

# We need access to the WSGI application that hosts our service
from myapp import wsgiapp


# We're using fixtures in the YAML files, we need to know where to
# load them from.
from myapp.test import fixtures

# By convention the YAML files are put in a directory named
# "gabbits" that is in the same directory as the Python test file.
TESTS_DIR = 'gabbits'


def load_tests(loader, tests, pattern):
    """Provide a TestSuite to the discovery process."""
    test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
    # Pass "require_ssl=True" as an argument to force all tests
    # to use SSL in requests.
    return driver.build_tests(test_dir, loader,
                              intercept=wsgiapp.app,
                              fixture_module=fixtures)

For details on the arguments available when building tests see build_tests().

Once the test loader has been created, it needs to be run. There are many options. Which is appropriate depends very much on your environment. Here are some examples using unittest or testtools that require minimal knowledge to get started.

By file:

python -m testtools.run -v test/test_loader.py

By module:

python -m testttols.run -v test.test_loader

python -m unittest -v test.test_loader

Using test discovery to locate all tests in a directory tree:

python -m testtools.run discover

python -m unittest discover test

See the source distribution and the tutorial repo for more advanced options, including using testrepository and subunit.

pytest

Since pytest does not support the load_tests system, a different way of generating tests is required. A test file must be created that calls py_test_generator() and yields the generated tests. That will look a bit like this:

"""A sample pytest module."""

# For pathname munging
import os

# The module that build_tests comes from.
from gabbi import driver

# We need access to the WSGI application that hosts our service
from myapp import wsgiapp

# We're using fixtures in the YAML files, we need to know where to
# load them from.
from myapp.test import fixtures

# By convention the YAML files are put in a directory named
# "gabbits" that is in the same directory as the Python test file.
TESTS_DIR = 'gabbits'


def test_gabbits():
    test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
    # Pass "require_ssl=True" as an argument to force all tests
    # to use SSL in requests.
    test_generator = driver.py_test_generator(
        test_dir, intercept=wsgiapp.app,
        fixture_module=fixtures)

    for test in test_generator:
        yield test

This can then be run with the usual pytest commands. For example:

py.test -svx pytest-example.py