REST API Test

The rest-test command line tool is used to test and validate REST API.

Features

YAML based test definition and validation
Test Plan -> Test Suite -> Test Case
Optimized for JSON, file and plain response body
JSON response Body value filter
Support JSON path value filter
Support gval expression
Customized gval operators
- string convert a number, boolean value to string
- int convert a number string to int
- float convert a number string to float
- bool convert a number, string, list and map to bool
- len get length of a string, a map or a slice/array
- contain check whether an element in a list or map. Check whether a string is a substring of another string
JavaScript to evaluate values
YAML report

Usage

Download the latest release from rest release.

Bash

./rest -h
Usage of ./rest:
  -level string
        log level. levels: debug, info(default), warn, error (default "info")
  -logPath string
        log path
  -plan string
        a test plan
  -suite string
        a test suite

JSON Path, `gval`, JavsScript and Validators

Tip

JSON path and gval are only used in the key of validators.

JSON path can be used to get JSON value.

You can check it here JSON Path specification.

You can test it here JSON Path Test. You need write a JSON and click format, then you'll see the JSON path field.
gval

GO's JSON path implementation PaesslerAG/jsonpath is integrated with PaesslerAG/gval, so we can use both the functionalities from JSON path and gval.
JavaScript

JavaScript is based on dop251/goja, it used to process HTTP request/response body, environment variable value expand, and validators value expand

Validators

Validators is a JSON based JSON path validators, the syntax is like below.

YAML

# top level is a list, then it will be converted to {"and": [...,...]}
validators:
  - $.id : ${ctx.resp.id}
  - len($.child): 5

Each validator key will be handled with gval, JSON path and customized operators. Each validator value will be expaned with JavaScript.

The above equals to the following.

YAML

# top level is a list, then it will be converted to {"and": [...,...]}
validators:
  and:
    - $.id : ${ctx.resp.id}
    - len($.child): 5

A complex example.

YAML

and:
  - $.entryCount: 2
  - $.entries[0].id: ${ctx.create.resp.id}
  - $.entries[1].id: ${ctx.create2.resp.id}
  - or:
      - $.entries[0].name: 'test'
      - $.entries[1].name: 'test2'

Concept

Rest API test supports test plan, test suite and test case.

Test minial execution is test suite
Test plan contains one or more test suites
Test suite contains one or more test cases
Test case define a minial execution unit, including
- The HTTP request definition
- The HTTP response validation definition

Test Case

HTTP Request Definition

Here is a simple Test Case definition with static body

YAML

- desc: create a github server # the test case description
  enabled: true # indicate whether this test case is enabled or not, by default, it's true
  name: create # this defines a name for the test case, a named test case allow to reference this test case's request and response data in later test case
  environment: # define the test case environments
    API_PREFIX: http://localhost:8080/api/rest
  request:
    url: http://localhost:8080/api/rest/github # the HTTP Request URL, it supports environment expand. For example: ${API_PREFIX}/github. Where API_PREFIX is an environment variable `API_PREFIX=http://localhost:8080/api/rest`.
    method: POST # define the HTTP method
    parameters:
      name: xxx # the URL parameters
    headers: # define the HTTP request headers
      "Content-Type": application/json
    body: '{"web": "https://github.com", "api": "https://api.github.com/v3", "name": "github"}' # define the HTTP request body
  response: # any of the expect filed is not matched will fail the test case
      # the JSON response: {"id": 1}
      code: 201 # the expect HTTP response code
      contentType: application/json # the expect HTTP response context type
      body:
        type: JSON # the HTTP response body must be a JSON object
        array: false # whether the JSON is a JSON array
        validators: # gval and JSON path based JSON validators
          and: # support *and* or *or* operator
            - $.id: 1 # the JSON path and expect value

Load the HTTP request body from file

YAML

- desc: the test case description
  # other fields omit
  request:
    body:
      file: create_gh_server2.json # you can define the data directory to find the file in test suite or test plan level

Process the request JSON before sending with JavaScript

YAML

- desc: the test case description
  # other fields omit
  request:
    body:
      file: create_gh_server2.json # you can define the data directory to find the file in test suite or test plan level
      # use the JavaScript to update the name
      script: |
        let data = JSON.parse(body)
        data.name = 'test2'
        JSON.stringify(data)

Upload a file

YAML

- desc: the test case description
  # other fields omit
  request:
    body:
      file: file_2_upload.tar.gz # you can define the data directory to find the file in test suite or test plan level
      uploadFile: true # indicate to upload the file directly, this will ignore the script

Reference another test case request definition

YAML

- desc: the test case description
  enabled: true # indicate whether this test case is enabled or not, by default, it's true
  name: create # this defines a name for the test case, a named test case allow to reference this test case's request and response data in later test case
  environment: # define the test case environments
    API_PREFIX: http://localhost:8080/api/rest
  requestRef: create # this test case will use the HTTP previous request definition **create**
  response: # any of the expect filed is not matched will fail the test case
    code: 400 # the expect HTTP response code
    contentType: application/json # the expect HTTP response context type
    body:
      type: JSON # the HTTP response body must be a JSON object
      array: false # whether the JSON is a JSON array
      validators: # gval and JSON path based JSON validators
        - contain("UNIQUE constraint failed:", $.errorMessages[0].message): true

HTTP Response definition

JSON response body validation

YAML

- desc: the test case description
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/github
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # the JSON response: {"entryCount": 1, "entries": [{"id":1}]}
    code: 200 # the expect HTTP response code
    contentType: application/json # the expect HTTP response context type
    body:
      type: JSON # the HTTP response body must be a JSON object
      array: false # whether the JSON is a JSON array
      validators: # gval and JSON path based JSON validators
        and: # support *and* or *or* operator
          - $.entryCount: 1 #
          - len($.entries): 1
          - $.entries[0].id: ${ctx.create.resp.id} # use ctx.create to reference a named test case, resp is the response object

Don't use validators

YAML

- desc: the test case description
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/github
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # only validate the HTTP response code and contextType
    code: 200 # the expect HTTP response code
    contentType: application/json # the expect HTTP response context type

Validate the JSON array or object length

YAML

- desc: the test case description
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/github
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # only validate the HTTP response code and contextType
    code: 200 # the expect HTTP response code
    contentType: application/json # the expect HTTP response context type
    body:
      type: JSON
      length: 2 # the JSON object has 2 keys

Validate the JSON with JavaScript

YAML

- desc: the test case description
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/github
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # only validate the HTTP response code and contextType
    code: 200 # the expect HTTP response code
    contentType: application/json # the expect HTTP response context type
    body:
      type: JSON
      // use variable body to access the HTTP response JSON object
      script: if (body.entries[0].name != 'github') {
          throw new Error("invalid name")
        }

Validate the large file response

YAML

- desc: large file response
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/file/download?fileId=1
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # only validate the HTTP response code and contextType
    code: 200 # the expect HTTP response code
    body:
      type: file
      max: 1 # file max size
      min: 0 # file min size
      length: 2 # file exact size
      sha256: file expect sha 256

Validate the plain text response

YAML

- desc: validate HTTP plain response
  # other fields omit
  request:
    # request definition
    url: http://localhost:8080/api/rest/file/1/info
    method: GET
  response: # any of the expect filed is not matched will fail the test case
    # only validate the HTTP response code and contextType
    code: 200 # the expect HTTP response code
    body:
      type: plain
      length: 2 # string length
      regex: ^hello\s+world$ # go regex

Test Suite

A Tests suite must have 1 test case at least.

YAML

name: github_api # test suite name
depends:
  - xxx # dependency test suites, this must be used with test plan
type: suite # mandantory
enabled: true # default is true
environment: # define the environment for all its test cases
  API_PREFIX: http://localhost:8080/api
  API_TOKEN: some_token
  GH_SERVER: github.com
global: # dynamic
  headers: # global HTTP headers
    Authorization: Bearer ${env.API_TOKEN}
    Content-Type: application/json
  dataDir: ./data # where to find the test data
  apiPrefix: ${env.API_PREFIX} # it will be auto expand the test case url use this varaible
cases:
  - desc: list github servers
    enabled: true # default is true
    name: list
    request:
      url: github # dynamic
      method: GET # default is get
    response:
      code: 200 #
      contentType: application/json
      body:
        type: JSON
        array: false # json array or not, default is false
        length: 0 # json array length

Test Plan

A test plan must have at least 1 test suites

YAML

name: rest test plan # name
type: plan # mandantory
enabled: true # default is true
environment: # define the environment for all its test suites, they will be overriden by test suite or test case envrionments
  API_PREFIX: http://localhost:8080/api
  API_TOKEN: some_token
  GH_SERVER: github.com
global:
  headers:
    Authorization: Bearer ${API_TOKEN}
    Content-Type: application/json
  dataDir: ./data # where to find the test suite and test data
suites:
  - rest_test_suite # test suite name without file extension

Test Report

After the test execution completed, it will generate the test report as a yaml file.

Info

The plan report will be only avaliable when running with a test plan.

YAML

type: plan
name: test plan
status: Completed
executionTime: 123
totalTime: 12
error: xxx
suites:
    - name: rest test suite
      status: Completed
      executionTime: 123
      totalTime: 12
      error: xxx
      cases:
          - desc: list github servers
            name: xxx
            status: Passed
            executionTime: 123
            totalTime: 12
            error: xxx
          - desc: create github server
            status: Passed
            name:
            executionTime: 123
            totalTime: 12
            error: xxx

Test Execution Logs

The application will write every HTTP request with,

HTTP request
- URL
- Method
- Headers
- Body
May don't have body.
HTTP Response
- Status
- Headers
- Body

A typical exectuion log example.

Bash

ls 2024-08-28_20-32-18
1_1_1_list github servers(list)_request.txt
1_1_1_list github servers(list)_response.txt
report.yml
rest_test.log

1_1_1_list github servers(list)_request.txt

HTTP request log. The 1_1_1 is the plan ID, suite ID and test case ID, plus the test case description, test case name if have.
1_1_1_list github servers(list)_response.txt

HTTP response log.
report.yml

The report file.
rest_test.log

Execution logs.

Examples

gh-webhook API Test

REST API Test

Features

Usage

JSON Path, gval, JavsScript and Validators

Concept

Test Case

HTTP Request Definition

Here is a simple Test Case definition with static body

Load the HTTP request body from file

Process the request JSON before sending with JavaScript

Upload a file

Reference another test case request definition

HTTP Response definition

JSON response body validation

Don't use validators

Validate the JSON array or object length

Validate the JSON with JavaScript

Validate the large file response

Validate the plain text response

Test Suite

Test Plan

Test Report

Test Execution Logs

Examples

JSON Path, `gval`, JavsScript and Validators