Testing REST API using K6

In general, not all developers do REST API performance testing 🔥 they developed. In my opinion that is because they do not found a tool that is easy to use and easy to learn. Always I am saying "Do not write your own performance testing tools". The industry has so many tools for testing performance, I think some of them will fit you. Think its crucial stage in testing, because I want that consumer of my API will have the best experience. 🤗

Quite often at work, I have tasks to write REST API. Of course, I write unit tests, integration tests to prove my solution correctness. And always next stand the question, how to test the performance of your REST API. 🤔

In my experience, I saw a lot of self-made performance testing solutions starting from console application which calls API in a loop (sounds crazy? 🤪) and ends with multi-threaded API calls which crash by themselves at higher loads 💥. The worst issue when you create and use self-made tools, they are hard to share in the whole organization 🏢. You have to spend a lot of time supporting that tool, test it, document it and of course, develop new features. Have you so much time 🕘? I did not. Sometimes the self-made tool is worth the time you spend on supporting and testing it.

I spend some time researching 🔍 easy tools for testing REST API. The main condition for the tool was easy to use and easy to learn. I found K6. This tool is a fully open-source written in Go (https://github.com/loadimpact/k6).

The best developer experience for load testing.
Open source load testing tool and SaaS for engineering teams.

K6 supports different types of testing 🧪: smoke testing, load testing, stress testing, and soak testing. When you look at K6 features, CI integrations, and extensions it feels a very developer orientated tool. That is exactly what we looking for. 🧡

In the following steps, we will create a simple REST API, generate a testing script, and run some performance testing using K6. 👍

Build steps:

• Install prerequisite tools
• Create a Web API project
• Enable Swagger documentation
• Generate K6 tests from swagger.json
• Run performance tests

Prerequisites 🔨

Tools you have to install before create this example project:

• Docker - https://docs.docker.com/install/
• K6 - https://k6.io/docs/getting-started/installation

I have tested Windows and Linux K6 (WSL on Windows) installation. You can choose at your preferences. When you install on Windows don`t forget to add K6 binaries path in your PATH that you can run it from any location.

• .NET Core 3.1 SDK - https://dotnet.microsoft.com/download/dotnet-core/3.1
• Visual Studio Code - https://code.visualstudio.com/

Create a Web API project

Create a new .NET Core Web API project by using the Windows command prompt.

dotnet new webapi -n WeatherApi

Open newly created application using Visual Studio Code by File -> Open Folder.

For testing purposes, we need to add some slow performance simulation code. Change controller code WeatherForecastController.cs by following. ☁️

using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;

namespace WeatherApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        [HttpGet]
        public async Task<int> Get()
        {
            var randomizer = new Random();
            int randomDelay = randomizer.Next(0, 5);

            await Task.Delay(TimeSpan.FromSeconds(randomDelay));

            return randomDelay;
        }
    }
}

This code will create a synthetic delay to slow down API performance 🐌. Try to run your application and call multiple times https://localhost:5001/weatherforecast from browser.

Enable Swagger documentation

To generate K6 testing script automatically we have to enable Swagger documentation for our API by adding NuGet package.

dotnet add WeatherApi.csproj package NSwag.AspNetCore

Change Startup.cs to enable Swagger documentation.

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();
    // Register the Swagger services
    services.AddSwaggerDocument();
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }

    app.UseRouting();

    // Register the Swagger generator and the Swagger UI middlewares
    app.UseOpenApi();
    app.UseSwaggerUi3();
    
    app.UseEndpoints(endpoints =>
        endpoints.MapControllers());
}

Try to run application and open URL http://localhost:5000/swagger. If everything is configured correctly you will see your API documentation. Using this UI you can also do API calls.

More documentation about Swagger you can find in Microsoft documentation At this point we finished with our API and start the performance testing stage.

Generate K6 tests from swagger.json

Before generating K6 tests we have to save locally our auto generated swagger.json file by opening URL http://localhost:5000/swagger/v1/swagger.json.

{
  "x-generator": "NSwag v13.6.1.0 (NJsonSchema v10.1.21.0 (Newtonsoft.Json v9.0.0.0))",
  "swagger": "2.0",
  "info": {
    "title": "My Title",
    "version": "1.0.0"
  },
  "host": "localhost:5000",
  "schemes": [
    "http"
  ],
  "produces": [
    "text/plain",
    "application/json",
    "text/json"
  ],
  "paths": {
    "/WeatherForecast": {
      "get": {
        "tags": [
          "WeatherForecast"
        ],
        "operationId": "WeatherForecast_Get",
        "responses": {
          "200": {
            "x-nullable": false,
            "description": "",
            "schema": {
              "type": "integer",
              "format": "int32"
            }
          }
        }
      }
    }
  }
}

To generate K6 tests we have to use openapi-generator-cli it is open sourced tool that allows us to generate API clients. In our case API client is K6.

Using command line open directory you placed swagger.json file. From this directory we will run openapi-generator-cli.  I use a generator from Docker container by executing the following command.

docker run --rm -v ${PWD}:/local 
 openapitools/openapi-generator-cli
 generate -i /local/swagger.json -g k6 -o /local/k6-test/
 --skip-validate-spec

After successfully executing in directory k6-test you will find test script script.js. The script is generated in Javascript language. As you will see there is a single test which calls our API and checks if response status is 200.

import http from "k6/http";
import { group, check, sleep } from "k6";

const BASE_URL = "https://localhost:5001";
// Sleep duration between successive requests.
// You might want to edit the value of this variable or remove calls to the sleep function on the script.
const SLEEP_DURATION = 1;
// Global variables should be initialized.

export default function() {
    group("/WeatherForecast", () => {
        let url = BASE_URL + `/WeatherForecast`;
        // Request No. 1
        let request = http.get(url);
        check(request, {
            "": (r) => r.status === 200
        });
        sleep(SLEEP_DURATION);
    });
}

What I point out that in generated script BASE_URL has one unnecessary slash https://localhost:5001/. Need to remove slash at the end.

Run performance tests 🧪

Finally, we did all the required steps to test our API. To run K6 test script is quite simple. You have to execute following command. But don`t forget to run your application locally dotnet run.

k6 run script.js

After executing you will get an error WARN[0002] Request Failed error="Get https://localhost:5001//WeatherForecast: x509: certificate signed by unknown authority", that is because locally it uses self signed certificate. K6 can skip certificate validation by adding parameter --insecure-skip-tls-verify.

 k6 run script.js --insecure-skip-tls-verify

After executing the test script ⏱️ you will get a full summary and aggregated testing metrics.

But it is not so interesting, because the test executes our API only one time (summary parameter iterations). Let`s add some considerable scenarios. To define different load scenarios you have to add global options. The following code defines scenarios.

// Global variables should be initialized.
export let options = {
    stages: [
      { duration: '10s', target: 100 }, // below normal load
      { duration: '20s', target: 200 }, // normal load
      { duration: '10s', target: 300 }, // around the breaking point
      { duration: '10s', target: 400 }, // beyond the breaking point
    ],
  };

By executing these scenarios you will get more interesting results 🚀. You can play with different options and test your API. What is really nice about K6 that they have multiple metrics output options, you can choose from cloud based to on premise running tools like InfluxDB.

Happy performance testing!️ 🧪

[eof]