Integration Testing for ASP.NET APIs (1/3) - Basics

Having automated tests to ensure that your API is working (and continues to work) as expected and designed is a huge productivity boost and safety net for rapid change.

This article will be the first of a series of posts that show different techniques that can be used that go beyond simple unit tests where you test a single method or class.

Key Ingredients

Since the topic here is integration tests, we want to be able to make API calls to our API methods and use a real data store for the data that it will return.

Three (possibly four, depending on your situation) key ingredients make for a super potent mix to make this type of testing easy, predictable, and fast:

• WebApplicationFactory - a class provided by Microsoft.AspNetCore.Mvc.Testing that lets us host an in-memory version of our API as well as swap out certain dependencies when needed
• XUnit Fixture functionality - ability to provide a way to share context among our test cases and a place where we can set up test data
• Bogus - a library that we can use to generate test data based on rules we provide.
• TestContainers - a way to create SQL Server or Postgres (or other databases) that we need for our API. Note that if you use SQLite you can use an in-memory version without needing a TestContainer instance.

Testing Mindset

A key concept regarding the use of automated tests to verify the behavior of our API is focusing on the results of the API. Here are some things that we should likely want to test (depending on whether the concepts are present in your API):

• anonymous call behavior
• GET requests with and without parameters
• Routes that don't exist
• Error responses
• POST or PUT requests - successful and validation errors
• authentication and authorization logic
• pagination logic
• existence (or not) of swagger endpoint(s)
• cascading object behavior (e.g. proper handling of parent-child relationships)
• handling of different responses / performance / errors from calls to other APIs
• health check behavior

Step One: Use XUnit and WebApplicationFactory

We'll start the journey with a super-simple API that expands only a little on the "weather forecast" API that you get as a sample when you create a new .NET 8 API.

I've added the following features that we'll want to test:

• GET method now requires a postalCode string parameter that is 5 digits
• Use of ProblemDetails for error responses
• Inclusion of the ASP.NET Core environment name in the results
• An invalid value for the postalCode parameter will return a BadRequest (400) with ProblemDetails
• The value of "error" for the postalCode will return an InternalServerError (500) with ProblemDetails

Our goal with the above is to establish a foundation where we can:

• Write tests that will call the API
• Overwrite the ASP.NET Core Environment variable to see something other than "Development"
• Verify the returned content and response codes - both for successful and failed calls

Create the Test Project

I created the test project by using the "XUnit Test Project (C#)" in the standard .NET project templates.

Then you need to add a reference to the Microsoft.AspNetCore.Mvc.Testing NuGet package.

Lastly you need to edit the .csproj file for the test project and make the top-level project node use the "Web" project type:

1<Project Sdk="Microsoft.NET.Sdk.Web">

Create Tests Using WebApplicationFactory

Once you've got the test project created, it's time to create tests.

Here's a starter class with a single test for the "happy path" of when the GET method is called with a valid postalCode parameter:

 1public class WeatherForecast(WebApplicationFactory<Program> factory)
 2    : IClassFixture<WebApplicationFactory<Program>>
 3{
 4    private readonly List<string> _possibleSummaries =
 5        ["Freezing", "Bracing", "Chilly", "Cool", "Mild",
 6         "Warm", "Balmy", "Hot", "Sweltering", "Scorching"];
 7
 8    [Fact]
 9    public async Task HappyPathReturnsGoodData()
10    {
11        var client = factory.CreateClient();
12
13        var forecastResult = await client.GetFromJsonAsync<ForecastAndEnv>(
14            "/weatherForecast?postalCode=12345");
15
16        Assert.Equal("Development", forecastResult?.Environment);
17        Assert.Equal(5, forecastResult?.Forecast.Length);
18        foreach (var forecast in forecastResult?.Forecast!)
19        {
20            Assert.Equal(forecast.TemperatureF, 32 + (int)(forecast.TemperatureC / 0.5556));
21            Assert.Contains(forecast.Summary, _possibleSummaries);
22        }
23    }
24}

Here are some notes about the code above.

The class declaration is this:

1public class WeatherForecast(WebApplicationFactory<Program> factory)
2    : IClassFixture<WebApplicationFactory<Program>>

It uses the new primary constructor syntax to get a WebApplicationFactory of type Program via dependency injection and is using the IClassFixture interface from XUnit.

To enable this to work properly, I needed to add the following line to Program.cs of the API project itself:

1public partial class Program { } // needed for integration tests

With those elements in place, you can create a client that can call your API with the following line:

1var client = factory.CreateClient();

To actually make an API call, we can do a line like the following (from the code above):

1var forecastResult = await client.GetFromJsonAsync<ForecastAndEnv>(
2            "/weatherForecast?postalCode=12345");

This uses the built-in extenstion method from System.Net.Http to deserialize the response and verify a successful status.

Note that we can't actually see what status is being verified and if the deserialization doesn't work we'll simply get an exception. We'll address these shortcomings soon. :)

Then we can do whatever assertions we want against the results. In the code above, I'm checking the environment value, and the contents of the 5-element array.

Here's another test:

 1[Fact]
 2public async Task MissingPostalCodeReturnsBadRequest()
 3{
 4    var client = factory.CreateClient();
 5
 6    var response = await client.GetAsync("/weatherForecast");
 7
 8    var problemDetails = await response.Content.ReadFromJsonAsync<ProblemDetails>();
 9
10    Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
11
12    Assert.NotNull(problemDetails);
13    Assert.Equal("Postal Code is required.", problemDetails.Detail);
14    Assert.Equal(400, problemDetails.Status);
15}

Note that in this test - we're expecting a status of 400 Bad Request and so we can't use the handy deserialization method from the previous test. We get the raw response and then have to evaluate status and handle deserialization inside the test code.

Step Two: Add HttpClient Extensions and a Base Class

To simplify our the calls to our API, where we want to both check various HTTP status codes as well as the returned JSON content, we'd like to have a simple helper method that would enable calls like these:

1var forecastResult = await Client.GetJsonResultAsync<ForecastAndEnv>(
2    "/weatherForecast?postalCode=12345", HttpStatusCode.OK);
3
4var problemDetails = await Client.GetJsonResultAsync<ProblemDetails>("/weatherForecast",
5    HttpStatusCode.BadRequest);

Note in both cases we're passing arguments that clarify the HttpStatusCode we're expecting from a given call.

Here's a method that does what we need and also supports outputing the JSON content to the test output if something was amiss with deserialization (check the full source code for the class on GitHub):

 1public static async Task<T> GetJsonResultAsync<T>(this HttpClient client, string uri,
 2    HttpStatusCode expectedHttpStatus, ITestOutputHelper? output = null)
 3{
 4    var response = await client.GetAsync(uri);
 5    Assert.Equal(expectedHttpStatus, response.StatusCode);
 6    var stringContent = await response.Content.ReadAsStringAsync();
 7    try
 8    {
 9        var result = JsonSerializer.Deserialize<T>(stringContent, JsonWebOptions);
10        Assert.NotNull(result);
11        return result;
12    }
13    catch (Exception)
14    {
15        if (output != null) WriteJsonMessage(stringContent, output);
16        throw;
17    }
18}

We can put the above code into a static class in the Utilities folder to avoid distracting it with our tests.

Additionally, we can create a simple base class to avoid the factory.CreateClient() call repetitions.

That class is as simple as this:

1public class BaseTest : IClassFixture<WebApplicationFactory<Program>>
2{
3    protected HttpClient Client { get; }
4
5    protected BaseTest(WebApplicationFactory<Program> factory)
6    {
7        Client = factory.CreateClient();
8    }
9}

And now the class declaration for the BetterWeatherForecast looks like this:

1public class BetterWeatherForecast(WebApplicationFactory<Program> factory, ITestOutputHelper output)
2    : BaseTest(factory)

Even better - the test that we need to write for a BadRequest response now becomes much more terse and readable (compare these 7 lines to the 13 in the other MissingPostalCodeReturnsBadRequest test above):

1[Fact]
2public async Task MissingPostalCodeReturnsBadRequest()
3{
4    var problemDetails = await Client.GetJsonResultAsync<ProblemDetails>("/weatherForecast",
5        HttpStatusCode.BadRequest);
6
7    Assert.Equal("Postal Code is required.", problemDetails.Detail);
8    Assert.Equal(400, problemDetails.Status);
9}

Getting better!

Using ITestOutputHelper

There is a skipped test in the BetterWeatherForecast set of tests:

 1//[Fact]
 2[Fact(Skip = "Run this when you want to see the output helper in action")]
 3public async Task ShowOutputHelperWhenTestFails()
 4{
 5    var problemDetails = await Client.GetJsonResultAsync<WeatherForecast>("/weatherForecast",
 6        HttpStatusCode.BadRequest, output);
 7
 8    // this test fails since I'm trying to deserialize a WeatherForecast instead of
 9    // a ProblemDetails but if you look at the test output you'll see the JSON response
10    // at the bottom of the output (under the exception)
11}

The comment explains the reason it's skipped (it would be a failing test), but running it shows how you can use the ITestOutputHelper to see what actually was returned by the API in the event that it didn't match your expectation (just by using its WriteLine method).

Here's a screenshot of the results when the test is included / run:

Code for the method that created the above output is in the HttpClientExensions.

Step Three: Use a Custom WebApplicationFactory

In many of our tests we will want to adjust the behavior or services of the API to support our tests. Using a custom (inherited) version of the WebApplicationFactory class can let us do that.

Here's a simple one:

1public class CustomApiFactory<TProgram> : WebApplicationFactory<TProgram>
2    where TProgram : class
3{
4    protected override void ConfigureWebHost(IWebHostBuilder builder)
5    {
6        builder.UseEnvironment("test");
7    }
8}

This class provides an override for the ConfigureWebHost method, and inside it updates the Environment to be "test" instead of the "Development" value that is set by the existing launchSettings.json file.

Note this line from BetterWeatherForecast.cs:

1Assert.Equal("Development", forecastResult.Environment);

This is part of a passing test that uses WebApplicationFactory.

A new test class called CustomWeatherForecast can be created that has the following declaration:

1public class CustomWeatherForecast(CustomApiFactory<Program> factory, ITestOutputHelper output)
2    : BaseTest(factory), IClassFixture<CustomApiFactory<Program>>

Note that it can use the same (unchanged!) base class introduced with the BetterWeatherForecast. But this is using our new CustomApiFactory<Program>, so when we get a client that will call this version of the API, it should be running with an environment of "test" (note line 7 of the following code):

 1[Fact]
 2public async Task HappyPathReturnsGoodData()
 3{
 4    var forecastResult = await Client.GetJsonResultAsync<ForecastAndEnv>(
 5        "/weatherForecast?postalCode=12345", HttpStatusCode.OK);
 6
 7    Assert.Equal("test", forecastResult.Environment);
 8    Assert.Equal(5, forecastResult.Forecast.Length);
 9    foreach (var forecast in forecastResult.Forecast)
10    {
11        Assert.Equal(forecast.TemperatureF, 32 + (int)(forecast.TemperatureC / 0.5556));
12        Assert.Contains(forecast.Summary, _possibleSummaries);
13    }
14}

Next Up... Data

This post established some basic foundations for integration tests in ASP.NET Core APIs. In the next post, we'll build on top of these concepts and add some data to our APIs with both SQLite and Postgres and see how we can create integration tests for that. The exact same techniques as Postgres could be used for SQL Server.

Stay tuned!