Introduction

During my day-to-day work, especially with Microservices or integrating third-party services, it is very common thing for me to deal with web requests. But, unfortunately, I usually see a lot of old style approaches to send requests or at times, I encounter code that was written with well-known issues.

So, here, I'd like to share some opinionated thoughts and alternative approaches on this matter.

The content of the article is for developers who have some experience in web development using .NET and C#.

For those who don't want to read the full article and have an unstoppable desire to see the code, here is the link to the source code repository. However, for those of you who want to learn new things, I hope to provide such an opportunity, so keep on reading.

I will be using .NET 6.0 and Minimal APIs endpoint creation approach in this article. I know that this is not the latest version of .NET at the moment of writing, but I'd like to point out that it is LTS (Long-Term Support) and... well, in a general sense, the fundamental aspects of Minimal API have not changed, so I'm not overly concerned about this.

Also, I'm going to use publicly available Deezer API to diversify and, in a way, distinguish my article from similar ones. For those who aren't familiar with Deezer, it is online music streaming service that provides access to tons of audio content worldwide.

This post aims to showcase the opinionated usage of HttpClient, focusing primarily on it. I won't delve into many details of the relatively new .NET 6 features which I'm going to use during the article.

Crafting the project

The idea behind the project is to build a web app that retrieves the Top100 tracks by any artist from Deezer and order them afterwards.

To enhance the readability of my code snippets, I'll omit using keywords and also add some indents, but you always can find the complete code example in a dedicated repository here.

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.MapGet(
    "/tracks",
    ([FromQuery(Name = "artist")] string artistName) => artistName);

app.Run();

To start off I create a new project using .NET 6 minimal hosting model with a new HttpGet endpoint that matches to /tracks route. Also, I've added [FromQuery] attribute to allow my endpoint to search artists by provided name.

Now, how could web-request be sent using .NET tooling? Typically we would stick to something like HttpClient class. It basically serves the idea of sending HTTP requests and receiving HTTP responses from specified resources located somewhere on the Internet.

To send requests to Deezer API I'm going to declare DeezerHttpClient class. It will contain two methods: one is to find an artist by the artistName, and the second is to retrieve the Top100 artist's tracks by artistId.

I'm using constructor injection to get required HttpClient dependency from DI container.

public class DeezerHttpClient 
{
    private readonly HttpClient _httpClient;

    public DeezerHttpClient(HttpClient httpClient) 
        => _httpClient = httpClient;

    public void GetArtist(string artistName) 
    {
    }

    public void GetArtistTopTracks(int artistId)
    {
    }
}

So, how do I get HttpClient dependency? Well, first of all, I need to register DeezerHttpClient class into DI Container, like this:

builder
    .Services
    .AddHttpClient<DeezerHttpClient>(
        x => x.BaseAddress = new ("https://api.deezer.com/"));

You can spot that I've used AddHttpClient extension. Under the hood, it will add DeezerHttpClient to DI container along with a few other services, where one is particularly important for us: HttpClientFactory. It simplifies configuration and management of HttpClient instances. Also, using HttpClientFactory helps me avoid common problems related to HttpClient class management. But now, you may wonder what these problems are? Let's take a look at them.

The first issue arises when I try to do something like this:

    public async Task<string> GetArtist(string artistName) 
    {
        using var httpClient = new HttpClient();

        var response = await httpClient.GetAsync(
            new Uri($"https://api.deezer.com/search/artist?q={artistName}&limit=1"));

        if (!response.IsSuccessStatusCode)
        {
            throw new HttpRequestException();
        }

        var content = await response.Content.ReadAsStringAsync();

        return content;
    }

This is an example of how usually people send HTTP requests. Pretty straightforward, right? Later on, I will show how to improve this bit of code.

Since HttpClient implements IDisposable interface, the obvious idea that comes to mind is to wrap it into using statement, because all resources should be cleaned up after each use, right? Or we can rely a bit on DI container and register HttpClient with a transient lifetime using AddTransient extension method like so:

builder
    .Services
    .AddTransient<HttpClient>();

Anyway, if any of these approaches would be followed, many HttpClient instances might be created. Thus, it leads to a problem known as socket exhaustion. In short, when a lot of HttpClient instances were created too many sockets were occupied. Since there is a limit on how many sockets can be opened at one time and we can be faced with SocketException error. For those who are curious enough here is a link to an article explaining this kind of issue in detail.

Well, to be honest, it seems to me that for sending just a few web requests or testing some ideas on your local machine, it would be fine to use the previously described approach. However, if you would like to avoid such an issue, consider instantiating it once and reusing it throughout the life of an application by making HttpClient static or try to register it with singleton lifetime using AddSingleton extension method. Don't worry, you can easily do it because HttpClient is thread-safe. Mutable but thread-safe. You might not believe me at this point, but you probably won't argue with Microsoft guide.

It's funny enough, but the next issue arises when we attempt to implement the solution for first problem. Let's see it in action.

I won't dispose HttpClient instance anymore, instead, I'll try to reuse it and make it static, like so:

public class DeezerHttpClient 
{
    private static readonly HttpClient HttpClient = new();

    public async Task<string> GetArtist(string artistName) 
    {
        var response = await HttpClient.GetAsync(
            new Uri($"https://api.deezer.com/search/artist?q={artistName}&limit=1"));

        if (!response.IsSuccessStatusCode)
        {
            throw new HttpRequestException();
        }

        var content = await response.Content.ReadAsStringAsync();

        return content;
    }
}

Nothing changed dramatically, right? Well, since we started reusing HttpClient it establishes a new connection and keeps it open indefinitely. It turns out to be a problem because now we can not get DNS updates anymore. So, what could this issue lead to in the real life? In scenarios when DNS change might occur (for example - the server is gone), we would be sending requests to nowhere.

For those who are interested in getting more details about this problem, please check materials section in the end of the article.

Since HttpClientFactory was designed to resolve these problems, we are going back to previously used AddHttpClient extension:

builder
    .Services
    .AddHttpClient<DeezerHttpClient>(
        x => x.BaseAddress = new ("https://api.deezer.com/"));

I've made some updates to DeezerHttpClient. Instead of returning raw string content using ReadAsStringAsync(), now I serialize it to the generic typed model using another available extension method: ReadFromJsonAsync<T>(), like this:

public class DeezerHttpClient 
{
    private readonly HttpClient _httpClient;

    public DeezerHttpClient(HttpClient httpClient) 
        => _httpClient = httpClient;

    public async Task<DeezerHttpResponse<ArtistModel>>
    GetArtist(string artistName) 
    {
        var response = await _httpClient.GetAsync(
            $"search/artist?q={artistName}&limit=1");

        if (!response.IsSuccessStatusCode)
        {
            throw new HttpRequestException();
        }

        var content = await response.Content
            .ReadFromJsonAsync<DeezerHttpResponse<ArtistModel>>();

        return content;
    }

    public async Task<DeezerHttpResponse<TrackModel>>
    GetArtistTopTracks(int artistId)
    {
        var response = await _httpClient.GetAsync(
            $"artist/{artistId}/top?limit=100");

        if (!response.IsSuccessStatusCode)
        {
            throw new HttpRequestException();
        }

        var content = await response.Content
            .ReadFromJsonAsync<DeezerHttpResponse<TrackModel>>();

        return content;
    }
}

My DeezerHttpResponse<TModel> class looks as follows:

public class DeezerHttpResponse<TModel> where TModel : class
{
    public int Total { get; init; }
    public IReadOnlyCollection<TModel> Data { get; init; } = Array.Empty<TModel>();
}

I also created a few model classes. For convenience, I've put them in a single file here, but you are free to separate them respectively.

public class ArtistModel
{
    public int Id { get; init; }
    public string Name { get; init; } = "";
}

public class TrackModel
{
    public int Id { get; init; }
    public int Rank { get; init; }
    public int Duration { get; init; }
    public string Title { get; init; } = "";
    public AlbumModel Album { get; init; } = null!;
}

public class AlbumModel
{
    public string Title { get; init; } = "";
}

I remember that I promised to show how to improve the preceding code snippet. Again, I'd like to show just opinionated code improvements, and you might not necessarily want to follow them.

So far, so good, and it seems that there's nothing to improve here, right? Well, you can notice that I have some repeated logic. To reduce it, I'm going to show one neat HttpClient extension method that shipped with .NET. Let's see it in action:

public class DeezerHttpClient
{
    private readonly HttpClient _httpClient;

    public DeezerHttpClient(HttpClient httpClient) 
        => _httpClient = httpClient;

    public Task<DeezerHttpResponse<ArtistModel>> 
    GetArtist(string artistName)
        => _httpClient
            .GetFromJsonAsync<DeezerHttpResponse<ArtistModel>>(
                $"search/artist?q={artistName}&limit=1");

    public Task<DeezerHttpResponse<TrackModel>>
    GetArtistTopTracks(int artistId)
        => _httpClient
            .GetFromJsonAsync<DeezerHttpResponse<TrackModel>>(
                $"artist/{artistId}/top?limit=100");
}

The method name GetFromJsonAsync is quite similar to ReadFromJsonAsync, which we have used previously to deserialize JSON to a typed model. But, essentially, under the hood, GetFromJsonAsync not only deserializes the model but also uses the whole bunch of code to make a request, receive a response and then deserialize message content. Let's take a brief look at the decompiled code of this extension method. I simplified the code snipped below a bit, but it remains the same and serves to the same idea as we had before:

public static async Task<TValue?> GetFromJsonAsync<TValue>(
    this HttpClient client, 
    string? requestUri)
{
    if (client == null)
    {
        throw new ArgumentNullException(nameof(client));
    }

    var taskResponse = client.GetAsync(
            requestUri, HttpCompletionOption.ResponseHeadersRead);

    using (HttpResponseMessage response = await taskResponse)
    {
        response.EnsureSuccessStatusCode();

        using (Stream contentStream = await content.ReadAsStreamAsync(content)) 
        {
            return await JsonSerializer.DeserializeAsync<TValue>(contentStream);
        }
    }
}

At this point, I am almost done with the example. The only thing left is to update my "/tracks" endpoint, where I'm going to retrieve the Top100 artist's tracks using created DeezerHttpClient:

app.MapGet(
    "/tracks", 
    async ([FromQuery(Name = "artist")] string artistName,
    DeezerHttpClient client) =>
    {
        var artists = await client.GetArtist(artistName);
        var artist = artists?.Data.First();

        if (artist is null)
        {
            return Results.BadRequest(
                new {message = $"Artist {artistName} not found"});
        }

        var tracks = await client.GetArtistTopTracks(artist.Id);

        return Results.Ok(new
        {
            Count = tracks!.Total,
            Tracks = tracks!.Data
                .Select(x => new
                {
                    x.Id, 
                    x.Rank,
                    x.Title,
                    Album = x.Album.Title,
                    Duration = TimeSpan.FromSeconds(x.Duration),
                })
                .OrderByDescending(x => x.Rank)
        });
    });

That's it! The last thing remaining is to try out what have done, but I'm going to skip this part in the article. For those who are curious enough, this task will be a home exercise. If you want, you can share your favourite artist's top tracks in the comments. Furthermore, I'll leave a quick tip below on what request you should send after you build and run the project. Keep in mind that you might have a different port number.

https://localhost:7001/tracks?artist=Imagine Dragons

Conclusion

In this article, I aimed to showcase a non-typical way to send web requests, usually not easily encountered on the internet. Additionally, I described common issues that popped out when HttpClient is used incorrectly.

Materials

Additionally, you can check the links, provided below, to find out more information on the article's topic.

Yet Another Way to Use ASP.NET Core HttpClient

Introduction

Crafting the project

Conclusion

Materials

Subscribe to my newsletter

Eduard

Eduard