Boosting C# Performance: A Guide to BenchmarkDotNet

In the fast-paced world of software development, the need to assess the performance of algorithms or compare the efficiency of libraries is a recurring challenge. Developers often find themselves racing against tight deadlines to deliver a certain feature or product on time, so benchmarking is a thing occasionally overlooked. However, investing some extra time might pay off in the future. Therefore, benchmarking can be a valuable addition to our toolkit, especially in scenarios where performance is critical.

In C#, we have BenchmarkDotNet to accomplish this task very precisely. Here is how to use it.

Example

Let's start comparing string concatenation algorithms. As it is well known, in .NET it is always preferred to use the StringBuilder class. Let's see if that is true.

Project setup

First, let's create a Console project and install the required Nuget package.

dotnet add package BenchmarkDotNet

Please, keep in mind that BenchmarkDotNet works only with Console Apps.

Next, we will create our benchmarking class like this:

using System.Text;
using BenchmarkDotNet.Attributes;

namespace Benchmarking;

public class StringConcatenationBenchmark
{
    private const int NumberOfConcatenations = 100;

    [Benchmark]
    public string AdditionAssignment()
    {
        var result = "";
        for (var i = 0; i < NumberOfConcatenations; i++)
        {
            result += i;
        }
        return result;
    }

    [Benchmark]
    public string StringConcat()
    {
        var result = "";
        for (var i = 0; i < NumberOfConcatenations; i++)
        {
            result = string.Concat(result, i);
        }
        return result;
    }

    [Benchmark]
    public string StringBuilder()
    {
        var builder = new StringBuilder();
        for (var i = 0; i < NumberOfConcatenations; i++)
        {
            builder.Append(i);
        }
        return builder.ToString();
    }

    [Benchmark]
    public string StringJoin()
    {
        var list = new List<string>();
        for (var i = 0; i < NumberOfConcatenations; i++)
        {
            list.Add(i.ToString());
        }
        return string.Join("", list);
    }
}

There, we have 4 algorithms to concatenate strings. Note that there is a [Benchmark] annotation in each of them. According to their good practices, it is recommended to use the result of the calculation to avoid dead code elimination by JIT. That is why I am using a string result type.

Finally, in our Program.cs file, we will add the following lines to run the benchmarking:

using BenchmarkDotNet.Running;
using Benchmarking;

var summary = BenchmarkRunner.Run<StringConcatenationBenchmark>();

There are more ways to run it, but the most important to make this work correctly is to build the application in Release mode and not attach the debugger during the benchmarking.

Running the benchmarking

Let's start the project and see what we get in the Console Window.

.NET SDK 8.0.100
  [Host]     : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2
  DefaultJob : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2


| Method             | Mean       | Error    | StdDev   |
|------------------- |-----------:|---------:|---------:|
| AdditionAssignment | 1,436.3 ns | 28.58 ns | 34.02 ns |
| StringConcat       | 1,678.7 ns | 32.37 ns | 44.31 ns |
| StringBuilder      |   389.3 ns |  4.66 ns |  4.36 ns |
| StringJoin         |   882.1 ns | 17.22 ns | 17.68 ns |

Alright! Sort of what we expected. The StringBuilder class was the fastest according to the statistics.

Now, it would be helpful to know how they perform in terms of memory allocation. This feature is not enabled by default, but we can do that by adding the [MemoryDiagnoser] annotation to the class.

[MemoryDiagnoser]
public class StringConcatenationBenchmark
{
  ...

Let's run this again.

| Method             | Mean       | Error    | StdDev   | Gen0   | Allocated |
|------------------- |-----------:|---------:|---------:|-------:|----------:|
| AdditionAssignment | 1,429.6 ns | 25.21 ns | 30.96 ns | 1.6613 |  20.37 KB |
| StringConcat       | 1,668.8 ns | 33.09 ns | 50.54 ns | 1.8520 |  22.71 KB |
| StringBuilder      |   383.6 ns |  3.81 ns |  3.56 ns | 0.1016 |   1.25 KB |
| StringJoin         |   889.4 ns | 14.97 ns | 16.64 ns | 0.2069 |   2.54 KB |

Great! StringBuilder also wins in GC and memory allocation.

Final thoughts

BenchmarkDotNet proves to be an invaluable tool to diagnose performance issues in our C# code from different metrics and points of view like execution time and memory allocation. The examples provided are just the tip of the iceberg; BenchmarkDotNet offers numerous features to delve deeper into performance evaluations. Take the time to explore their documentation and unlock the full potential of this powerful performance-testing tool for your C# projects.