Enhancing API Security with Middleware: Filtering and Masking Requests in .NET Core

Middleware is a key part of how modern web apps are built, working as a middle layer that handles requests and responses. In API systems, middleware helps make things safer, work better, and follow rules about protecting data.

One big problem developers often face is dealing with private information that moves through API requests. Requests not appropriately checked can create weak spots that let bad people see or take private information. To fix this, teams can use middleware to check, clean, and hide private information before it gets to the main program.

This article will show you how to build middleware in .NET Core that makes APIs safer by cleaning and hiding private data. You will learn how to catch and change incoming requests, work with settings that can change, and make things run fast while letting your program grow bigger. After reading this guide, you will know how to make a strong, safe middleware layer for your APIs.

Middleware in .NET Core and How It Handles API Requests

Middleware is a key part of the Asp.Net Core system, working as the main piece of how requests and responses flow. In a web app, middleware parts handle HTTP requests as they move through this flow, letting developers add special tasks at different steps. These parts can change requests, add safety checks, save records of what happens, or fix errors, making them very important for building strong APIs.

Middleware works between when a client asks for something and when an API gives an answer. In Asp.Net Core, middleware is a small, separate part that can either work with requests and send them to the next part in the line or stop and make an answer. Middleware is set up in the Program.cs or Startup.cs file, letting developers make a custom and growing request line.

You need to use Startup.cs if your project is targeting

1. Asp.Net Core 1.x

2. Asp.Net Core 2.x

3. Asp.Net Core 3.x

4. .Net 5

You need to use Program.cs if your project is targeting

1. .Net 6

2. .Net 7

3. .Net 8

The Asp.Net Core middleware line works with requests one after another, ensuring each middleware part runs in the order it was put in. This setup lets developers make middleware for many uses, including:

• Checking who someone is and what they can do.

• Writing down what happens and watching how things work.

• Dealing with problems when they happen.

• Changing and checking data.

APIs often handle private data, like personal, money, and login details. Showing this information without protection can cause safety problems, like data theft and people getting in who should not. Middleware helps fix this by letting developers filter and hide private data before it gets to the main Program.

Filtering requests means looking at what is in them to find and remove harmful or unwanted parts. Hiding data means replacing private information with fake values so it is not shown during work or record-keeping. Together, these make API work safer and more trustworthy.

Main Things About Middleware for Filtering and Hiding

Using middleware for filtering and hiding requests in .NET Core gives several good things:

1. Better Safety: Middleware can stop private data from being shown in records, answers, or other systems, making data theft less likely.

2. Following Rules: Middleware can help APIs follow data protection laws like GDPR or HIPAA by keeping private information safe.

3. Working Better: By filtering out wrong or bad requests, middleware makes program errors less likely and makes the whole system work better.

4. Easy to Change and Grow: Middleware parts are easy to make, test, and update, letting developers fix new safety needs without significant program changes.

5. Exact Data Handling: Middleware ensures private data is handled the same way across all API endpoints, making fewer mistakes and differences.

Before we start making middleware in .NET Core to check and hide parts of API requests, you need to have the right tools and basic understanding. Here's what you need:

1. You will need the Visual Studio code

2. You will need a .NET framework

3. You will need the postman for testing

!! Important Note !!

If you have all the tools and setup ready, you can skip the installation steps. Go straight to the Middleware Design and Workflow section to learn how to make it work and how middleware handles requests and responses.

Setting up Visual Studio Code

Visual Studio Code (VS Code) is a small, free code-writing tool made by Microsoft. Many developers use it because it is flexible, fast, and works with many coding languages. VS Code runs well on Windows, macOS, and Linux.

Key features of Visual Studio Code include:

Extensibility: A considerable store of add-ons for languages, frameworks, and tools.

Built-in Git Integration: This makes working together and tracking changes easier.

Intelligent Code Editing: Gives IntelliSense for competent code help, color-coded text, and problem-finding.

Customizability: This lets you change looks, keyboard shortcuts, and workspace settings to match what you like.

Visual Studio Code is a small editor with strong abilities and has become popular with developers who make different things, from websites to cloud programs.

Type Visual Studio code download into Google and click the first link.

On the next page, you must pick which matches your computer type. I will pick Mac and get it.

You will find the file in your downloads folder when it finishes downloading. You need to open it by clicking on the zip file.

Setting up .Net Framework

Microsoft's .NET Framework is a strong and complete software building system, mainly for creating and running Windows programs. It gives a controlled setting with many valuable tools and libraries, letting developers make different programs, including desktop, web, and server programs.

Key components of the .NET Framework include:

1. Common Language Runtime (CLR): The main engine that controls program running, memory, cleaning unused data, and safety.

2. Base Class Library (BCL): A set of ready-to-use tools and libraries for everyday tasks like working with files, databases, and XML.

3. Language Support: Works with many coding languages, like C#, VB.NET, and F#, making developers' choices easier.

In 2002, the .NET Framework changed Windows program building by making things like memory control and safety setup easier. Over time, newer systems like .NET Core and .NET 5/6/7/8/9 have come after it, focusing on better speed, growth ability, and support for Windows, macOS, and Linux. Even with these changes, the .NET Framework remains important for keeping old Windows programs working and ensuring older software runs well and stays stable.

Go to Install .NET on macOS Microsoft page and click on Download .Net

On the next page, let’s download .NET 8.0 (Long Term Support)

After selecting .NET 8.0, you must choose an option that matches your computer type. I will pick Arm64 because I have an M chip.

If you have an M chip computer, you should download Arm64; if not, download x64 instead.

When the download finishes correctly, you will find the package in your download folder

Now install the .NET framework. Open the package to begin installing and select Continue

On the next screen, you need to select Install

If you want to put it somewhere else, you can select Change Install Location.

When the installation finishes appropriately, you will see a screen and can end it by selecting Close

If everything works right, we can check the version in the terminal. Open a terminal and type dotnet --version.

Setting up Postman

Postman is a well-liked API-making and testing tool that makes building, testing, and handling APIs easier. It gives developers a simple way to work with APIs without writing complex code. You can get it as a computer program or use it in a web browser, which works on Windows, macOS, and Linux.

Key features of Postman include:

API Testing lets users send requests and look at answers in different forms like JSON and XML.

Automation: Works with JavaScript to write scripts and run tests by itself.

Collaboration: Let teams share their work, settings, and written guides.

Mock Servers: Creates fake API endpoints for building and testing.

With its many useful features and simple design, Postman has become an essential tool for making, fixing, and connecting APIs in many different types of work.

When we finish coding, we need to check our work, so we must download Postman. Pick the download option that matches your computer's chip.

Once the download finishes, you will find the zip file in your downloads folder

Click the zip file to start Postman

If you get asked Move to Application Folder, go ahead and move it

If everything works right, you will see this screen. We will not make an account now, so choose Continue without an account and keep going

On the next screen, select Open Lightweight API Client

If you did each step right, you should now see this form

Middleware Design and Workflow

Middleware in .NET Core helps manage API requests, letting developers check and change requests and responses at different points. When making middleware to filter and hide sensitive data, you need to plan what it will do, how it fits in, and how it handles requests.

In ASP.NET Core, middleware pieces work one after another to handle web requests coming in and going out. Middleware does two main things:

Intercepting Requests: Middleware grabs and works on requests as they come in. It can change the request, pass it on, or stop it.

Processing Responses: Middleware can also handle responses, letting developers change them before sending them back. Middleware that filters and hides sensitive data mainly focuses on catching and changing requests before they reach the main program. This keeps sensitive info safe as early as possible.

The middleware follows clear steps to handle requests well:

Intercepting Requests

The middleware catches every web request that comes in. At this point:

1. The HttpContext object lets you see request details, like headers, query strings, and what is in the request.

2. The middleware checks if the request has content that needs looking at (like POST or PUT requests with JSON data)

Filtering Requests

The middleware looks at the request content for sensitive data. This means:

1. Breaking down the request content (like JSON) into a format it can check.

2. Looking for sensitive info by comparing what is there with a list of words, patterns, or data types.

Masking or Deleting Data

If it finds sensitive data, the middleware hides it by putting placeholder text (like *****) instead. Alternatively, the middleware can remove the sensitive parts altogether.

{
  "username": "firat",
  "password": "verysecret",
  "ssn": "123456789"
}

The above JSON would be modified to

{
  "username": "firat",
  "password": "*****",
  "ssn": "*****"
}

After filtering and hiding, the middleware makes a new changed request:

1. The updated content is turned back into JSON.

2. The old request content has been replaced with new content.

3. The request moves on to the next middleware or handler. This ensures everything downstream gets a clean request, stopping any chance of showing sensitive data by accident.

Creating a New ASP.NET Core Web API Project

We are making an API project with middleware to change requests and responses.

Start by opening a terminal and run dotnet new webapi -n MiddlewareProcess

• dotnet new webapi is used to create a Web API project template

  -n <ProjectName> is used to give a name for the project

Execute code . to open the project in Visual Studio Code

When it is done correctly, you will see several project files

To test if it works, execute dotnet run and look for the port number your computer shows

You can visit http://localhost:5256/swagger/index.html in your browser

In Program.cs, you will find code for a Minimal API. This is something different we might learn later. You can delete it since we will not use Minimal API

app.MapGet("/weatherforecast", () =>
{
    var forecast =  Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast
        (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            summaries[Random.Shared.Next(summaries.Length)]
        ))
        .ToArray();
    return forecast;
})

First, install the C# extension

After installing C#, make a new folder called Controller, then make a new C# file with any name you choose

Now, implement the API code in your new file

using Microsoft.AspNetCore.Mvc;

namespace MiddlewareExample.Controllers
{
    [ApiController]
    [Route("api/[controller]")]
    public class MiddlewareController : ControllerBase
    {
        [HttpGet]
        public IActionResult GetText()
        {
            return Ok("test");
        }
    }
}

[ApiController] indicates that the class is a controller and enables automatic binding and validation.

[Route("api/[controller]")] sets the web address to api/middleware (using the controller's name).

[HttpGet] specifies that this action method responds to HTTP GET requests.

GetText() is what we call this piece of code, which is not part of the web address itself, and it sends back the word test with a message 200 OK saying everything worked.

return Ok("test"); returns the string "test" in an OkObjectResult to indicate a successful response.

You also need to change Program.cs to make your controller work. Go to Program.cs and write builder.Services.AddControllers(); and app.MapControllers();

Program.cs should match what is shown below

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Services.AddControllers(); // Add this line 

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();
app.MapControllers(); // add this line

app.Run();

Execute dotnet run after setting everything up and using Postman to send a request to http://localhost:{your port}/api/middleware

Now we know our controller and API work correctly. We need to make a list of words that we want to hide or remove. This list will help us filter content.

Make a new folder called Model, and inside it, make a new C# file called SensitiveData.cs.

Write your code precisely like what's shown below

public static class SensitiveData
{
    public static readonly List<string> Keywords = new()
    { 
        "password", 
        "ssn", 
        "creditCard", 
        "accountnumber"
    };
}

We use a static class because it holds information everyone shares, and we will not need to start the class.

Readonly makes sure the list cannot be changed.

Now, let us make our middleware class. Create a folder called Middleware and put a new C# file called FilteringMiddleware.cs in it

Let us begin writing our middleware class. We will start by adding the namespaces we need

using System.Text.Json;
using System.Text;

System.Text.Json has classes and tools for JSON data. It helps turn JSON data into C# objects we can use.

System.Text lets us read and write data streams. We need this to handle data coming into our program.

The next function will be RequestDelegate.

private readonly RequestDelegate _next;

The line private readonly RequestDelegate _next; is very important because it helps pass web requests to the next part of your ASP.NET Core program.

In ASP.NET Core, middleware uses a pattern called chain of responsibility. This means each request goes through a line of helpers, and each helper can either work on it or pass it along.

RequestDelegate is like a pointer showing which middleware is next in the request line.

We need two settings to help us hide or remove private information.

private readonly bool _maskSensitiveData;  
private readonly bool _removeSensitiveFields;

We'll make a starter (constructor) function for our middleware.

public RequestFilteringMiddleware(RequestDelegate next, bool maskSensitiveData = true, bool removeSensitiveFields = false)
{
   _next = next;
   _maskSensitiveData = maskSensitiveData;
   _removeSensitiveFields = removeSensitiveFields;
}

By default, maskSensitiveData will be set to true and removeSensitiveFields will be set to false.

RequestFilteringMiddleware starts up (constructor) when we make new middleware. ASP.NET Core calls it when we add app.UseMiddleware<RequestFilteringMiddleware>() to our program.

RequestDelegate next shows what comes next in line. ASP.NET Core gives us this information when it makes the middleware.

_next = next; saves the next step so we can use it later with _next(context).

Finally, we will make the core part that does the work. This runs every time a web request comes in.

public async Task InvokeAsync(HttpContext context)

HttpContext context represents all HTTP-specific information about the current request and response.

We will put our logic work into the InvokeAsync method to clean up or hide data. Here's how it works.

public async Task InvokeAsync(HttpContext context)
{
    if (context.Request.Method == HttpMethods.Post)
    {
        context.Request.EnableBuffering(); 

        using (var reader = new StreamReader(context.Request.Body, Encoding.UTF8, leaveOpen: true))
        {
            var body = await reader.ReadToEndAsync();
            context.Request.Body.Position = 0; 

            if (!string.IsNullOrWhiteSpace(body))
            {
                using (var jsonDocument = JsonDocument.Parse(body))
                {
                    var rootElement = jsonDocument.RootElement.Clone();
                    var filteredJson = ProcessJsonElement(rootElement);

                    var modifiedBody = JsonSerializer.Serialize(filteredJson);
                    var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(modifiedBody));
                    context.Request.Body = memoryStream;
                    context.Request.Body.Position = 0; 
                }
            }
        }
    }

    await _next(context);
}

Let’s look at each step

if (context.Request.Method == HttpMethods.Post)

This part makes sure we only work on POST requests.

context.Request.EnableBuffering();

This step is important because, normally, you can only read a request once in ASP.NET Core. EnableBuffering() let’s read it many times.

using (var reader = new StreamReader(context.Request.Body, Encoding.UTF8, leaveOpen: true))
{
    var body = await reader.ReadToEndAsync();
    context.Request.Body.Position = 0;
}

StreamReader helps us read the requested data as text.

await reader.ReadToEndAsync() gets all the data at once (asynchronously).

leaveOpen: true keeps the data available for others to use later.

context.Request.Body.Position = 0; goes back to the start so others can read the data, too.

using (var jsonDocument = JsonDocument.Parse(body))
{
    var rootElement = jsonDocument.RootElement.Clone();
}

JsonDocument.Parse(body) turns the text into a format we can work with.

RootElement.Clone() makes a copy we can change.

var filteredJson = ProcessJsonElement(rootElement);

ProcessJsonElement looks through the data and changes sensitive fields based on our rules.

var modifiedBody = JsonSerializer.Serialize(filteredJson);

It turns our changed data back into text using JsonSerializer.Serialize.

var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(modifiedBody));
context.Request.Body = memoryStream;
context.Request.Body.Position = 0;

MemoryStream puts our changed data in place of the original.

context.Request.Body.Position = 0 resets the stream position so endpoints or downstream middleware can read the processed body.

await _next(context);

It ensures the request continues to be processed after the middleware has completed its logic.

Now, we will make the ProcessJsonElement function. Here is how it looks.

private object ProcessJsonElement(JsonElement element)
{
    switch (element.ValueKind)
    {
        case JsonValueKind.Object:
            var processedObject = new Dictionary<string, object>();
            foreach (var property in element.EnumerateObject())
            {
                if (SensitiveData.Keywords.Contains(property.Name))
                {
                    if (_maskSensitiveData)
                    {
                        processedObject[property.Name] = "*****"; // Mask the sensitive value
                    }
                    else if (!_removeSensitiveFields)
                    {
                        processedObject[property.Name] = null; // Set sensitive value to null
                    }
                }
                else
                {
                    processedObject[property.Name] = ProcessJsonElement(property.Value);
                }
            }
            return processedObject;

        case JsonValueKind.Array:
            var processedArray = new List<object>();
            foreach (var item in element.EnumerateArray())
            {
                processedArray.Add(ProcessJsonElement(item));
            }
            return processedArray;

        case JsonValueKind.String:
            return element.GetString();

        case JsonValueKind.Number:
            return element.GetDouble();

        case JsonValueKind.True:
        case JsonValueKind.False:
            return element.GetBoolean();

        case JsonValueKind.Null:
        default:
            return null;
    }
}

Let’s investigate our code and understand the steps

case JsonValueKind.Object:
    var processedObject = new Dictionary<string, object>();
    foreach (var property in element.EnumerateObject())
    {
        if (SensitiveData.Keywords.Contains(property.Name))
        {
            if (_maskSensitiveData)
            {
                processedObject[property.Name] = "*****"; 
            }
            else if (!_removeSensitiveFields)
            {
                processedObject[property.Name] = null; 
            }
        }
        else
        {
            processedObject[property.Name] = ProcessJsonElement(property.Value);
        }
    }
return processedObject;

JSON can have layers inside layers, so we check every level for sensitive information.

Dictionary<string,object> is used to store the processed object.

EnumerateObject helps us look at property.Name and property.Value

SensitiveData.Keywords.Contains(property.Name) checks for sensitive

• Changes values to "*****" if _maskSensitiveData is true

• Makes values null if _maskSensitiveData is false and _removeSensitiveFields is false

• Takes out the whole piece if _removeSensitiveFields is true

return processedObject; gives back the processed data.

case JsonValueKind.Array:
 var processedArray = new List<object>();
 foreach (var item in element.EnumerateArray())
 {
   processedArray.Add(ProcessJsonElement(item));
 }
return processedArray;

JSON arrays can have nested objects inside, so we check everything.

case JsonValueKind.String:
return element.GetString();

JsonValueKind.String keeps primitive string values correctly included in the modified JSON without modification.

case JsonValueKind.Number:
return element.GetDouble();

JsonValueKind.Number keeps numerical values preserved in the modified JSON.

case JsonValueKind.True:
case JsonValueKind.False:
return element.GetBoolean();

JsonValueKind.True and JsonValueKind.False keep boolean values preserved in the modified JSON.

case JsonValueKind.Null:
default:
    return null;

JsonValueKind.Null handles empty values correctly.

We're done implementing our middleware. Now add app.UseMiddleware<RequestFilteringMiddleware>(false, true); to Program.cs. Your Program.cs should look like this.

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Services.AddControllers();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseMiddleware<RequestFilteringMiddleware>(false, true);

app.MapControllers();

app.Run();

One more thing - we need to update our controller to send back what it gets. It should look like this.

using Microsoft.AspNetCore.Mvc;

namespace MiddlewareExample.Controllers
{
    [ApiController]
    [Route("api/[controller]")]
    public class RequestController : ControllerBase
    {
        [HttpPost]
        public IActionResult ReturnResponse([FromBody] Dictionary<string, object> requestData)
        {
            return Ok(new
            {
                Data = requestData
            });
        }
    }
}

We use Dictionary<string,object> because JSON data is structured as key-value pairs

Time to test our work. Here's what to do:

1. Run dotnet run

2. 

   Use Postman to make a request to http://localhost:{your port}/api/middleware

You can set your port in launchSettings.json

3. Click Body, pick raw then JSON. Use this test data.

{
    "data": {
        "user": {
            "name": {
                "first": "Firat",
                "last": "Tonak"
            },
            "email": "firattonak.com@firattonak.com",
            "ssn": "123-45-6789"
        },
        "order": {
            "paymentMethod": {
                "card": {
                    "accountnumber": "1111 222 3333 1111"
                }
            }
        }
    }
}

Let's start testing. First, set maskSensitiveData to true and removeSensitiveData to false in Program.cs

app.UseMiddleware<RequestFilteringMiddleware>(true, false);

After updating Program.cs, run dotnet run and try it. You should see this.

{
    "data": {
        "data": {
            "user": {
                "name": {
                    "first": "Firat",
                    "last": "Tonak"
                },
                "email": "firattonak.com@firattonak.com",
                "ssn": "*****"
            },
            "order": {
                "paymentMethod": {
                    "card": {
                        "accountnumber": "*****"
                    }
                }
            }
        }
    }
}

See how sensitive information is now masked.

Next, let's try removing sensitive information. Change Program.cs to this.

app.UseMiddleware<RequestFilteringMiddleware>(false, true);

Send another request, and you should see this.

{
    "data": {
        "data": {
            "user": {
                "name": {
                    "first": "Firat",
                    "last": "Tonak"
                },
                "email": "firattonak.com@firattonak.com"
            },
            "order": {
                "paymentMethod": {
                    "card": {}
                }
            }
        }
    }
}

Notice how sensitive information is now gone from the response.

The logic will work no matter how complex your response is.

Middleware is critical in today's API development, working as the primary system for handling and changing requests before they get to the main app code. When we use middleware to check and hide parts of requests, we fix two big problems in API safety: protecting private information and following data privacy rules like GDPR, HIPAA, and PCI-DSS. This helps developers reduce risks early, making people trust the systems they create.

Middleware for checking and hiding requests is a tool and a foundation for safe and strong API systems. Using this method creates a base for protecting data, following rules, and being reliable. However, it can do even more than that. By making innovative additions like recording hidden request data, adding tools to understand usage, or limiting how many requests can come in to stop misuse, this middleware can grow into a complete answer for what modern APIs need.

APIs are not fixed things; they get bigger, change, and face new problems in our always-changing digital world. Middleware becomes the quiet helper in this process, letting APIs change easily for new needs while staying safe and fast.

In the end, middleware is more than just computer code; it's a way of thinking about change, a promise to keep things safe, and a path to new ideas in API development.

You can access the code here