Beyond Basics: Implementing Advanced Rate Limiting and URI Whitelisting in AWS WAF

AWS WAF is a critical component of any cloud-based security strategy, providing powerful, customizable protection for your web applications. By understanding its architecture and how it integrates with other AWS services, you can build a resilient and adaptive security framework that meets the unique needs of your business.

As web applications continue to grow in complexity and popularity, ensuring their security is a top priority. One critical aspect of web security is controlling the rate at which requests are allowed to reach your application. AWS Web Application Firewall (WAF) provides powerful tools to implement rate limiting, but sometimes the basic features aren’t enough. In this blog post, we’ll explore how to implement advanced rate limiting with custom rules and URI whitelisting in AWS WAF, complete with a step-by-step guide and practical examples.

Scenario: Protecting a Login Endpoint and Whitelisting Specific URIs

Consider a scenario where you're managing a web application that has a critical login endpoint (/login). This endpoint is a prime target for brute-force attacks and other malicious activities. To protect it, you want to:

• Implement a strict rate limit for POST requests to the /login endpoint.

• Apply a more lenient rate limit for other types of requests.

• Whitelist specific URIs, allowing them to bypass rate limits entirely.

• Exempt trusted IP addresses from all rate limiting.

This setup ensures that while your application remains accessible to legitimate users, it is protected against potential attacks.

Step 1: Creating an IP Set for Trusted IPs

The first step is to create an IP set that contains the IP addresses you trust. These addresses will be exempt from any rate limiting, ensuring smooth access for known partners, internal users, or other trusted entities.

Creating the IP Set

1. Log in to AWS Management Console: Navigate to the AWS WAF section.

2. Create IP Set: Under "IP Set," create a new set named TrustedIPs.

3. Add IP Addresses: Populate the set with the IP addresses you want to whitelist.

This IP set will be referenced in the custom rules we'll create next.

Step 2: Implementing Custom Rate Limiting Rules

Rule 1: Whitelist Specific URIs

Before applying any rate limits, let’s first whitelist certain URIs. For instance, you may want to ensure that requests to a URI like /public-api/* are always allowed without restriction.

Creating the Whitelist URI Rule

Here’s how to create a rule that bypasses rate limiting for specific URIs:

jsonCopy code{
  "Name": "WhitelistSpecificURI",
  "Priority": 1,
  "Action": {
    "Allow": {}
  },
  "Statement": {
    "ByteMatchStatement": {
      "SearchString": "/public-api/",
      "FieldToMatch": {
        "UriPath": {}
      },
      "TextTransformations": [
        {
          "Priority": 0,
          "Type": "NONE"
        }
      ],
      "PositionalConstraint": "STARTS_WITH"
    }
  },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "WhitelistSpecificURI"
  }
}

This rule allows all requests to URIs starting with /public-api/, bypassing any rate limits that follow.

Rule 2: Apply Rate Limit for Login Endpoint

Next, we’ll create a rate-based rule that specifically targets POST requests to the /login endpoint. This rule ensures that any attempt to flood the login endpoint with requests is mitigated.

Creating the Login Rate Limit Rule

{
  "Name": "RateLimitPOSTToLogin",
  "Priority": 2,
  "Action": {
    "Block": {}
  },
  "Statement": {
    "RateBasedStatement": {
      "Limit": 100,
      "AggregateKeyType": "IP",
      "ScopeDownStatement": {
        "AndStatement": {
          "Statements": [
            {
              "ByteMatchStatement": {
                "SearchString": "/login",
                "FieldToMatch": {
                  "UriPath": {}
                },
                "TextTransformations": [
                  {
                    "Priority": 0,
                    "Type": "NONE"
                  }
                ],
                "PositionalConstraint": "EXACTLY"
              }
            },
            {
              "IPSetReferenceStatement": {
                "ARN": "arn:aws:wafv2:region:account-id:global/ipset/TrustedIPs",
                "Negated": true
              }
            },
            {
              "ByteMatchStatement": {
                "SearchString": "POST",
                "FieldToMatch": {
                  "Method": {}
                },
                "TextTransformations": [
                  {
                    "Priority": 0,
                    "Type": "NONE"
                  }
                ],
                "PositionalConstraint": "EXACTLY"
              }
            }
          ]
        }
      }
    }
  },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "RateLimitPOSTToLogin"
  }
}

This rule applies a strict rate limit of 100 requests per 5 minutes for POST requests to /login, ensuring protection against brute-force attacks.

Rule 3: General Rate Limiting

Finally, we’ll set up a more lenient, general rate limit that applies to all other requests. This will prevent any single IP from overwhelming your application with traffic, while still allowing for normal usage patterns.

Creating the General Rate Limit Rule

{
  "Name": "RateLimitGeneral",
  "Priority": 3,
  "Action": {
    "Block": {}
  },
  "Statement": {
    "RateBasedStatement": {
      "Limit": 1000,
      "AggregateKeyType": "IP"
    }
  },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "RateLimitGeneral"
  }
}

This rule imposes a broader rate limit, preventing excessive requests from any single IP.

Step 3: Assembling the Web ACL

With the rules created, it’s crucial to ensure they’re applied in the correct order within your Web ACL. AWS WAF evaluates rules sequentially, so the most specific rules should come first.

Final Web ACL Structure

1. WhitelistSpecificURI: Allow traffic to the whitelisted URIs, skipping further rules.

2. RateLimitPOSTToLogin: Apply strict rate limiting to the login endpoint.

3. RateLimitGeneral: Enforce a general rate limit for all other traffic.

This structure ensures that your application is protected comprehensively, while legitimate users and specific URIs remain unaffected by the rate limiting.

Step 4: Testing the Configuration

Before deploying the rules to production, it’s essential to test them in a staging environment. Here’s how you can do it:

1. Deploy to Staging

• Apply the Web ACL: Attach your newly created Web ACL to the staging environment of your application.

2. Simulate Traffic

• Use curl: Generate traffic using a loop in curl to simulate both legitimate and abusive requests.

    for i in {1..150}; do
       curl -X POST "https://your-staging-url.com/login"
    done
  

• Verify Whitelisting: Test the whitelisted URIs by sending requests and ensuring they are not blocked.

3. Monitor the Results

• Check CloudWatch Metrics: Review the metrics for each rule to see how often they’re triggered.

• Examine WAF Logs: Analyze WAF logs to confirm that requests are being handled as expected.

If everything works as intended, you can confidently deploy the configuration to your production environment.

Conclusion: Leveraging AWS WAF for Enhanced Security

AWS WAF offers powerful tools to protect your web applications, but to truly harness its potential, you need to move beyond basic configurations. By implementing advanced rate limiting with custom rules, URI whitelisting, and IP set exemptions, you can create a security setup that is both robust and tailored to your specific needs.

This approach ensures that your critical endpoints, like login pages, are well-protected against abuse, while trusted users and specific URIs remain unaffected. By following the steps outlined in this post, you can enhance the security of your application and maintain a smooth, uninterrupted user experience.