Spring Microservices in Action

In distributed system architecture, critical behaviors like security, logging and tracking users are necessities to ensure. Implementing these functionalities to individual services by a common library, etc, has these side effects.

It can be easily forgotten to implement in a fast development cycle.
Delegating the responsibilities of implementing cross-cutting concerns such as security and logging to the individual development teams increases the risk of improper implementation.
It can lead to creating a tight coupling across all our services. The harder it is to modify or add behavior in our shared code without needing to recompile and redeploy all our services.

The API gateway is handy for solving these problems and more, such as simplifying client interaction, load balancing, routing, rate limiting, and throttling. It acts as a filter and router for all the microservices calls in our architecture.

8.1 What is a service gateway?

A service gateway acts as an intermediary between the client and the invoked service. The client interacts only with the service gateway, and the service gateway parses the path coming from the service client and delegates the request to a corresponding service. Additionally, the service gateway can implement cross-cutting service concerns without needing individual team development. List of cross-cutting concerns that can be implemented in a service gateway:

Dynamic routing: incoming requests are inspected by a service gateway, and then intelligent routing is performed for the service caller.
Authentication and Authorization: Since all service calls are routed through a service gateway, it serves as a logical point to verify the authentication of service callers.
Metric collection and logging: A service gateway centralized the collection of metrics and logs as service calls pass through it, ensuring uniform logging and verifying critical information in user requests. While metrics can still be gathered from individual services, the gateway provides a centralized point for basic metrics such as service invocation counts and response times.

Does a service gateway lead to a single point of failure and a potential bottleneck?

Centralized points like routers and load balancers can be a single point of failure or bottleneck for your system. Therefore, an inappropriately implemented service gateway carries those risks.
The mentioned matters below should be kept in mind while implementing a service gateway:

Installing a load balancer in front of multiple service gateway instances is a good design practice, ensuring that your service gateway implementation can scale as needed.
Ensure that any code you write for a service gateway is stateless. Avoid storing any information in a service gateway, as this can limit the gateway's scalability. Otherwise, you must ensure the data gets replicated across all service gateway instances.
A service gateway should consist of light, only needed code. It can be considered a "chokepoint" in our system. Complex business and operations will negatively affect its performance.

8.2 Introducing Spring Cloud Gateway

Spring Cloud Gateway is a non-blocking gateway implementation built within the Spring ecosystem. In non-blocking applications, the main threads are never blocked and are continuously available to handle requests and process them asynchronously in the background to return a response once processing is done.
These are the spring cloud gateway capabilities:

Define multiple route entry points, providing extremely fine-grained route mapping (each service endpoint gets its own route mapping). However, the most common use case is defining a single entry point through which all service client calls flow.
Filters can be created to inspect and modify the requests and responses passing through the gateway. These filters let us inject policy enforcement points into our code and perform a wide range of actions to be consistently applied to all service calls. In a nutshell, these filters allow us to alter incoming and outgoing HTTP requests and responses.
Predicates are objects that allow us to verify whether a request fulfills a set of given conditions before executing or processing it. Spring Cloud Gateway provides a set of built-in Route Predicate Factories for this purpose.

8.2.1 Setting up the Spring Boot Gateway Project
Spring Cloud Gateway can be integrated with the Netflix Eureka Discovery service. To achieve this integration, you need to add the Eureka configuration to the gateway service's configuration file and annotate the API gateway application with @EnableEurekaClient to register it as a client with the Eureka server.

8.3 Configuring Routes in Spring Cloud Gateway

In a simple sense, the Spring Cloud Gateway is a reverse proxy, an intermediary server that stands between the client and the resource it seeks. The client remains unaware even while interacting with a server. The reverse proxy intercepts the client's request and then calls the remote sources on the client's behalf.

In the microservice architecture, Spring Cloud Gateway takes a microservice call from a client and routes it to the corresponding service. The service client is only interested in communicating with the gateway.

However, the gateway needs to know how to route incoming requests to the upstream services. The Spring Cloud Gateway has two mechanisms for performing it.

Automated mapping of routes using service discovery
Manual mapping of routes using service discovery.

8.3.1 Automated mapping of routes via service discovery

All gateway route mappings are defined by specifying routes. Spring Cloud Gateway can automatically route requests using their service IDs by adding specific configurations to the gateway-server configuration file, as demonstrated in the following example.

spring:
  cloud:
    gateway:
        discovery.locator:              
          enabled: true
          lowerCaseServiceId: true

Configuring the discovery locator in the gateway's configuration file allows Spring Cloud Gateway to automatically map the Eureka service ID of the requested service to a corresponding downstream service instance. In this way, we can make calls through a single endpoint, as in the example below.

http://api.example.com:8072/order-service/v1/order/958aa1bf-18dc-405c-b84a-b69f04d98d4f

Another critical advantage is that we can add and remove instances of a service without modifying the gateway.

8.3.2 Manually mapping routes using service discovery

Instead of relying entirely on the automated routes created with the Eureka service ID, Spring Cloud Gateway allows you to define route mappings explicitly. This approach provides a structured and controlled way to manage service interactions.

  cloud:
    gateway:
      routes:
        - id: ORDER-SERVICE
          uri: lb://ORDER-SERVICE
          predicates:
            - Path= /order/**
          filters:
            - name: CircuitBreaker
              args:
                name: ORDER-SERVICE
                fallbackuri: forward:/orderServiceFallBack
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 1
                redis-rate-limiter.burstCapacity: 1

        - id: PAYMENT-SERVICE
          uri: lb://PAYMENT-SERVICE
          predicates:
            - Path= /payment/**
          filters:
            - name: CircuitBreaker
              args:
                name: PAYMENT-SERVICE
                fallbackuri: forward:/paymentServiceFallBack
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 1
                redis-rate-limiter.burstCapacity: 1

8.3.2 Dynamically reloading route configuration

Another important feature of Spring Cloud Gateway is its ability to dynamically reload routes without needing to restart the Gateway Servers. You only need to make the changes to the configuration file and commit those changes to the source where the gateway pulls its configuration data. Then, you can trigger the update by reaching the POST-based endpoint, actuator/gateway/refresh.

8.4 The Real Power of Spring Cloud Gateway: Predicate and Filter Factories

One of the greatest advantages of Spring Gateways is its ability to implement custom logic that applies to all service calls passing through it. This custom logic is typically used to enforce consistent application policies, such as security, logging, and tracking, across all services.

These Spring Cloud Gateway predicate and filter factories match or intercept a wide range of behaviors and decorate or change the call's behavior. While a servlet filter and Spring aspect are localized to a specific service, using the Gateway along with its Predicate and Filter factories enables us to implement cross-cutting concerns across all the services routed through the gateway.

Predicates let us check if the request meets a set of certain conditions before processing the request, while Filters enable the modification of incoming requests or outgoing responses. This method allows us to manipulate the request before it reaches the destination or the response before it is sent back to the client.

  cloud:
    gateway:
      routes:
        - id: EXAMPLE-SERVICE
          uri: lb://EXAMPLE-SERVICE
          predicates:
            - Path= /api/**
          filters:
            - AddRequestHeader=X-Custom-Header, Value
            - RewritePath=/api/(?<segment>.*), /new-api/${segment}

In the example above, a predicate is used to match all requests to /api/**, and then filters are used to add a customer header and rewrite the request path. This is one of the demonstrations of the capability to match a wide range of behaviors and then change the call's behavior.

The next sections, Built-in Predicate and Built-in Filter Factories are not deeply examined; there are many handy documents out there that can help you find specific use cases.

8.4.3 Custom filters
Spring Cloud Gateway enables us to create custom logic using filters within the gateway. Filters implement a chain of business logic that each service request goes through. Spring Cloud Gateway supports two types of filters listed below:

Pre-filters: Pre-filters are invoked before the actual request is sent to the target destination. Pre-filters ensure a consistent message format (such as key HTTP headers) or act as gatekeepers to make sure that the user calling the service is authenticated.
Post-filters: Post-filters are invoked after the target service has processed the request and sent a response back to the client. Post-filters are usually used to log the response, handle errors, or audit the response for sensitive information.

References

[1] Spring Microservices in Action, 2nd Edition, John Carnell.
https://www.manning.com/books/spring-microservices-in-action-second-edition

[2] "Spring Cloud Gateway" on Spring.io
https://docs.spring.io/spring-cloud-gateway/docs/current/reference/html/

[3] Spring Cloud API Gateway with Filter, Security and Circuit Breaker, AMIT DATTA
https://m edium.com/@the.amisoft/spring-cloud-api-gateway-with-filter-c85a6f60685e

Notes of Spring Microservices in Action (Chapter 8)