Microservices simple E2E Testing Guide

Hi. This is the sixth part of the diary of developing the “Programmers’ diary” blog. The open source code of this project is on https://github.com/TheProgrammersDiary. The fifth part: https://medium.com/@vievaldas/implementing-ci-cd-and-splitting-monolith-to-two-microservices-c4e7b006a21b.

-————

2023-09-03

Today’s goal is to create and launch first end to end test.

I have created a new Maven project for E2E (end-to-end) testing.

It’s located in docker folder: docke/GlobalTests/pom.xml.

It contains this test class:

@Testcontainers
public class PostCommentTest {

    @Container
    private static final DockerComposeContainer<?> dockerComposeContainer =
            new DockerComposeContainer<>(
                    new File("src/test/resources/docker-compose-test.yaml")
    )
                    .withExposedService("postgres", 5432)
                    .withExposedService("mongodb", 27017)
                    .withExposedService("blog", 8080)
                    .withExposedService("post", 8080);

    @Test
    public void serviceResponds() {
        get("<http://localhost:8080/comments/list-comments/1>")
                .then()
                .assertThat()
                .statusCode(200)
                .body("", Matchers.hasSize(0));
    }
}

There are lots of ways to write an E2E test. I am using testcontainers’ DockerComposeContainer, however you could use separate containers (2 GenericContainer for our services, MongoDbContainer and PostgreSQLContainer). Or you could take an alterily different approach: write a Selenium app, launch containers and test them with Selenium.

Continuing with selected approach:

DockerComposeContainer approach requires the docker file. Here it is:

version: "3.8"
services:
    blog:
        image: docker-blog
        ports:
            - "8080:8080"
        depends_on:
            - mongodb
        environment:
            SPRING_PROFILES_ACTIVE: docker
    post:
        image: docker-post
        ports:
            - "8081:8080"
        depends_on:
            - postgres
        environment:
            SPRING_PROFILES_ACTIVE: docker
    postgres:
        image: postgres:15.4
        ports:
            - "5432:5432"
        environment:
            POSTGRES_USER: posts
            POSTGRES_PASSWORD: admin
    mongodb:
        image: mongo:7.0
        ports:
            - "27017:27017"
        environment:
            MONGO_INITDB_DATABASE: blog

There are few things different from docker-compose.yaml which we use in local (non-test) build:

There is no pgadmin - since it’s an automated test, we won’t be needing it.
There is no container_name property - testcontainers do not allow to set container names and instead create containers with random names. Ugly names I would say, but these tests run and stop, so we should not bother.
Version is “3.8” - I have to confess that I have not fully understood the properties I used. I thought that version is my naming, e.g. then the file is modified you can increase the version. Actually, higher versions provide more docker compose functionality and are documented here: https://docs.docker.com/compose/compose-file/compose-versioning/.
Ports are quoted - https://www.jetbrains.com/help/phpstorm/docker-compose-unquoted-port-mappings.html.

Returning to the test method:

    @Test
    public void serviceResponds() {
        get("<http://localhost:8080/comments/list-comments/1>")
                .then()
                .assertThat()
                .statusCode(200)
                .body("", Matchers.hasSize(0));
    }

It’s a simple integration test to check that our monolith responds. It uses REST Assured library. The test is working. I can now copy the CommentRequest.proto and PostRequest.proto and copy pom.xml config to auto-generate them https://github.com/TheProgrammersDiary/Docker/commit/a72ded89e83de8459a02e1aafe4e3039daa1d9f4. Having the test done, we can now write first E2E test:

@Testcontainers
@ExtendWith({SnapshotExtension.class})
public class PostCommentTest {

    private Expect expect;

    @Container
    private static final DockerComposeContainer<?> dockerComposeContainer =
            new DockerComposeContainer<>(
                    new File("src/test/resources/docker-compose-test.yaml")
            )
                    .withExposedService("postgres", 5432)
                    .withExposedService("mongodb", 27017)
                    .withExposedService("blog", 8080)
                    .withExposedService("post", 8080);

    @Test
    @SnapshotName("createsPostWithComments")
    public void createsPostWithComments() throws IOException {
        PostRequest postRequest = PostRequest
                .newBuilder()
                .setAuthor("Human")
                .setTitle("Testing matters")
                .setContent("You either test first, test along coding, or don't test at all.")
                .build();

        String postId = given()
                .baseUri("<http://localhost:8081>")
                .body(postRequest.toString())
                .when()
                .post("/posts/create")
                .getBody()
                .jsonPath()
                .get("id")
                .toString();
        int commentCount = 2;
        String[] commentIds = new String[commentCount];
        for(int i = 0; i < commentCount; i++) {
            commentIds[i] = given()
                    .baseUri("<http://localhost:8080>")
                    .body(
                            CommentRequest
                                    .newBuilder()
                                    .setAuthor("author" + i)
                                    .setContent("content" + i)
                                    .setPostId(postId)
                            .build().toString()
                    )
                    .post("/comments/create")
                    .getBody()
                    .jsonPath()
                    .get("id");
        }
        ArrayNode comments = (ArrayNode) new ObjectMapper().readTree(
                get("<http://localhost:8080/comments/list-comments/>" + postId)
                        .getBody()
                        .asString()
        );
        assertThat(comments.size()).isEqualTo(commentCount);
        for(int i = 0; i < comments.size(); i++) {
            assertThat(comments.get(i).get("id").textValue()).isEqualTo(commentIds[i]);
        }
        maskProperties(comments, "id", "postEntryId");
        expect.toMatchSnapshot(comments.toString());
    }

    private void maskProperties(ArrayNode node, String... properties) {
        node.forEach(element ->
                Arrays
                        .stream(properties)
                        .forEach(property -> ((ObjectNode) element).put(property, "#hidden#"))
        );
    }
}

It’s the same test as our first integration test was (in the monolith app), however, since we don’t have Post and Comment classes: json is used instead, so we need to update pom.xml:

<dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.15.2</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-core</artifactId>
      <version>2.15.2</version>
    </dependency>

In monolith and Post microservice Spring dependencies have jackson as a transient dependency, however since GlobalTests testing services does not have Spring, jackson needs to be added manually.

Also, to run these tests test/resources/snapshot.properties config is needed:

serializer=au.com.origin.snapshots.jackson.serializers.v1.JacksonSnapshotSerializer
serializer.string=au.com.origin.snapshots.serializers.v1.ToStringSnapshotSerializer
serializer.base64=au.com.origin.snapshots.serializers.v1.Base64SnapshotSerializer
serializer.json=au.com.origin.snapshots.jackson.serializers.v1.JacksonSnapshotSerializer
serializer.orderedJson=au.com.origin.snapshots.jackson.serializers.v1.DeterministicJacksonSnapshotSerializer
comparator=au.com.origin.snapshots.comparators.v1.PlainTextEqualsComparator
reporters=au.com.origin.snapshots.reporters.v1.PlainTextSnapshotReporter
snapshot-dir=__snapshots__
output-dir=src/test/java
ci-env-var=CI
update-snapshot=none

However, I may be considering to change the snapshot library. I wanted to create snapshot of well-formed json, however during a second run, a weird bug occurred: test fails, however snapshot debug file is not generated. Reruning results in the same issue. Not sure why.

So, for now, I am generating snapshots in raw String which is very uneasy to read:

createsPostWithComments=[
  "[{\\"author\\":\\"author0\\",\\"content\\":\\"content0\\",\\"id\\":\\"#hidden#\\",\\"postEntryId\\":\\"#hidden#\\"},{\\"author\\":\\"author1\\",\\"content\\":\\"content1\\",\\"id\\":\\"#hidden#\\",\\"postEntryId\\":\\"#hidden#\\"}]"
]

This, at least, works.

[Json snapshot is useful when you want to assert complex object. Instead of many assert statement, json is formed and compared to older json. If they match, no changes occured and the test passes, otherwise the test fails. However, to benefit for it json must be readable. In this case it is not.]

I’m not disturbed by this tiny inconvenience. We have our first E2E test. Let’s celebrate!

And next time, we will bring this E2E test to the cloud, running it in the pipeline of our services.

————

Thanks for reading.

The project logged in this diary is open source, so if you would like to code or suggest changes, please visit https://github.com/TheProgrammersDiary.

Next part: https://hashnode.programmersdiary.com/running-microservices-e2e-tests-on-a-pipeline.

E2E testing microservices

Subscribe to my newsletter

Evaldas Visockas

Evaldas Visockas