Data Validation using Cypress with JSON Schema

In our last exploration, we tackled various methods for asserting APIs using Postman. The positive feedback on that article reinforces the importance of robust API testing. While assertions are essential for verifying responses, what if we could move beyond simply checking if data exists? This is where JSON Schema comes in.

The Need for JSON Schema

JSON Schema offers a powerful approach to proactive data validation. We can define the expected data format upfront instead of waiting to see what the API returns. This ensures data adheres to the specified structure and format, leading to more efficient and reliable E2E testing.

Benefits of JSON Schema

Think about it. E2E testing is all about making sure your application runs smoothly from beginning to end. But what if we could ensure the data flowing through that application is on point from the get-go? That's where JSON Schema swoops in to save the day.

With JSON Schema, we can define the data contracts. We're laying out exactly what the JSON data exchanged within your application should look like – its structure, format, and whole shebang. This means we can catch any funky data issues before they even have a chance to mess with your tests. Boom! Less wasted time debugging weird data anomalies, more time spent building rock-solid tests.

Plus, JSON Schema makes your code way more maintainable. We can create reusable validation checks that practically document themselves. Think of it as adding superpowers to your code – readable, efficient, and ready to tackle any data challenge.

What is JSON Schema?

JSON Schema is a vocabulary that provides a structured way to validate JSON data. It defines the structure of JSON data, including types, required fields, and constraints.

Basic Components of JSON Schema:

Types: Define data types (e.g., string, number, array, object).
Properties: Specify the attributes of an object.
Required Fields: List mandatory fields in a JSON object.
Constraints: Enforce rules such as string length, numerical ranges, or specific patterns.

Simple Example:

Consider a scenario where an API returns user data. By defining a JSON Schema for the expected response, we can ensure the data conforms to the expected structure:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "name": {
      "type": "string"
    },
    "email": {
      "type": "string",
      "format": "email"
    }
  },
  "required": ["id", "name", "email"]
}

The schema example above specifies that the response body should be an object with three required properties: id (integer), name (string), and email (string in email format). Integrating this schema into your test suite ensures any deviations from this structure are caught immediately, providing immediate feedback and maintaining data integrity.

Benefits of Using JSON Schema in E2E Testing

Catching errors early: Identifying invalid data before it disrupts your tests, saving time and effort.
Defining data contracts: Clearly outlining the expected structure and format of JSON data exchanged within the application.
Enhancing code maintainability: Creating reusable and self-documenting validation checks that improve code readability.

Implementing JSON Schema Validation in E2E Testing

Step-by-Step Guide:

Define JSON Schemas: Create JSON Schema files that define your application's data structure.
Integrate with Testing Framework: Use a framework that supports JSON Schema validation. Popular choices include Cypress and Selenium with appropriate libraries.
Validate API Responses: Implement tests that validate API responses against your JSON Schemas.
Incorporate in CI/CD: Ensure JSON Schema validation is part of your CI/CD pipeline to catch issues early.

Tools and Libraries:

Ajv: A JSON Schema validator for JavaScript.
JSON schema: A Python library for JSON Schema validation.
ZSchema: A JSON Schema validator that supports drafts 04, 06, and 07.

Practical Example:

Let's use the Star Wars API (SWAPI) to create Cypress tests with JSON Schema validation. We'll interact with the SWAPI to fetch details about a specific Star Wars character and validate the response against a JSON Schema.

https://swapi.dev/api/people/1

Setting Up

First, ensure you have Cypress installed in your project.

npm install cypress

Create a directory named schemas to store JSON Schema files.

JSON Schema Definition

Create a directory named schemas in your Cypress project and create a file within it with the name schemas/character-schema.json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "height": { "type": "string" },
    "mass": { "type": "string" },
    "hair_color": { "type": "string" },
    "skin_color": { "type": "string" },
    "eye_color": { "type": "string" },
    "birth_year": { "type": "string" },
    "gender": { "type": "string" },
    "homeworld": { "type": "string" },
    "films": {
      "type": "array",
      "items": { "type": "string" }
    },
    "species": {
      "type": "array",
      "items": { "type": "string" }
    },
    "vehicles": {
      "type": "array",
      "items": { "type": "string" }
    },
    "starships": {
      "type": "array",
      "items": { "type": "string" }
    },
    "created": { "type": "string" },
    "edited": { "type": "string" },
    "url": { "type": "string" }
  },
  "required": [
    "name", "height", "mass", "hair_color", "skin_color",
    "eye_color", "birth_year", "gender", "homeworld", "films",
    "species", "vehicles", "starships", "created", "edited", "url"
  ]
}

"$schema": "http://json-schema.org/draft-07/schema#":
- This line specifies the version of the JSON Schema standard being used. In this case, it is using Draft 7.
"type": "object":
- This indicates that the root of the JSON data being described is an object.
"Properties":
- This section defines the properties that the object can have and their respective data types.
Properties Definition:
- "name":
  - Type: string
  - Description: The name of the character.
- "height":
  - Type: string
  - Description: The height of the character.
- "mass":
  - Type: string
  - Description: The mass of the character.
- "hair_color":
  - Type: string
  - Description: The hair color of the character.
- "skin_color":
  - Type: string
  - Description: The skin color of the character.
- "eye_color":
  - Type: string
  - Description: The eye color of the character.
- "birth_year":
  - Type: string
  - Description: The birth year of the character.
- "gender":
  - Type: string
  - Description: The gender of the character.
- "homeworld":
  - Type: string
  - Description: The URL of the character's homeworld.
- "films":
  - Type: array
  - Items Type: string
  - Description: An array of URLs representing the films the character appears in.
- "species":
  - Type: array
  - Items Type: string
  - Description: An array of URLs representing the species of the character.
- "vehicles":
  - Type: array
  - Items Type: string
  - Description: An array of URLs representing the vehicles the character has used.
- "starships":
  - Type: array
  - Items Type: string
  - Description: An array of URLs representing the starships the character has used.
- "created":
  - Type: string
  - Description: The timestamp when the character entry was created.
- "edited":
  - Type: string
  - Description: The timestamp when the character entry was last edited.
- "url":
  - Type: string
  - Description: The URL of the character resource.
"required":
- This array lists all the properties that are mandatory in the JSON object. The object must contain all these properties for it to be considered valid according to this schema.
- In this case, all defined properties are listed as required: name, height, mass, hair_color, skin_color, eye_color, birth_year, gender, homeworld, films, species, vehicles, starships, created, edited, and url.

2. Implement JSON Schema Validation Command

In your Cypress project, create the cypress/support/commands.js file and add the following code to define a custom command for JSON Schema validation:

install Ajv using npm install ajv ajv-formats --save-dev

import Ajv from 'ajv';
import addFormats from 'ajv-formats';

const characterSchema = require('../schemas/character-schema.json');

const ajv = new Ajv({ allErrors: true });
addFormats(ajv);

Cypress.Commands.add('validateSchema', (schemaName, data) => {
  let schema;
  if (schemaName === 'character') {
    schema = characterSchema;
  } else {
    throw new Error(`Invalid schema name: ${schemaName}`);
  }
  const validate = ajv.compile(schema);
  const valid = validate(data);
  expect(valid, `Response body matches the JSON schema (${schemaName})`).to.be.true;

  if (!valid) {
    console.error(validate.errors);
  }
});

3. Write Cypress API Tests

Create a new test file in cypress/integration (e.g., swapi.spec.cy.js) and add the following test cases to interact with the SWAPI and validate the responses:

describe('Star Wars API Tests', () => {
  it('validates character details using JSON schema', () => {
    const characterId = '1'; // Luke Skywalker

    cy.request(`https://swapi.dev/api/people/${characterId}`)
      .then((response) => {
        // 1. Assert status code
        expect(response.status).to.eq(200);

        // 2. Validate response body against schema
        cy.validateSchema('character', response.body);
      });
  });
});

schemaName ('character'):
- The string 'character' is used in the custom Cypress command to determine which JSON schema to use for validation. It is a label used to select the correct schema (in this case, characterSchema).
cy.validateSchema('character', response. body):
- This line calls the custom command validateSchema with two parameters:
  - schemaName: 'character', which selects the characterSchema.
  - data: response.body, which contains the JSON response from the SWAPI.

4. Run the Tests

Run the Cypress tests to interact with the SWAPI and validate the responses against the defined JSON schemas:

npx cypress open

This will open the Cypress Test Runner. Select the swapi.spec.js test file to run the tests.

JSON Schema: Defined in schemas/character-schema.json to specify the expected structure and types for a Star Wars character.
Custom Command: The validateSchema command in commands.js uses Ajv to validate JSON data against the specified schema.
API Test:
- The test sends a GET request to fetch details about a specific Star Wars character (in this case, Luke Skywalker, whose ID is 1).
- The response is validated against the character-schema.json to ensure it conforms to the expected structure and data types.

In this example, I demonstrated how to use Cypress for API testing and integrate JSON Schema validation to ensure the integrity and correctness of API responses. Using the Star Wars API (SWAPI) as our test case, we set up JSON Schema validation for a specific character's data, implemented a custom Cypress command for schema validation, and wrote Cypress tests to fetch and validate API responses.

Key Takeaways:

Robust API Testing: By incorporating JSON Schema validation, you can ensure that the data returned by APIs adheres to the expected structure and data types, catching any discrepancies early in the testing process.
Maintainable Tests: Using JSON Schema helps in maintaining clear and concise test cases, making it easier to update and manage tests as the API evolves.
Reusable Components: Defining JSON Schemas for different API responses allows for reusable validation logic across multiple tests, enhancing the scalability of your test suite.
Enhanced Reliability: Validating API responses against predefined schemas improves the reliability and robustness of your application by ensuring consistent and accurate data handling.

By integrating JSON Schema validation into your Cypress API testing strategy, you can automate data validation, detect errors early, and maintain consistency across your application. This approach enhances your software's quality and streamlines the testing process, leading to more efficient and reliable releases.

Feel free to adapt these practices to your specific API testing needs and use the powerful combination of Cypress and JSON Schema to achieve comprehensive and effective test coverage.

Additional Resources

💬 Please share your experiences and insights on using JSON Schema in API testing in the comments below.
📬 Subscribe to my blog for more QA tips and best practices.
🎉 Happy testing!

Beyond Assertions: Data Validation using Cypress with JSON Schema in End-to-End Testing

Table of contents