Hoppscotch API Live-Sync

I wonder how many of us have come across a problem and thought, "There should be an AI or at least a tool/software for this." Don't worry, this blog post and all the ones coming after in this series have nothing to do with AI.

As a backend engineer, Postman is my go-to tool for API documentation. It does the trick for me with its polished UI and collection-style documentation. The drawback is that you have to document your APIs manually. And after having written many APIs, it has now become a chore to create and update the API documentation manually, especially as API specifications change a lot. Think of it as wanting to watch a live television broadcast instead of watching recorded videos—you're always seeing the most current information without any manual intervention required.

I have tried other API testing and documentation tools in the past, such as Swagger UI and VS Code's ThunderClient. I especially loved Swagger UI for its auto-sync feature when you make changes to your API; however, I don't find its UI as attractive as that of Postman.

Postman does not offer this auto-sync feature out of the box, and since my fingers are itching to build something, I have cloned its open-source alternative, Hoppscotch, and I am going to add this auto-sync feature to it. This is the first post in a series that will cover my journey from cloning the Hoppscotch codebase to finalizing this feature. I am excited and intimidated by this endeavour, and I hope you will enjoy this journey with me.

In this first post, I will be describing Hoppscotch's current codebase structure before I dive into the implementation.

Hoppscotch Architecture

Hoppscotch uses a monorepo structure, housed in multiple packages under a single repository, and managed with pnpm workspaces. I first learned about pnpm while exploring this codebase, and it is described as a faster and more efficient npm. Its name is Performant-NPM. It shines in large projects with many dependencies by caching these dependencies and saving disk space by downloading only a single copy of each version of a package.

Let's say you have two packages that depend on express.js to work; with npm, these two packages will download and unpack express.js twice. With pnpm, the packages only use one copy.

Cool stuff!

The core of the source code is organized into packages, which I’ll highlight below.

• hoppscotch-common: The hoppscotch-common package serves as the primary frontend application, built with Vue.js and TypeScript. This package contains all the UI components, pages, and business logic for handling various API protocols, including REST, GraphQL, and WebSocket connections. It uses Vite for fast development and optimized builds, Tailwind CSS for utility-first styling, and urql for efficient GraphQL client management. This is most probably where I'll spend the most time adding the auto-sync feature.

• hoppscotch-backend: The backend infrastructure is built on NestJS. The backend handles user authentication, data persistence, and team collaboration features through a well-designed GraphQL API. Prisma is the ORM, while GraphQL is for flexible and efficient API queries. I'll also spend a significant amount of time here.

• hoppscotch-data: The hoppscotch-data package defines the core data structures, types, and validation rules used throughout the application. This centralized approach ensures data consistency between frontend and backend components while providing a single source of truth for data models. I'll create the type definitions here.

Other packages include:

• hoppscotch-kernel: The kernel package provides a cross-platform runtime that abstracts platform-specific operations. This abstraction layer allows the same core logic to run seamlessly across web browsers, desktop applications, and other environments.

• hoppscotch-js-sandbox: A secure environment for executing user-provided JavaScript code, such as pre-request scripts and test scripts.

• hoppscotch-selfhost-web: The self-hostable web version of Hoppscotch. It bundles the hoppscotch-common package into a standalone application that can be deployed on a user's server.

• hoppscotch-desktop: The desktop application for Hoppscotch, built with Tauri. It wraps the web application and provides additional features like offline support and deeper integration with the operating system.

• hoppscotch-cli: A command-line interface for running Hoppscotch test scripts in CI/CD environments.

• codemirror-lang-graphql: Provides GraphQL language support for the CodeMirror editor used in the app.

• hoppscotch-sh-admin: A separate admin interface for managing self-hosted Hoppscotch instances.

• hoppscotch-selfhost-desktop: A self-hosted version of the desktop app.

If you are interested, I have created images that aim to represent the different interactions of the packages here.

Contributing to the Codebase

Hoppscotch encourages contributions through GitHub, following a structured process:

• Discuss proposed changes via issues or direct communication with maintainers.

• Submit pull requests.

• Follow the project’s code of conduct and update documentation with changes.

The large codebase can be daunting for new contributors; however, it's nothing that can't be understood with a few hours spent on it.

Hoppscotch’s codebase is a well-organized, modular system that balances functionality and maintainability. For more details, check out the official Hoppscotch repository on GitHub or explore the documentation at docs.hoppscotch.io.

Next steps

With this basic understanding of the codebase, I'll go ahead and get started with some groundwork for this live-sync implementation. In essence, I'll define data models for storing live source configurations and sync history.

See you in a few days.