Refactoring Compas code-gen: part 1

Recently we started working on refactoring the code generators of Compas.js. Compas.js is a framework based on a structure providing generated routers, validators, typed api clients and more.

Code generators

Shortly presented, Compas.js is able to do the following:

The input:

R.post("/posts", "postList")
  .body({
    filters: {
       since: T.date().inThePast(),
    }
  }).response({
    posts: [{
      id: T.uuid(),
      title: T.string(),
      publishedAt: T.date(),
    }]
  })

And turn it in to a router:

appHandlers.postList = async (ctx, next) => {
  // ctx.validatedBody contains `{ filters: { since: new Date("2022-09-10") } }`
  ctx.body = await getPostList(ctx.validatedBody);
}

Or a typed api client

type AppListBody = {
  filters: {
    since: Date;
  };
};

type AppListResponse = {
  posts: {
    id: string;
    title: string;
    publishedAt: Date;
  }[];
}

export async function apiAppPostList(axiosInstance: AxiosInstance, body: AppListBody): Promise<AppListResponse);

export function useAppPostList({ body }: { body: AppListBody }) {
  // @tanstack/react-query based hooks
  return useQuery(/* ... */);
}

Necessary improvements

We have a few reasons to take on this endeavour rewriting a relatively stable, but complex, system.

Combining generate calls

Compas.js currently has three different generate calls:

• App#generate
• App#generateTypes
• App#generateOpenApi

App#generate is the main entrypoint to generate validators, routers, api clients, etc. Over time we introduced App#generateTypes. This could combine multiple App#generate outputs, and derive and dedupe all the types necessary for the App#generate calls and combine those types in to a single output.

Next we added App#generateOpenApi. This became a separate call since its necessary options had no place between the options accepted by App#generate

Generator targets

Compas.js currently supports three generator targets: isBrowser, isNode and isNodeServer. If you ask me what the difference is between isNode and isNodeServer, I would need to look it up in the code. The isBrowser target also has some edge cases. For example FormData in the browser behaves differently than in React-Native, so we have extra options in App#generate to support this.

Adding support for, for example, Fetch in the api client generator would currently result a new option for App#generate, and a whole bunch of inline logic in the generators and the maintenance burden that follows.

Error handling

Errors reported by the code generator caused by invalid input structures is a bag of mixed emotions. Some errors contain suggestions on how to fix them, like in the first image below. Some just throw an error and good luck trying to find where this came from.

Readable output

I have always had the opinion that generated code should be readable. The current output is, but only to a trained eye. Another related topic is correctness. Like how many lint rules do I need to disable for the generated code. Here, we can have easy wins when redesigning the generators

Internals

And lastly we need to think about potential contributors to the generator code. How easy is it to navigate? How quickly can we add support for a new target language, library or platform? How quickly can someone write good test coverage?

We currently use at least 4 forms of string templating in the code generators. And all of them have served their purpose, but I think we can unify them. All ways of templating has trade-offs. We just need to make sure that we are aware of these for the templating type that we are going to use.

Up next

This may read like the Compas.js code generators are not that good. However, much of the supported features and outputs are perfectly fine and happily used in quite a lot of projects. But there is a lot of room for DX improvements, both for users and contributors. And there is a lot of potential in targets that we can support and new features that come with those.

The refactor is currently in progress. You could peek along at feat/experimental-code-gen.

Keep watching this blog for upcoming posts about how we combine the different App#generate calls in to one, or the one about the different templating options we currently have and the trade-offs associated with them.