Exploring Shared Code Challenges in Poly-Repo Microservices Architecture

Last month, I found myself navigating a classic challenge in a microservices project: how do you handle code that needs to be shared across multiple services, especially when each service lives in its own repository (a poly-repo)? This question sent me down a path of research, trial, and ultimately, a much deeper understanding of the tools I use every day.

The Initial Problem

The setup was simple: multiple NestJS services, each in its own repo. The problem was also simple: things like utility functions, interfaces, and constants were being duplicated everywhere. The solution, however, was not so straightforward.

Exploring the Options

Git Submodules

My first instinct was to look at Git submodules. On the surface, they seem perfect for this. You can include one repo inside another. I gave it a try, but the developer experience wasn't smooth. Managing which version or state of the common code each service should use got complicated quickly. Trying to link specific commits of the shared repo across all the service repos felt like a constant battle and a potential source of future errors.

Monorepo Tools - Nx

Next, I looked into monorepo tools, specifically Nx. Monorepos are a popular solution where all your code lives in one big repository, and tools like Nx make managing dependencies between different parts of the project much easier. While it seemed great for local development, it would have required a fundamental change to our project structure. Since our setup was already a poly-repo, adopting a monorepo wasn't a direct fit. I also had some doubts about how we would manage different versions of the common code for each service if needed. Honestly, there was also a skill and time constraint; I didn't have the time to deep dive into Nx to become fully confident with it for our production flow.

The Chosen Path: Private NPM Packages

Ultimately, I settled on a solution that felt much cleaner and more suited to our poly-repo setup: publishing the shared code as a private npm package.

This approach neatly solved the versioning and distribution issues. Each service could simply install the commons package just like any other dependency, specifying the exact version it needed in its package.json.

But the real value came from an unexpected place. To implement this solution correctly, I had to go on a deep dive into package.json and the npm ecosystem.

A Deep Dive into package.json

This journey forced me to really understand what was going on inside my package.json file. I learned about various keys and npm lifecycle scripts that are crucial for publishing packages.

NPM Lifecycle

One of the first things I had to get right was the npm lifecycle. I learned the difference between the npm install and npm publish lifecycles. A key script for me was prepare. The prepare script runs before the package is packed and published for npm publish and it runs at the end of the cycle in npm install, making it the perfect place to run commands like generators, as the case with my project.

"scripts": {
    "build": "yarn prepare && tsc",
    "prepare": "yarn generate:proto && yarn generate:prisma && yarn seed",
    "generate:proto": "npx protoc --ts_proto_out=./src/proto-types/ ./proto/*.proto --ts_proto_opt=nestJs=true --ts_proto_opt=addGrpcMetadata=true",
    "generate:prisma": "prisma generate --schema=./prisma/schema/",
}

By adding this, I could be sure that I was always publishing the latest compiled JavaScript code, not my raw TypeScript files.

“files” Key

The next major discovery was the files key. I learned this gives you precise control over exactly what gets included in your final published package. It acts as a whitelist. This is incredibly useful because if you don’t specify the files field, npm includes everything that isn't explicitly ignored by a .npmignore file.

Using files ensures that only what you need gets published. For a typical TypeScript project, this might look like:

"files": [
    "dist",
    "generated/prisma",
    "proto"
  ],

In this snippet, I'm only including the compiled dist folder, the generated/prisma folder for the types generated from my schema, and lastly, the proto folder for my gRPC setup in all various microservices. Everything else, like test files, tsconfig.json, or local environment files, is left out.

.npmignore

This also brought me to the .npmignore file. It works like .gitignore but for your npm package. However, there's a tricky part: if a .npmignore file exists, your .gitignore file is completely ignored by npm. This can lead to accidentally publishing things you thought were ignored. Because of this, I found that using the files key as a whitelist is often a safer and clearer approach.

The Not-So-Easy Parts

Of course, publishing a private package isn’t a perfect solution and comes with its own set of challenges.

The biggest one is the friction in local development. When I made a change to the commons package, I couldn't just see it reflected in my services immediately. I had to install the package locally using npm install package-name@file:../path-to-commons-folder. While this works, it can introduce its own issues with symlinks, changes not reflecting until a manual rebuild, and potential mismatches between the local and published versions.

Additionally, managing access tokens for the private registry across different environments (local, CI/CD, production) adds another layer of setup and complexity.

Final Thoughts

This entire experience was a powerful reminder that sometimes, solving a high-level architectural problem forces you to strengthen your understanding of the fundamental tools you use every day. What started as a quest to share code ended up being a valuable lesson in the npm ecosystem. It's these kinds of challenges that truly push our growth as developers.