For a long time, it was normal to initially release a new feature or update into a test environment. If the feature passed, it was then released to the production environment. While this approach was highly respected and beneficial, it introduced more complexity into software development workflows, and releases took longer to reach end-users. Fortunately, with a mechanism known as feature flagging, you can deploy directly to production and ship releases faster while maintaining reliability.

Definition and purpose of test environments

Let's unpack what a test environment is and describe why it is useful. A test environment is an isolated space separate from the production environment. It serves as a sandbox where developers and testers can evaluate new features and updates. When the test results become acceptable, the new feature is deployed to the production environment. Without a test environment, developers run the risk of introducing bugs that could affect the user experience.

Challenges and limitations of test environments

While there are many benefits to setting up and using test environments, it can come with challenges that may outweigh the benefits. These can include higher costs, increased complexity, and even coverage gaps. Let's talk about these.

Higher cost

A test environment may require technical resources to host and maintain it. Whether or not you actively use a test environment or spin one up on demand, it still utilizes storage space, CPU, RAM, etc. If a change you need to test is as simple as changing the text on a button, and your release policy is to run it through your test environment, you'd have to spin up your test environment to preview and test the changes. Let's say you now need an extra 32GB of RAM to run the new version of your software; then, you need to mirror the same in your test environment, thus increasing the cost.

Complexity

A test environment adds complexity to development workflows and steepens the learning curve for new developers. This can result in longer debugging times when issues arise.

As an example, imagine that you added a new feature to your app. During testing, you noticed that you need to upgrade a specific npm package. If you forgot to do the same in your production environment to match, you could potentially introduce a bug. The same applies to hardware, where you provision the correct amount of resources in the test environment but forget to make similar adjustments in the production environment.

Coverage

Coverage issues can arise when your test environment is unable to account for and test all the conditions usually found in a production environment. In cases where companies choose to invest only in the bare minimum resources to run their test environment, it can be difficult to truly test performance bottlenecks because these tend to manifest only in a production environment with real users.

Testing in production with feature flags

What if you could achieve the same or better benefits without the complexity and cost of a test environment? Enter the world of feature flags. They allow you to remotely switch your features on or off at will, just as you do with the lights in your home. As a result, you can show or hide a feature by toggling its flag, eliminating the need for editing and redeploying your application. This approach enables you to test the new feature on a portion or even your entire user base in production.

Let's look at some of the benefits of testing in production:

Faster feedback

Most often, developers are eager to know how a new feature performs in the real world. A test environment usually slows down this feedback loop and can disrupt developer momentum. By releasing a new feature directly to production, this momentum remains intact. If new bugs show up, the new feature can be rolled back with just a click, allowing developers to rectify them.

Higher quality

Because real users are present in a production environment, feedback from a new feature release can be of better quality. This provides a better chance of minimizing or even preventing test regressions.

Lower risk

Releasing a new feature to production can be risky without a feature management system. Feature flags can keep your risk low by allowing you to roll back a problematic feature with just a few clicks. If you need to get even more creative, you can apply targeting rules to your feature flags and perform a canary release or progressive deployment.

Companies testing in production

Up to this point, we talked about the benefits that testing in production brings to the table, but the question is, has it proven itself? Short answer, yes.

Kantan.tech moves faster than their competition

Kantan.tech is a company that outpaces its competitors by offering a platform via a React Native app. This platform facilitates tradespeople in managing customers, while customers can conveniently connect and book their services.

The company's emphasis on speed has allowed them to accelerate the release of product features. Engineers employ feature flags for new features, which are set to be off by default. These flags are initially activated for internal users for testing and subsequently rolled out to production, becoming available to all users.

Following a successful deployment and testing period, feature flags are removed from the codebase approximately one week later. This approach enables Kantan.tech to swiftly introduce and refine new features while maintaining operational stability.

Eucalyptus growth is improved

Eucalyptus has achieved notable growth by adopting a Telehealth platform that utilizes a shared tech stack to establish distinctive brands. Central to their success is the wholehearted integration of feature flags, which play a pivotal role in their operations. By effectively managing features through the use of these flags, Eucalyptus has gained a competitive edge within the market. This approach allows them to reduce potential risks and enhance their response times, resulting in an improved market position compared to their counterparts in the industry.

Conclusion

Using test environments was the de facto standard for testing. It offered benefits but added more complexity to software development workflows. With a good feature management system in place, you can skip using them and test directly in production with the help of feature flags. If set up correctly, you can easily roll back new updates or changes without the risk of damage while performing improvements and fixes. Putting aside the theory, I've mentioned two case studies of companies that took advantage of this approach. It has served them well along the way.

Eager to give feature flags a try? Get a generous free tier ConfigCat account for low-volume use cases which is ideal for getting started with feature flags. Created by developers for developers with ❤️.

For more awesome content, keep up with ConfigCat on X, Facebook, LinkedIn, and GitHub.

Skipping Test Environments for Faster and Safer Deployments