OpenResty: The overpowered web server used by 40M websites (that people rarely talk about)

For the last three years, I have been working with server-side technologies, especially web servers to deploy my proxies and API gateways. I always thought Nginx was the best open-source web server you could get. Event-driven architecture, ability to load balance with upstreams, etc. They even claim to be a proxy server for email protocols such as POP3, and IMAP. But a lot of limitations and lack of modifiability had me looking for other options, and then I discovered the option I’ll never trade for another web server. I’m the type of person who yearns to spread the word about everything awesome.

Limitations of Nginx - What it didn’t allow me to do

Nginx had limitations that nobody usually complained about and that they used to solve in a lower layer of their infrastructure. Let’s say you want to check for authentication before sending the request to your app servers. With Nginx, there’s no way to run a middleware for this purpose. Rather, people tend to have an API gateway that receives all the requests from Nginx, and they run the authentication check there; after that, they pass the request to the app servers. This is slow, complex, and can even cost you more if you are using serverless infrastructure.

A few weeks ago, I was creating a web application where I needed to balance the traffic to Python Flask servers, which were registered on a Consul service mesh. Before even passing the traffic, I wanted my Nginx server to validate the authentication token so that my Python application wouldn’t have to do it. But that wasn’t possible with Nginx alone.

Nginx Plus tried to solve it in a way I didn’t like. The biggest thing it DIDN'T solve is the ability to modify the behavior of Nginx. It did offer njs, which is JavaScript you can run on Nginx; however, it doesn’t allow you to turn Nginx into a web application. Again, Nginx Plus is 2500$ per year, which is pretty expensive for small companies or independent developers. Plus, I’m a BIG fan of open-source software. Because of all that, Nginx Plus was a big no-no for me.

Behold, the almighty OpenResty!!!

First of all, let me get something straight. I am usually never fazed by technology. But OpenResty was probably one of the things that made me go “WHY DIDN’T I KNOW ABOUT THIS EARLIER?!?!?!?!” and there are many good reasons for that. When I first went to their website, I saw

OpenResty® is a dynamic web platform based on Nginx and LuaJIT. It integrates the standard Nginx core, LuaJIT, many carefully written Lua libraries, lots of high quality 3rd-party Nginx modules, and most of their external > dependencies. It is designed to help developers easily build scalable web applications, web services, and dynamic web gateways.

I thought they were capping (basically, a brand new word for lying). A dynamic web platform that runs on Nginx, and can be modified using Lua? I don’t know about others, but I was convinced after reading this single line. It’s like you are getting all the features of Nginx, but also the ability to dynamically program it and make it work like your own thing. This means you get the performance of C, but write code with the ease of Lua. I was a bit concerned at first since I thought they were doing the same thing as Nginx Plus, but I was wrong.

You can use OpenResty for developing scalable web apps, web services, dynamic web gateways, APIs, and so much more! There’s no end to what you can do since you have been given the freedom to program its features.

OpenResty as a full-on web application? CAN YOU EVEN DO THAT?

OpenResty gives you the power to execute your application logic inside it. It’s made in such a way that you don’t even need an application server running and proxy requests to that! In terms of performance, you are getting the speed of C while executing application logic. You can also interface with databases such as MySQL, PostgreSQL, Redis, etc., directly in OpenResty and the community has plenty of modules/libraries for it.

Someone even made their blogging site only with OpenResty and Redis; they did not use any other backend application: https://github.com/torhve/LuaWeb. Unfortunately, they implied OpenResty to be a bunch of plugins; however, it’s a fully functional web platform on its own with its vast library of official and third-party modules.

OpenResty as a proxy and API-Gateway? Is that a thing?

Most people already have their application written in some other programming language such as Python, JavaScript, Java, etc., so they would want to use OpenResty as an API gateway instead of an application server. Using OpenResty, you can write code in Lua to make it work as an API gateway.

Some API gateways such as Spring Cloud Gateway, run server-side code (for authentication, or input formatting) and then proxy the requests. But with the JVM, the execution is less performant. Since OpenResty runs your code in Lua (which runs on C), you get blazing fast performance; which might not be visible for a small amount of traffic but will play a big effect when it comes to huge traffic.

Also, if you are talking about the Kong API Gateway, it’s built on top of OpenResty.

I used OpenResty for load balancing my Flask applications registered on a Consul service mesh. I sent DNS requests to the Consul server using Lua to fetch the application instances, and then proxied the requests to the application. Without OpenResty, I would have to use another Python application as an API gateway, which would be slower, and more agonizing. With a single Lua file, I turned OpenResty into a dynamic load balancer that uses Consul as its service discovery. Here’s the repository for my project: https://github.com/ahnaf-zamil/openresty-consul-proxy/

But who uses OpenResty?

I was a bit surprised when I first found out that Cloudflare had been supporting OpenResty since its early days and even uses it for a lot of their products. It is also worth mentioning that OpenResty was first sponsored by the Chinese e-commerce giant Taobao. According to NetCraft’s May 2022 web server survey, 40.7 million websites run on OpenResty.

NetCraft states this in its September 2016 survey,

“Switching from Nginx to OpenResty is not such a paradigm shift as moving to, say, Apache or Microsoft IIS. The OpenResty web application platform is built around the standard Nginx core, which offers some familiarity, as well as allowing the use of third-party Nginx modules. One of the key additional features provided by OpenResty is the integration of the LuaJIT compiler and many Lua libraries – this gives scope for high performance web applications to be run completely within the bundled Nginx server, where developers can take advantage of non-blocking I/O.”

Companies such as Kong, Shopify, GrubHub, Strava, AlgoExpert, Tumblr, etc, use OpenResty for powering their backends as well. The Kong API gateway itself is powered by OpenResty under the hood. It’s the 3rd most popular web server on the internet.

Even a lot of people who claim to be using Nginx are actually using the Lua module which was made by and for OpenResty. The Kubernetes Nginx ingress controller uses OpenResty under the hood through its Lua module: https://github.com/kubernetes/ingress-Nginx/blob/88e96decd95e7bd10e7de62b102284ff2bc82593/docs/how-it-works.md

One of the posts in Shopify’s Engineering blog states this,

“One of Shopify's secret weapons is our edge tier, which uses a combination of Nginx and OpenResty's Lua module. This module integrates into Nginx's event model allowing us to write Lua scripts which operate on requests and responses.”

Source: https://shopify.engineering/surviving-flashes-of-high-write-traffic-using-scriptable-load-balancers-part-i

I’m excited, how do I get started?

Firstly, you need to send $100 to a very fishy Bitcoin wallet to get a TOR link for the OpenResty binary on the dark web… just kidding. OpenResty is 100% open-source and FREE!!!!!! I mean, if it weren’t open-source, there wouldn’t be a point in having “Open” at the start of its name. I don’t know… I’m not the one who came up with the name :/

Anyways, I recommend you check out OpenResty’s website (https://openresty.org) first. OpenResty offers binary packages, but you can also compile from source code. I used the latter option (because I have an obsession with compiling stuff from scratch), and it didn’t take me long to do it, about 4-5 minutes.

Next, you can get started with OpenResty by reading the “Getting Started”(https://openresty.org/en/getting-started.html) page on their website. It doesn’t even scratch the surface, but you will get the idea of how simple it is to use.

If you want to know how you can make your own Lua module in OpenResty (that’s what you should do), then check out this video: Write Your Own Lua Modules in OpenResty/Nginx Applications.

Here’s the API reference for OpenResty: https://openresty-reference.readthedocs.io/en/latest/Lua_Nginx_API/

OpenResty is HEAVILY underrated

Compare the popularity of Nginx, Apache, or IIS with OpenResty. While OpenResty offers a lot more features and better performance in many cases, it’s not as popular as the servers that dominate the industry. I’m not sure why this is happening.

Is it because we tend to keep the most powerful weapons secret to have an advantage over our competitors? Or is it because the OpenResty company/community has been too humble and low-key to show off their amazing features?

I’m very curious about the psychology behind it, because if OpenResty was not popular, then people wouldn’t use it. Being used by 40.7 million sites and lots of big companies means that it is indeed popular. But I don’t understand why people barely talk about it. Whatever it may be, OpenResty is something I would look out for in the future as well. It has a LOT of potential, and I’ve just barely scratched the surface!

Conclusion

I made this post because I wanted OpenResty to gain more popularity, since it’s really an amazing piece of technology that is barely spoken of. It brings in a lot of things required in the current era of the web, where scalability and extensibility are key to having good and performant applications. I hope this post made you consider looking into OpenResty. Once you really get to understand the true power and beauty of this technology, I doubt you will move away from it. With that said, thank you for reading this long post and have a great day!