Solving SEO Challenges with Cloudflare Workers: A Guide to Implementin

Introduction: A Problem I Didn't See Coming

A few weeks ago, I was working on a client project that had a unique challenge: they had their homepage hosted on Webflow at example.com and their SaaS application hosted on a separate subdomain, app.example.com. This setup seemed fine at first glance, but the client brought up a valid concern—this configuration was negatively impacting their SEO performance.

Having the homepage and the main application split across different subdomains can confuse search engines, reducing the overall domain authority and lowering the search rankings. The client’s goal was to have the entire application served from the apex domain, example.com without moving any files or modifying the existing Next.js application(spoiler alert, some modifications had to be made!). It sounded straightforward, but as always, the devil is in the details. The solution? A reverse proxy using Cloudflare Workers.

Problem Statement: Why Separate Domains Harm SEO

The client’s website was structured as follows:

Homepage: example.com (Hosted on Webflow)
SaaS App: app.example.com (Hosted on Next.js)

Having multiple subdomains for a single site can dilute the site's overall SEO authority. The search engine treats each subdomain as a separate entity, making it harder to rank for competitive keywords. With example.com and app.example.com operating independently, this setup could result in:

Lower SEO Rankings: Search engines may not understand the relationship between the homepage and the app, leading to lower rankings for both.
Splitting Backlinks: Any backlinks pointing to app.example.com would not contribute to the authority of example.com.
Inconsistent User Experience: Users navigating between example.com and app.example.com would experience a shift in URL structure, making the navigation less intuitive.

The goal was to map the subdomain (app.example.com) as a path (example.com/app) on the main domain. This required creating a reverse proxy setup.

Reverse Proxy: What is It, and Why Use It?

A reverse proxy is a type of server that sits between the client (browser) and the server (origin) to intercept and manage requests. Instead of directly communicating with the origin server, the client sends the request to the proxy server. The proxy server then forwards the request to the origin server, retrieves the response, and sends it back to the client.

In simple terms, it’s like a middleman. You send your request to the middleman, who then decides which actual server should handle your request. This allows you to control, modify, or reroute traffic in a highly customizable way.

Why Use a Reverse Proxy?

In our case, a reverse proxy allows us to:

Consolidate URLs: Map app.example.com/login to example.com/app/login.
Optimize SEO: Improve search engine rankings by serving all content under a single domain.
Enhance Security: Hide internal details of server architecture.
Improve Performance: Cache responses at the edge to speed up load times.

With Cloudflare Workers, we could easily set up a reverse proxy to serve our Next.js app under a single domain.

Solution: Cloudflare Worker Script

Using Cloudflare Workers and the Wrangler CLI, I was able to create a reverse proxy solution that accomplished exactly what we needed.

A Bit of Backstory

I’ll be honest—initially, I was absolutely clueless about how I could solve this problem. I knew what the issue was and even had a rough idea of the solution, but I couldn’t figure out a way to implement it effectively. I was stuck in a loop of “I know what needs to be done, but I have no idea how to do it.”

Just when I was about to give up, my manager nudged me in the right direction and handed me the holy grail—a GitHub repository that was almost a perfect fit for my requirements (well, almost). It didn’t solve the problem 100%, but it did show me the right path. I just had to add a bit of my own secret sauce to get it working.

This repository was a lifesaver. It confirmed that I was on the right track and gave me a concrete example of how to approach the solution. With the foundation in place, I modified the script and implemented the tweaks I needed to handle the peculiarities of my project.

Here's the base code snippet that got me started:

async fetch(request, env, ctx): Promise<Response> {
        const { origin, hostname, pathname, search } = new URL(request.url);
        const main_origin = create_origin(env.DOMAIN);

        const paths = pathname.split('/').filter(Boolean);

        const matcher = new Matcher(env.SUBDOMAINS);
        // Handle trailing slashes
        if (paths.length && has_trailing_slash(pathname)) {
            const redirect_url = build_url([origin, paths], search);

            return Response.redirect(redirect_url, 301);
        }

        // Path matches reverse proxied subdomain
        const match = matcher.path_to_subdomain(paths);
        if (match) {
            const { subdomain, wildcard_paths } = match;
            const target_origin = create_origin(`${subdomain}.${env.DOMAIN}`);
            const target_url = build_url([target_origin, wildcard_paths], search);
            const response = await fetch(target_url);

            // Serve the content directly, maintaining the current URL
            return new Response(response.body, {
                status: response.status,
                headers: response.headers,
            });
            //return response;
        }
        console.log("modified reuqest yoo")
        return fetch(request);
    },

Explanation

Request Interception: The worker listens for incoming requests and checks if the URL starts with /app.
URL Modification: If the URL matches, the worker modifies the hostname to point to app.example.com and removes the /app prefix from the path.
Response Handling: The worker then fetches the content from app.example.com and serves it back to the client as if it originated from example.com.

Problems Faced During Execution

Initially, everything seemed to work smoothly. When I accessed example.com/app/login, the page loaded, but it was just an empty skeleton of the site—no styling, no interactivity. After hours of troubleshooting and scouring forums, I realized the issue: since my Next.js app primarily used client-side rendering, the JavaScript and CSS modules were not being fetched from the correct source.

The worker was trying to fetch these resources from example.com/app, but they were only available at app.example.com. To resolve this, I modified the worker to fetch JavaScript and CSS chunks from app.example.com and return them as part of the response to example.com/app.

It was like a wholesale market—retailers get their products from a single supplier but rebrand and present them under their own name. Similarly, my worker fetched resources from app.example.com and served them under example.com/app.

Here's the updated code:

async fetch(request, env, ctx): Promise<Response> {
        const { origin, hostname, pathname, search } = new URL(request.url);
        const main_origin = create_origin(env.DOMAIN);

        const paths = pathname.split('/').filter(Boolean);

        const matcher = new Matcher(env.SUBDOMAINS);

        // Handle asset rewriting
        if (hostname === env.DOMAIN) {
            // Rewrite requests for static assets to app.example.com
            if (pathname.includes('_next') || pathname.includes('/static/') || pathname.includes('/images/')) {
                let asset_url = request.url.replace(env.DOMAIN, `app.${env.DOMAIN}`);

                // Remove '/app/' from the URL if it is present
                if (pathname.includes('/images/') && pathname.includes('/app/')) {
                    asset_url = asset_url.replace('/app/', '/'); // Remove '/app/' from the URL
                }

                return fetch(asset_url);
            }
        }
        // Handle trailing slashes
        if (paths.length && has_trailing_slash(pathname)) {
            const redirect_url = build_url([origin, paths], search);

            return Response.redirect(redirect_url, 301);
        }

        // Path matches reverse proxied subdomain
        const match = matcher.path_to_subdomain(paths);
        if (match) {
            const { subdomain, wildcard_paths } = match;
            const target_origin = create_origin(`${subdomain}.${env.DOMAIN}`);
            const target_url = build_url([target_origin, wildcard_paths], search);
            const response = await fetch(target_url);

            // Serve the content directly, maintaining the current URL
            return new Response(response.body, {
                status: response.status,
                headers: response.headers,
            });
            //return response;
        }
        console.log("modified reuqest yoo")
        return fetch(request);
    },

Results: A Partially Solved Problem

After making these tweaks, I was able to successfully serve app.example.com content under example.com/app without any styling issues or missing resources. However, one challenge remained unsolved—the automatic redirection from app.example.com to example.com/app.

Ending Remarks: A Problem Still Left Unsolved

Although I managed to reverse proxy example.com/app to app.example.com, I faced an unexpected roadblock when trying to automatically redirect app.example.com to example.com/app. As soon as I configured the CNAME record, Vercel’s domain settings showed an invalid configuration error, and the site became unlinked from the domain.

After searching through Cloudflare and Vercel communities, I stumbled upon a crucial detail—Vercel does not currently support CNAME flattening(checkout this link). Without this support, achieving seamless redirection is nearly impossible. This remains an unsolved mystery, so if anyone reading this has a solution, feel free to reach out and let me know.

Until then, this is where the story ends (for now). If you have any thoughts, ideas, or solutions, hit me up. Adios!

P.S.: I went through the trial by fire so you don’t have to! 😉 Here’s the GitHub repo link to the full code. Happy coding! 🔥

How I Solved an SEO Problem with Cloudflare Workers: A Reverse Proxy Story

Table of contents