Web Data Interception: Building a Browser Extension for API Monitoring

In today's web applications, most of the interesting data flows through API calls. Whether you're debugging a complex web app, analyzing network traffic patterns, or simply curious about what data is being exchanged in the background, having the ability to intercept and examine this data can be invaluable.

This article demonstrates how to build a Chrome extension that intercepts and logs all web requests and their responses, with a particular focus on JSON data, though it works with other data types as well.

I made an example on GitHub

How It Works

My extension works on three levels:

1. Content Script: Injects our interceptor script into the web page

2. Interceptor Script: Hooks into the page's JavaScript environment to monitor fetch and XHR requests

3. Background Script: Processes and stores the intercepted data

This architecture allows us to capture data that might otherwise be inaccessible through the standard DevTools network panel, especially for single-page applications that heavily rely on JavaScript for their functionality.

The Extension Structure

Our extension consists of four key files:

1. manifest.json # Extension configuration

2. content.js # Injects our interceptor script

3. interceptor.js # Contains the interception logic

4. background.js # Processes and stores captured data

Manifest.json: The Extension Blueprint

The manifest.json file serves as the configuration for our extension, defining its capabilities and permissions:

{
    "manifest_version": 3,
    "name": "API Response Listener",
    "description": "Listens to responses from a specific [PAGE]",
    "version": "7.0",
    "permissions": [
        "activeTab",
        "tabs",
        "storage",
        "webRequest",
        "declarativeNetRequest",
        "bookmarks",
        "cookies",
        "downloads",
        "notifications",
        "webNavigation",
        "contextMenus",
        "scripting"
    ],
    "host_permissions": [
        "<all_urls>"
    ],
    "background": {
        "service_worker": "background.js"
    },
    "content_scripts": [
        {
            "matches": [ "<all_urls>" ],
            "js": ["content.js"],
            "run_at": "document_start"
        }
    ],
    "web_accessible_resources": [
        {
            "resources": ["interceptor.js"],
            "matches": [ "<all_urls>" ]
        }
    ]
}

Key components:

• Manifest Version: Uses the modern Manifest V3

• Permissions: Requests various capabilities like storage access and web request monitoring

• Host Permissions: Allows the extension to run on all websites

• Background Script: Defines a service worker for background tasks

• Content Scripts: Scripts injected into web pages, configured to run at document_start for early interception

• Web Accessible Resources: Makes our interceptor.js available for injection

Content.js: The Bridge

The content script runs in the context of the web page and serves as a bridge between the page and our extension:

// Inject the external script file
const script = document.createElement('script');
script.src = chrome.runtime.getURL('interceptor.js');
(document.head || document.documentElement).appendChild(script);

// Listen for messages from the injected script
window.addEventListener('message', function(event) {
    // Make sure the message is from our page script
    if (event.source === window && event.data && event.data.type === 'API_RESPONSE') {
        // Forward the message to the background script
        chrome.runtime.sendMessage(event.data);
    }
});

This script:

1. Creates a script element that points to our interceptor.js

2. Injects it into the page's DOM

3. Sets up an event listener to receive messages from the injected script

4. Forwards those messages to the background script

This approach is necessary because content scripts run in an isolated environment and can't directly modify the page's JavaScript environment. By injecting our script, we gain access to the page's native API objects.

The Interception Technique (Interceptor.js)

The core of our extension is the interception technique. We replace the native fetch and XMLHttpRequest implementations with our own versions that:

1. Call the original methods to ensure normal functionality

2. Clone and examine the responses before they reach the application

3. Forward the data to our background script for processing

This is a form of method hooking, or monkey patching, where we modify existing functions to add our functionality while preserving the original behavior.

Code Walkthrough

Let's examine the key components:

Intercepting Fetch Requests

// Store the original fetch function
const originalFetch = window.fetch;

// Replace with our interceptor
window.fetch = async (...args) => {
    // Log the request
    console.log('Fetch intercepted:', args[0]);

    try {
        // Call the original fetch function
        const response = await originalFetch.apply(window, args);
        // Clone the response so we can read it without consuming it
        const clone = response.clone();

        // Process the response body
        clone.text().then(body => {
            // Try to parse as JSON if possible
            let parsedBody;
            try {
                parsedBody = JSON.parse(body);
            } catch (e) {
                parsedBody = body;
            }

            // Send the data to our content script
            window.postMessage({
                type: 'API_RESPONSE',
                source: 'fetch',
                url: typeof args[0] === 'string' ? args[0] : args[0].url,
                method: typeof args[0] === 'string' ? 'GET' : args[0].method || 'GET',
                status: response.status,
                responseBody: parsedBody,
                headers: Object.fromEntries(response.headers.entries())
            }, '*');
        });

        // Return the original response to the application
        return response;
    } catch (error) {
        console.error('Fetch interception error:', error);
        throw error;
    }
};

Intercepting XMLHttpRequest

For older applications or those not using the Fetch API, we also intercept XMLHttpRequest:

// Store original XHR methods
const originalXHROpen = XMLHttpRequest.prototype.open;
const originalXHRSend = XMLHttpRequest.prototype.send;

// Intercept the 'open' method to capture URL and method
XMLHttpRequest.prototype.open = function(method, url) {
    this._url = url;
    this._method = method;
    return originalXHROpen.apply(this, arguments);
};

// Intercept the 'send' method to capture responses
XMLHttpRequest.prototype.send = function() {
    const xhr = this;

    this.addEventListener('load', function() {
        if (xhr.readyState === 4) {
            // Process and forward response data
            // ...
        }
    });

    return originalXHRSend.apply(this, arguments);
};

Background.js: Data Processing and Storage

The background script runs separately from any web page and handles the captured data:

chrome.webRequest.onBeforeRequest.addListener(
    function(details) {
        console.log("Request started:", {
            url: details.url,
            requestId: details.requestId,
            method: details.method,
            type: details.type
        });
    },
    { urls: ["<all_urls>"] }
);

chrome.webRequest.onCompleted.addListener(
    function(details) {
        console.log("Request completed:", {
            url: details.url,
            requestId: details.requestId,
            statusCode: details.statusCode,
            type: details.type
        });
    },
    { urls: ["<all_urls>"] }
);

chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
    if (message.type === 'API_RESPONSE') {
        console.log('Full Response Data:', {
            url: message.url,
            method: message.method,
            status: message.status,
            responseBody: message.responseBody,
            headers: message.headers,
            timestamp: new Date().toISOString()
        });

        // Store in chrome storage
        chrome.storage.local.set({
            'lastResponse': {
                timestamp: Date.now(),
                url: message.url,
                data: message.responseBody
            }
        });
    }
});

This script:

1. Uses the webRequest API to log all network requests at their start and completion

2. Listens for messages from the content script

3. Processes the intercepted API responses

4. Stores the most recent response in the extension's local storage

The background script serves as the central hub for data collection, providing a persistent environment for monitoring and storing intercepted data.

Putting It All Together

Here's how the data flows through our extension:

1. A web request is made by the page using fetch or XMLHttpRequest

2. Our interceptor.js hooks capture the request and its response

3. The intercepted data is sent via postMessage to our content script

4. The content script forwards the data to the background script

5. The background script logs and stores the data

This multi-layered approach allows us to:

• Capture requests directly from the page's context

• Preserve the original functionality of the web APIs

• Process and store the data outside the page's lifecycle

Practical Applications

This extension can be helpful for:

• Debugging: See exactly what data is being exchanged between your browser and servers

• Learning: Understand how APIs work by examining real-world examples

• Data Analysis: Collect and analyze data patterns from websites you visit

• Create filters!!! For example, suppose you visit a shopping website daily to buy a specific item with certain characteristics, but the site lacks advanced filtering options. In that case, you can set up personalized filters to streamline your search.

Conclusion

Building a data interception extension provides a window into the often-hidden data exchanges that power modern web applications. By combining the browser extension APIs with JavaScript monkey patching, we can create powerful tools for development, debugging, and analysis.

This extension serves as a starting point - it could be extended to filter specific domains, transform data, or even modify requests before they're sent.