Running a Game Boy Emulator Smoothly on HTML Canvas

A Game Boy Emulator (or any game system emulator for that matter) is able to understand a game that has been loaded into memory through a main loop that steps through each opcode one at a time. The program counter points to a location in memory, and the CPU processes the correct opcode by reading bytes from this location. Furthermore, as the CPU steps through each opcode, the GPU is also doing work and needs to stay in sync with the CPU.

For example, In Rust, I used minifb to give me a simple interface where I could easily render a frame buffer. The following code shows a simplified example of what this kind of loop utilizing minifb would look like:

const WIDTH: usize = 160;
const HEIGHT: usize = 144;

fn open_gameboy_emulator_window(emulator: &mut Emulator) {
    let mut window = Window::new(
        "Game Boy Emulator",
        WIDTH,
        HEIGHT,
        WindowOptions {
            resize: true,
            ..WindowOptions::default()
        },
    )
    .expect("Unable to create window");

    while window.is_open() && !window.is_key_down(Key::Escape) {
        let minifb_renderer = |buffer: &Vec<u32>| {
            window
                .update_with_buffer(&buffer, WIDTH, HEIGHT)
                .expect("Unable to update window"); 
        };

        step(emulator, minifb_renderer);
    }
}

Where step is defined as:

pub fn step(emulator: &mut Emulator, render: impl FnMut(&Vec<u8>)) {
    cpu::opcodes::step(emulator);
    gpu::step(emulator, render);
}

So, each loop iteration will step the CPU and GPU once. A render closure is passed into the GPU, and when the timing is correct, the GPU will call render, passing the current frame buffer as a parameter. Then with minifb it's a simple matter of calling window.update_with_buffer(...).

This works great with minifb.

But then I was feeling ambitious. I asked myself, could I compile this code down to WebAssembly using wasm-pack and then run my emulator directly in the browser?

I hit a little roadblock when I attempted this task. I created a function that could be called directly by the Javascript app which originally looked something like this:

#[wasm_bindgen]
extern "C" {
    pub fn render(frame_buffer: &[u8]); 
}

#[wasm_bindgen]
pub fn start_emulator() {
    EMULATOR.with(|emulator_cell| {
        let mut emulator = emulator_cell.borrow_mut();

        loop {
            emulator::step(&mut emulator, |buffer: &Vec<u8>| {
                render(buffer.as_slice());
            });
        }
    })
}

I define an external render function which calls a function in my Typescript app that looks something like this:

const renderFrame = (
    canvasContext: CanvasRenderingContext2D,
    buffer: number[],
): void => {
    const data = new Uint8ClampedArray(buffer);
    const imageData = new ImageData(data, GAMEBOY_WIDTH, GAMEBOY_HEIGHT);
    canvasContext.putImageData(imageData, 0, 0);
};

window.render = (buffer: number[]): void => {
    renderFrame(canvasContext, buffer);
};

It just takes the buffer and converts it to a Uint8ClampedArray, then puts the image data onto the canvas. Straightforward, right?

There's a problem with this code.

The browser does all of its work through a single main thread which renders the webpage, handles user events, executes your Javascript, and more. This means that long running Javascript can actually freeze the user interface and render your app useless.

For a while, I was stuck on this issue. I understood the problem fundamentally, but I couldn't think of any good ideas to solve it. Web Workers can help in these kinds of situations, but I learned that it is hard to get it to play well with WebAssembly.

So what's the simplest solution?

A function in Javascript called window.requestAnimationFrame.

This function accepts a callback function as a parameter, and this callback is called before each repaint, at a rate that generally matches your display's refresh rate, typically 60Hz.

The trick is the callback passed to window.requestAnimationFrame should only step one frame. The callback will automatically be called before each repaint, so there is no need to step more than one frame within the callback.

Here's an example of how I did it in Rust:

#[wasm_bindgen]
extern "C" {
    pub fn render(frame_buffer: &[u8]); 
}

#[wasm_bindgen(js_name = stepFrame)]
pub fn step_frame() {
    EMULATOR.with(|emulator_cell| {
        let mut emulator = emulator_cell.borrow_mut();

        let mut frame_rendered = false;

        while !frame_rendered {
            emulator::step(&mut emulator, |buffer: &Vec<u8>| {
                render(buffer.as_slice());
                frame_rendered = true;
            });
        }
    })
}

Using a mutable variable, we can track when we render the frame. Then, we simply exit the loop once the mutable variable is set to true.

Then inside Typescript we can do something like this:

const renderLoop = (): void => {
    stepFrame();
    window.requestAnimationFrame(() => renderLoop());
};

This way, we don't freeze the user interface.