Why Ray Tracing is Slow (and Why We’re About to Break It)

Day 1: Breaking Ray Tracing

DAY 1: Of Breaking Ray Tracing Forever.

Ray tracing has long been heralded as the pinnacle of achieving photorealistic lighting in computer-generated imagery (CGI), gaming, and visual effects (VFX). This sophisticated technique meticulously simulates the complex interactions of light with objects, tracing the paths of countless rays as they bounce, refract, and scatter throughout a scene. While this process is undeniably fascinating and produces stunning visuals, it comes with a significant drawback: it is incredibly slow.

The Problem with Traditional Ray Tracing

To fully grasp the challenges associated with traditional ray tracing, let's explore the standard pipeline in detail:

1. Emission of Rays: A light source emits rays in every conceivable direction, setting the stage for a complex interplay of light and shadow. This initial step is crucial for creating the foundation of the scene's lighting. The sheer number of rays emitted can be overwhelming, as each ray represents a potential path that light can take through the scene.

2. Interaction with Surfaces: As these rays traverse the scene, they encounter various surfaces, where they may bounce, refract, or be absorbed, depending on the material properties. Each interaction adds layers of complexity to the rendering process. The behavior of light at each surface is governed by the material's properties, such as its reflectivity, transparency, and texture.

3. Image Formation: The camera captures the rays that eventually reach it, piecing together the final image pixel by pixel. This step involves solving intricate mathematical equations to determine the color and intensity of each pixel. The rendering equation, which we'll delve into shortly, plays a central role in this process.

At the heart of this process lies the rendering equation, a mathematical model that describes the behavior of light:

$$L_o(p, \omega_o) = L_e(p, \omega_o) + \int_{\Omega} f_r(p, \omega_i, \omega_o) L_i(p, \omega_i) (\omega_i \cdot n) \, d\omega_i$$

In this equation:

𝐿ₒ represents the outgoing radiance at a point (p) in the direction 𝜔ₒ. This is the light that ultimately reaches the camera and contributes to the final image.

• 𝐿ₑdenotes the emitted radiance, accounting for any light that originates from the surface itself. This is particularly relevant for surfaces that emit light, such as light bulbs or glowing objects.

• 𝑓ᵣ is the bidirectional reflectance distribution function (BRDF), which characterizes how light is reflected at an opaque surface, influencing the appearance of materials. The BRDF is a complex function that depends on the material's properties and the angles of incoming and outgoing light.

• 𝐿ᵢ is the incoming radiance from direction 𝜔ᵢ, representing the light arriving at the surface from various directions. This includes light that has been reflected or refracted from other surfaces in the scene.

• (n) is the surface normal, a vector perpendicular to the surface at point (p), essential for calculating the angle of incidence and reflection. The surface normal plays a crucial role in determining how light interacts with the surface.

This integral must be solved for every single pixel in the image, a task that quickly becomes computationally expensive and time-consuming. The complexity of the rendering equation is a major factor contributing to the slow performance of traditional ray tracing.

The Brutal Cost of Ray Tracing

Consider the following scenario:

• 1 million rays per frame: A typical figure for high-end ray tracing applications, necessary to achieve detailed and realistic lighting effects. Each ray represents a potential path that light can take through the scene, and the sheer number of rays contributes to the computational load.

• Average of 50 bounces per ray: Each ray interacts with multiple surfaces, compounding the complexity and increasing the computational load. The number of bounces determines how many times a ray can interact with surfaces before it is absorbed or exits the scene.

• 60 frames per second (FPS) target: The standard for smooth, real-time rendering, essential for immersive experiences in gaming and interactive media. Achieving this frame rate requires solving the rendering equation for each frame in a fraction of a second.

In this setup, the GPU is tasked with solving the rendering equation 50 million times per frame and doing so 60 times per second. This amounts to a staggering 3 billion equations per second. 🤯

Even with the advent of RTX hardware acceleration, the computational cost remains exorbitant. This is why real-time ray tracing often relies on aggressive denoising techniques to approximate the desired visual fidelity, sacrificing some accuracy for performance. Denoising algorithms attempt to reduce the noise in the rendered image, which is a byproduct of the limited number of rays and bounces used in real-time applications.

The RTX Bottleneck

RTX graphics cards employ bounding volume hierarchies (BVH) to expedite the ray tracing process. BVH is a spatial data structure that organizes objects within a scene, allowing for more efficient ray-object intersection tests. By grouping objects into hierarchical bounding volumes, BVH reduces the number of intersection tests required, speeding up the rendering process. However, BVH traversal, while optimized, is still a brute-force method at its core. We can do better.

The issue isn't with ray tracing itself; it's with the methodology we use to trace rays. The current approach, though effective, is not the most efficient way to achieve the desired results. The reliance on brute-force methods and the sheer number of calculations required are significant bottlenecks.

The Plan: Reinventing Ray Tracing

Imagine a world where, instead of relying on brute force, we could predict the optimal paths for rays to take. What if we harnessed the power of physics and artificial intelligence to bypass the laborious number-crunching and achieve real-time lighting computation?

That's precisely the direction we're heading. By leveraging advanced algorithms and machine learning techniques, we aim to revolutionize the way light is simulated in digital environments. Machine learning models can be trained to predict the behavior of light in complex scenes, reducing the need for exhaustive calculations.

In the upcoming posts, we'll embark on a journey to revolutionize the mathematics of light simulation—eschewing shortcuts and hacks in favor of a comprehensive rewrite. Prepare yourself for an exhilarating exploration of cutting-edge techniques that promise to redefine the future of ray tracing. We'll delve into the potential of neural networks to predict light paths, explore the use of real-time physics simulations, and examine how these innovations can be integrated into existing rendering pipelines.

Stay tuned, because this is going to be nothing short of extraordinary. The future of ray tracing is bright, and we're on the cusp of a new era in digital lighting. As we continue to push the boundaries of what's possible, we invite you to join us on this exciting journey. Together, we'll explore the possibilities of a world where real-time ray tracing is not just a dream, but a reality.