Home

Creating a ray-marching renderer in Three.js

In this post, we'll dive into the exciting world of ray marching - a powerful rendering technique that lets you create stunning visuals by shooting rays from each pixel and evaluating a distance function along their path. To make all this possible, we will start by creating a simple renderer to help you understand how it works and how to build your own. Click here to see the final result of this tutorial.

OK but, what is ray marching exactly?

A simple definition: Ray marching is a method of rendering a scene by shooting rays from each pixel and determining if the ray collided with an object by evaluating a distance function along the ray's path.

The following sections will showcase how I applied this rendering technique in ThreeJs. If you want to deep dive into what ray marching is, I recommend watching these two videos:

For this post, I'll assume you have some basic knowledge in ThreeJs and shaders, but if you are new to this, I recommend you check out this video first:

1 - Project setup

This repository has a basic Vite setup with ThreeJs with a custom shader. To clone it, run:

git clone git@github.com:matiasngf/ray-marching-tutorial-starter.git

// OR 

git clone https://github.com/matiasngf/ray-marching-tutorial-starter.git

To start development, run:

npm i
npm run dev --port 3000

Go to http://localhost:3000 to see your app.

Let's take a look at the file /src/renderer.ts. I'd like to focus on three pieces of code:

In this chunk of code, we create a new shader pass and add it to the composer, setting up the variables (uniforms) that the shader will need to render properly:

// line 51
const rayMarchingPass = new ShaderPass({
  ...RayMarchingShader,
  uniforms: {
    cPos: { value: camera.position.clone() },
    resolution: {value: new Vector2(initDeviceSize.width, initDeviceSize.height)},
    cameraQuaternion: {value: camera.quaternion.clone()},
    fov: {value: camera.fov}
  }
});
composer.addPass(rayMarchingPass);

Then, on each frame, it updates the camera information on the shader:

// line 68
const worldPos = new Vector3();
rayMarchingPass.uniforms.cPos.value.copy(camera.getWorldPosition(worldPos));
const cameraQuaternion = new Quaternion();
rayMarchingPass.uniforms.cameraQuaternion.value.copy(camera.getWorldQuaternion(cameraQuaternion));
rayMarchingPass.uniforms.fov.value = camera.fov;

Finally, if the window resizes, it updates the resolution uniform of the shader:

// line 79
const windowResizeHanlder = () => {
  const { width, height, innerHeight, innerWidth } = getDeviceSize();
  renderer.setSize(innerWidth, innerHeight);
  composer.setSize(innerWidth, innerHeight);
  camera.aspect = innerWidth / innerHeight;
  camera.updateProjectionMatrix();
  rayMarchingPass.uniforms.resolution.value.set( width, height );
};
windowResizeHanlder();
window.addEventListener('resize', windowResizeHanlder);

Now that we know how the code provides variables to the shader, let's start coding. Open the file src/shaders/ray-marching.ts. You will see an empty string variable called fragmentShader. Add the following code to it:

uniform vec2 resolution;
uniform vec3 cPos;
uniform vec4 cameraQuaternion;
uniform float fov;

#define MAX_STEPS 200
#define SURFACE_DIST 0.01
#define MAX_DISTANCE 100.0

Let's see what each variable means:

  • resolution = Screen resolution in px.
  • cPos = Camera position.
  • cameraQuaternion = Camera rotation in quaternion.
  • fov = Field of view.
  • MAX_STEPS = How many steps each ray can handle per pixel.
  • SURFACE_DISTANCE = If a ray gets that close to a surface, it will be considered a hit.
  • MAX_DISTANCE = How much distance each ray can travel.

2 - Calculating distances

Ray-marching is about a ray that travels through a scene until it hits something. But how do we know when it hits something? We will first create a function called getDistance:

float getDistance(vec3 p) {
  // xPos, yPos, zPos, size
  vec4 sphere = vec4(0.0, 2.0, -0.0, 2.0);
  float dist_to_sphere = length(p - sphere.xyz) - sphere.w;
  return dist_to_sphere;
}

This function takes any point in space p and returns the distance from it to a sphere. We can use this function to know how far our ray is from the sphere's surface.

Adding more objects to the scene:

If we want to place more objects in the scene, we can use the min() function to return the minimum distance of two objects. Just for fun, let's add another sphere and a plane:

float getDistance(vec3 p) {

  vec4 sphere = vec4(0.0, 2.0, -0.0, 2.0);
  float dist_to_sphere = length(p - sphere.xyz) - sphere.w;

  vec4 sphere2 = vec4(3.0, 4.0, 0.0, 1.5);
  float dist_to_sphere2 = length(p - sphere2.xyz) - sphere2.w;

  float dist_to_plane = p.y;

  float d = min(dist_to_sphere, dist_to_plane);
  d = min(d, dist_to_sphere2);
  return d;
}

3 - Traversing the scene

Now that we have a getDistance function, we know how much our ray can travel without hitting any object. Also, if the distance between the ray and the surface is less than SURFACE_DIST, we can consider it a hit.

Using that information, we can create the rayMarch function:

// ro: ray origin
// rd: ray direction
// ds: distance to the surface
// d0: distance from origin
float rayMarch(vec3 ro, vec3 rd) {
  float d0 = 0.0;
  for(int i = 0; i < MAX_STEPS; i++) {
    // Calculate the ray's current position
    vec3 p = ro + d0 * rd;
    // Get the distance from p to the closest object in the scene
    float ds = getDistance(p);
    // Move the ray
    d0 += ds;
    // Evaluate if we need to break the loop
    if(ds < SURFACE_DIST || d0 > MAX_DISTANCE) break;
  }
  // Return the ray distance
  return clamp(d0, 0.0, MAX_DISTANCE);
}

This function will traverse the scene until it hits an object (or reaches the max distance), and it will just return the distance from the camera to the end of the ray. We can then use that distance to color our scene like a depth map. (We can add colors and lights, but more on that later).

4 - Shooting the rays

Now that we have our rayMarch function, we need to calculate a cameraOrigin and a rayDirection, in other words, placing our camera in the scene.

To make our life easy, we want to mimic our camera from ThreeJs. If you followed the project setup, you should have some basic variables defined at the top of your fragment shader.

We can now use those variables to calculate where each ray should go. The following code will calculate each pixel's cameraOrigin and rayDirection. Then, those variables will be used in the rayMarch function that will traverse the scene until it hits an object.

vec3 quaterion_rotate(vec3 v, vec4 q) {
  return v + 2.0 * cross(q.xyz, cross(q.xyz, v) + q.w * v);
}

void main() {
  float aspectRatio = resolution.x / resolution.y;
  vec3 cameraOrigin = cPos;

  float fovMult = fov / 90.0;
  
  vec2 screenPos = ( gl_FragCoord.xy * 2.0 - resolution ) / resolution;
  // Place the vector along the x-axis using the aspectRatio
  screenPos.x *= aspectRatio;
  // Move the vector using the field of view to match the ThreeCamera
  screenPos *= fovMult;
  vec3 ray = vec3( screenPos.xy, -1.0 );
  // Rotate the camera
  ray = quaterion_rotate(ray, cameraQuaternion);
  ray = normalize( ray );
  
  // Run the rayMarch function
  float d = rayMarch(cameraOrigin, ray);
  float normal_d = d / MAX_DISTANCE;

  gl_FragColor = vec4(vec3(normal_d), 1.0);
}

This function will use the returned distance to color the image, creating a depth field. The result should look like this:

And your code should look something like this.

🥳 Congratulations! You just rendered your first scene using ray-marching. This is a solid start. There are some other topics that you can research to expand your knowledge:

  • Colors
  • Boolean functions
  • Lights and normals
  • Shadows
  • Reflections

Most of those are experiments on my blog; you can see them at: https://matiasgf.dev.

I'd like to thanks the following people for their feedback: @celescript, @motiontx, @sebadom