Instancing

Imagine a scene with a great many copies of the same mesh: a meadow of grass blades, a forest, a crowd, a field of rocks. Each copy is the same vertex data, only placed, turned and scaled differently. The obvious way to draw them is a loop:

for each rock:
    write its model matrix into the constant buffer
    Draw(rock's vertex count)

It works, and for a few hundred objects it works fine. But with tens of thousands it grinds to a halt, and not because the GPU is short of power. Each draw call costs CPU time: the runtime and the driver check state, update the constant buffer, and package the work up for the GPU. For a small mesh of a few hundred triangles, that overhead is far larger than the time the GPU needs to actually draw it, so the GPU spends most of the frame waiting for the next instruction. The fix is to hand the GPU all the copies in one go.

That is instancing: one draw call that draws the same mesh many times. In Direct3D the call is DrawInstanced (and DrawIndexedInstanced for indexed meshes), OpenGL's glDrawArraysInstanced. It takes the same arguments as Draw, plus the number of instances to draw:

DrawInstanced(vertex count per instance, instance count, first vertex, first instance)

On its own, that draws the same mesh a thousand times in the same place, which looks exactly like drawing it once. Each copy needs to know which copy it is. HLSL provides that as a system value, SV_InstanceID (GLSL's gl_InstanceID): an unsigned integer the vertex shader can take as an input, counting from 0 for the first instance up to the instance count minus one.

With it, a vertex shader can look up per-copy data in an array. LearnOpenGL starts with a grid of a hundred small coloured quads, each moved by an offset from a uniform array. In HLSL that would be a constant buffer (this is a sketch, not part of our lesson):

cbuffer grid : register(b1)
{
    float4 offsets[100];   // only xy used; each array element takes a register anyway
};

quad_vertex vs_main(float2 position : POSITION, float3 color : COLOR, uint instance : SV_InstanceID)
{
    quad_vertex o;
    o.position = float4(position + offsets[instance].xy, 0.0, 1.0);
    o.color = color;
    return o;
}

and one DrawInstanced(6, 100, 0, 0) draws all hundred quads. Note the float4: as Advanced HLSL explained, every element of a constant buffer array starts a new 16-byte register, so a float2 offsets[100] would take nearly as much space, with a gap after every element, and the CPU side would have to match the gaps.

Instanced vertex data

The array approach has a limit: a constant buffer holds at most 64 KB, 4096 registers. That is 1024 model matrices, and real scenes want more. The more flexible tool, which LearnOpenGL calls instanced arrays, is to put per-copy data in a vertex buffer, and tell the input assembler to step through it once per instance instead of once per vertex.

In OpenGL that is an ordinary vertex attribute with glVertexAttribDivisor(attribute, 1): advance this attribute once every instance. Direct3D says it in the input layout. Each D3D11_INPUT_ELEMENT_DESC has two fields, the last two, that we have always set to per-vertex and zero:

Per-instance and per-vertex data cannot share a slot, since a slot has one stride and one rate of stepping, so per-instance data lives in its own buffer, in its own slot. This is the multi-slot binding from Advanced Data, put to a new use. Within the vertex shader, a per-instance input looks like any other; the shader does not know how often it changes.

An asteroid field

Instancing really shows its worth with large numbers, so our lesson goes straight to LearnOpenGL's main example: a planet in the middle of a ring of rocks. Our ring has twenty thousand of them.

A planet surrounded by a wide ring of twenty thousand rocks

Both meshes are loaded with the model loader from Model, model_load in model.rasm. The last argument flips their texture coordinates, since these two .obj files assume v grows upward, as OpenGL's does:

; these two .obj files expect v up the image, so v is flipped
invoke(model_load, planet&, "resources/objects/planet/", "planet.obj", 1)
invoke(model_load, rock&, "resources/objects/rock/", "rock.obj", 1)

Placing the rocks

Each rock gets a model matrix, made once at startup. The rocks are spread evenly by angle around a circle of radius 45, then nudged off the circle by a random amount up to 6 units in x and z, and less in y, so that the ring is wide and flat. Each gets a random size and a random spin around a tilted axis. place_rocks does it for all twenty thousand, writing each matrix through rsi into a global array of them, one mat4 per rock; here is the start of its loop, the x coordinate:

#define AMOUNT 20000
matrices: [AMOUNT]mat4 = {}
    invoke(srand, 7)
    lea rsi, matrices
    xor ebx, ebx
  rocks:
    ; angle = i / AMOUNT * 2 * pi
    cvtsi2ss xmm0, ebx
    divss xmm0, 20000.0
    addss xmm0, xmm0
    mulss xmm0, math_pi
    movss place_rocks_angle, xmm0
    ; x = sin(angle) * 45 + (random * 2 - 1) * 6
    call sinf
    mulss xmm0, 45.0
    movss place.x, xmm0
    call random_signed
    mulss xmm0, 6.0
    addss xmm0, place.x
    movss place.x, xmm0
; ...

The array's size is known before the program runs, so it is declared at that size rather than allocated, and the debugger shows it matrix by matrix. The float constants, 20000.0, 45.0, 6.0, are literals that rasm puts in read-only memory for the instructions to read, and the angle, kept across the calls, is a global named after the proc, place_rocks_angle.

y and z follow the same pattern, then the size and the spin, and the matrix is built in the usual order: scale first, then spin, then move into place.

    ; matrices[i] = translate(x, y, z) * spin * scale
    invoke(m4_identity, scale&)
    movss xmm0, place_rocks_scale
    movss scale.m[0], xmm0
    movss scale.m[5], xmm0
    movss scale.m[10], xmm0
    invoke(m4_mul, spin&, spin&, scale&)
    invoke(m4_translate, rsi, place&)
    invoke(m4_mul, rsi, rsi, spin&)
    add rsi, sizeof(mat4)
    inc ebx
    cmp ebx, AMOUNT
    jb rocks

scale.m[5] is the sixth float of the matrix, the second entry of its diagonal: an index on a global counts elements. AMOUNT is a #define, an immediate here. The random numbers come from rand, seeded with a fixed value so that every run makes the same ring, and turned into a float between 0 and 1 the way LearnOpenGL does, rand() % 10000 / 10000:

; -> xmm0: a random number in 0..1, in steps of 1 / 10000
random01: proc() -> f32 = {
    sub rsp, 40                                ; 32 shadow, and rsp back on 16
    call rand
    xor edx, edx
    mov ecx, 10000
    div ecx
    cvtsi2ss xmm0, edx
    divss xmm0, 10000.0
    add rsp, 40
    ret
}

div divides edx:eax by its operand, leaving the quotient in eax and the remainder, the %, in edx; hence edx is cleared first, and converted to a float after. Even a proc this small makes the 40-byte frame of any proc that calls another: the callee may use the 32 bytes of shadow space above its return address, and rsp must be on 16 at the call.

The lessons call nothing from the C runtime, so srand and rand live in math.rasm, written to match the Microsoft C runtime's exactly: the same linear congruential generator, \(s = 214013\,s + 2531011\), returning bits 16 to 30 of the state. Seeded with 7, it gives the same sequence as the C and rin versions, so the ring comes out rock for rock the same as theirs.
; -> eax: 0..32767
rand: proc() -> i32 = {
    imul eax, math_rand_state, 214013
    add eax, 2531011
    mov math_rand_state, eax
    shr eax, 16
    and eax, 0x7FFF
    ret
}

LearnOpenGL first draws this ring the slow way, with one draw call per rock, to show how quickly the frame rate collapses as the count rises. Our lesson skips straight to the instanced version, but you can feel the difference yourself in the exercises.

The instance buffer

All twenty thousand matrices go into one more vertex buffer, 64 bytes each, 1.28 MB in all. They never change, so it is immutable:

; every rock's matrix, in one vertex buffer read once per instance
call place_rocks
invoke(create_buffer, D3D11_BIND_FLAG.D3D11_BIND_VERTEX_BUFFER, matrices&, sizeof(matrices))
mov instance_buffer, rax

Now the input layout. The first three elements are the model's usual vertex format, from slot 0, as model.rasm's model_layout has them. Then comes the matrix. One input element can hold at most four 32-bit components, a float4, so a 4x4 matrix does not fit in one. As in OpenGL, where a mat4 attribute occupies four consecutive locations, we send it as four float4 elements, each 16 bytes further into the instance's 64:

#define INSTANCE_LAYOUT_COUNT 7

; model_layout's three per-vertex elements from slot 0, then a matrix's four
; rows from slot 1, stepping once per instance
instance_layout: [INSTANCE_LAYOUT_COUNT]const D3D11_INPUT_ELEMENT_DESC = {
    { .SemanticName = "POSITION", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32_FLOAT, .AlignedByteOffset = 0 },
    { .SemanticName = "NORMAL", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32_FLOAT, .AlignedByteOffset = 12 },
    { .SemanticName = "TEXCOORD", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32_FLOAT, .AlignedByteOffset = 24 },
    { .SemanticName = "INSTANCE", .SemanticIndex = 0, .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32A32_FLOAT, .InputSlot = 1, .AlignedByteOffset = 0,
      .InputSlotClass = D3D11_INPUT_CLASSIFICATION.D3D11_INPUT_PER_INSTANCE_DATA, .InstanceDataStepRate = 1 },
    { .SemanticName = "INSTANCE", .SemanticIndex = 1, .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32A32_FLOAT, .InputSlot = 1, .AlignedByteOffset = 16,
      .InputSlotClass = D3D11_INPUT_CLASSIFICATION.D3D11_INPUT_PER_INSTANCE_DATA, .InstanceDataStepRate = 1 },
    { .SemanticName = "INSTANCE", .SemanticIndex = 2, .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32A32_FLOAT, .InputSlot = 1, .AlignedByteOffset = 32,
      .InputSlotClass = D3D11_INPUT_CLASSIFICATION.D3D11_INPUT_PER_INSTANCE_DATA, .InstanceDataStepRate = 1 },
    { .SemanticName = "INSTANCE", .SemanticIndex = 3, .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32A32_FLOAT, .InputSlot = 1, .AlignedByteOffset = 48,
      .InputSlotClass = D3D11_INPUT_CLASSIFICATION.D3D11_INPUT_PER_INSTANCE_DATA, .InstanceDataStepRate = 1 },
}

Each element names the fields it sets, with the SDK's own field names; the ones left out are zero, which for the first three means slot 0, semantic index 0, per-vertex data. This is the first time we have used a semantic index, SemanticIndex. One semantic name can be used several times, told apart by its index, and in HLSL the index is written as a number on the end of the name: the four elements are INSTANCE0 to INSTANCE3 in the shader. (TEXCOORD is really TEXCOORD0; a second set of texture coordinates would be TEXCOORD1.) Each of the four reads slot 1, per instance, advancing by one element every instance. The shader is created with this layout, seven elements:

invoke(shader_create, rock_shader&, "src/4.advanced_d3d11/10.1.instancing/asteroids.hlsl", instance_layout&, INSTANCE_LAYOUT_COUNT)

The shader

The rock's vertex shader takes the four rows and puts the matrix back together:

struct rock_in
{
    float3 position : POSITION;     // per vertex, slot 0
    float3 normal : NORMAL;
    float2 uv : TEXCOORD;
    float4 row0 : INSTANCE0;        // per instance, slot 1
    float4 row1 : INSTANCE1;
    float4 row2 : INSTANCE2;
    float4 row3 : INSTANCE3;
};
...
vs_out vs_main(rock_in v)
{
    float4x4 placement = float4x4(v.row0, v.row1, v.row2, v.row3);
    float4x4 m = mul(model, placement);
    vs_out o;
    o.position = mul(projection, mul(view, mul(m, float4(v.position, 1.0))));
    // only rotations and uniform scales here, so the model matrix itself
    // keeps normals pointing the right way
    o.normal = mul((float3x3)m, v.normal);
    o.uv = v.uv;
    return o;
}

They are rows because math.rasm's mat4 stores its sixteen floats row by row, so the first 16 bytes of each instance are the first row, and HLSL's float4x4 constructor takes its vectors as rows. (The pack_matrix pragma does not come into it: it applies to matrices read from constant buffers, and this one is assembled from vertex inputs.)

The model matrix from the constant buffer is still there, and applies to the whole ring at once: multiplying it in front of each rock's own placement turns the entire ring slowly around the planet with one matrix per frame. LearnOpenGL's rocks hold still; ours drift.

The normal is transformed by the 3x3 part of the combined model matrix rather than by a proper normal matrix. That is only correct because every matrix here is made of rotations, translations and uniform scales, which leave angles alone; a rock stretched along one axis would need the inverse transpose, as in Basic Lighting. Computing an inverse per vertex would be costly, which is one more reason to keep instance scales uniform, or to store a normal matrix per instance as well.

Drawing

The planet is an ordinary model, drawn the ordinary way with model_draw, with its own shader. Then the rocks: the ring's rotation goes into the constant buffer, the rock shader is bound, and we bind two vertex buffers at once, the rock's mesh in slot 0 with its per-vertex stride, and the matrices in slot 1 with a stride of one mat4. The three arrays IASetVertexBuffers wants are globals, the strides initialised with the sizes of the two types, the buffers filled in once at startup:

rock_buffers: [2]*ID3D11Buffer = {}            ; slot 0 the rock's vertices, slot 1 the matrices
rock_strides: [2]u32 = { sizeof(model_vertex), sizeof(mat4) }
rock_offsets: [2]u32 = { 0, 0 }
mov rax, rock.vertices
mov rock_buffers[0], rax
mov rax, instance_buffer
mov rock_buffers[1], rax
; the rocks: one draw, the whole ring turning slower still
movss xmm1, current_frame
mulss xmm1, 0.02
lea rcx, t.model
lea r8, y_axis
call m4_rotate
invoke(update_buffer, transforms_buffer, t&, sizeof(transforms))
invoke(shader_use, rock_shader&)
; rock.mtl names its only texture as map_Bump, so that is the one to show
invoke(bind_texture, 0, rock.normal)
vcall(context, ID3D11DeviceContextVtbl.IASetVertexBuffers, 0, 2, rock_buffers&, rock_strides&, rock_offsets&)
mov edx, rock.vertex_count                 ; a u32: into its argument register first
vcall(context, ID3D11DeviceContextVtbl.DrawInstanced, rdx, AMOUNT, 0, 0)

One call, and twenty thousand rocks. The vertex count is a 32-bit field of the model, and vcall moves every argument as 64 bits, so the count is loaded into edx, the register it travels in, and passed as rdx. For every instance the input assembler walks through all of the rock's vertices in slot 0, while slot 1 stays on that instance's matrix. The draw is written out by hand rather than going through model.rasm's model_draw_instanced, which would bind only the model's own vertex buffer, and its diffuse texture into t0: the rock's material file happens to list its only image, the colour, as a bump map (map_Bump), so the loader filed it under normal, and the lesson binds that instead. The topology needs no setting here; model_draw for the planet left it at triangle list.

The camera starts high and far back, at (0, 12, 70), and moves at 15 units a second rather than the usual 2.5, since the ring is about a hundred units across:

movss xmm0, 15.0
movss cam.speed, xmm0
movss xmm0, 50.0
movss cam.zoom, xmm0

The far plane is at 500 for the same reason, so the lesson builds its projection itself rather than with camera_projection, whose far plane is 100. m4_perspective takes five arguments, four of them floats: the field of view, aspect and near plane go in xmm1 to xmm3, by position, and the fifth, the far plane, on the stack, just above the 32 bytes of shadow space:

; the ring is large: depth out to 500
invoke(camera_view, t.view&, cam&)
movss xmm0, 500.0
movss dword ptr [rsp + 32], xmm0           ; z_far, the fifth argument
movss xmm0, cam.zoom
call radians
movaps xmm1, xmm0                          ; fovy
cvtsi2ss xmm2, scr_width
cvtsi2ss xmm0, scr_height
divss xmm2, xmm0                           ; aspect
movss xmm3, 0.1                            ; z_near
lea rcx, t.projection
call m4_perspective

[rsp + 32] is the first stack argument's place in main's own frame, sub rsp, 56, which is sized for IASetVertexBuffers' two stack arguments and so has room for this one too.

Fly through the ring and around it.

How far this goes depends on the hardware, but the shape of the result is always the same: the per-draw cost on the CPU is paid once instead of twenty thousand times, and what remains is the GPU's actual work of drawing triangles, which it is very good at. The same technique scales to grass, foliage, crowds and particles. The instance data does not have to be static, either: put it in a dynamic buffer and rewrite it each frame with Map, and every copy can move independently.

The full program is here, the rock shader here and the planet's here.

Additional resources

Exercises

  1. Draw the rocks the slow way: a loop of twenty thousand ordinary draws, each writing its matrix into the constant buffer, with a shader that reads the model matrix from there. Compare the frame rate, then try smaller and larger values of AMOUNT in both versions.
  2. Use SV_InstanceID in the rock's vertex shader to tint the rocks: for example, a slightly different shade for every fifth rock.
  3. Make the rocks orbit at different speeds, faster near the inner edge: move the matrix construction into a dynamic instance buffer rewritten every frame, or compute the angle in the vertex shader from a per-instance radius and phase.
Credit: this chapter follows LearnOpenGL's Instancing by Joey de Vries, rewritten for Direct3D 11 and x64 assembly in rasm. LearnOpenGL's text and code are licensed CC BY-NC 4.0, and so is this page; the wording, code and screenshots here are new, and diagrams marked as LearnOpenGL's are its own.