Stalls part two: beware of SetData

Originally posted to Shawn Hargreaves Blog on MSDN, Tuesday, April 15, 2008

ajmiles got it in one.

This harmless looking code:

    vertexBuffer.SetData(particlePositions);
    graphicsDevice.VertexBuffer = vertexBuffer;
    graphicsDevice.DrawPrimitives(...);

will cause a pipeline stall if the CPU reaches the SetData call for frame #2 before the GPU finishes processing the DrawPrimitives call from frame #1.

This occurs because resources such as vertex buffers and textures are passed by reference, not by value. You can think of each resource as a separate piece of paper filled with data:

• During LoadContent(), Charles writes 1 million numbers on a piece of paper
• He labels this paper #42, then sets it to one side
• During Draw(), Charles encounters a "render terrain" instruction
• He writes "draw 1 million triangles, using position data from paper #42" on his "instructions for George" list

This is efficient, because Charles does not have to bother touching the actual vertex data while processing the draw instruction.

But what if the game later calls SetData on this same vertex buffer?

• Charles encounters an instruction that says "erase paper #42, then write these new numbers onto it"
• Before he can proceed, Charles must check if there are any references to the previous data still waiting in his "instructions for George" list
• Even if George has already read the instructions, Charles must interrupt him and say "hey, are you finished with paper #42 yet? Can I have it back please?"

If the GPU is still using the resource, the second SetData call must stall until it has finished.

There are several ways to avoid such a stall:

• Treat vertex buffers and textures as read-only: do not call SetData outside your load methods. Beginners often think they need dynamic SetData in places where it would be more efficient to leave the source data alone and apply dynamic effects via a vertex or pixel shader.

• If you are generating new vertex data every frame, use DrawUserPrimitives instead of DrawPrimitives. This does not use a vertex buffer: instead, the vertex data is copied directly into the main "instructions for George" list, avoiding any possibility of a stall. Think of it as pass-by-value instead of pass-by-reference.

• Double-buffer. Create a pair of identical resources, and alternate which one you use per frame. This gives the GPU more time to finish with each resource before the next time you SetData on it. Good for dynamic texture scenarios such as video playback.

• Use SetDataOptions.NoOverwrite. This tells Charles "I know you would normally have to wait for George here, but please just carry on regardless. I promise I won't overwrite any part of the vertex buffer that he might still be using". You must then keep track of which specific vertices have recently been drawn, and only modify parts of the buffer once you are 100% sure the GPU has finished with them. We used this technique in our GPU particle system: check out the monster comment in ParticleSystem.cs for an explanation.