Now for a closer look. Tessellation changes the shader pipeline landscape in some important ways. First, it adds two new shader stages called tessellation control and evaluation right after the vertex shader. If shaders are bound to each stage geometry data would flow through the shader pipeline in the following order:
Vertex -> Tessellation Control -> Evaluation -> Geometry -> Fragment -> (framebuffer)
The tessellation stages operate on a patch of geometry which consists of a collection of vertices and per-vertex attributes as well as per-patch attributes. You can control the number of vertices in a patch directly through the API. Tessellation control shaders are run once for each vertex in a patch and have access to the attributes in all other vertices for the patch. The shader then can output a new vertex modified based on patch information. Tess control shaders can access uniforms and textures in much the same way as a vertex shader can. This shader also controls the level of tessellation to be performed. Tessellation control shaders are optional.
The resulting patch, either directly output by the vertex shader or processed by the tess control shader if one is present, is then tessellated by the hardware primitive generator. Input geometry is tessellated according to the specified tessellation level. For instance, an incoming triangle will generate concentric smaller triangles inside the original. Then the space between each concentric triangle is divided into smaller triangles. Quads are handled similarly.
Evaluating Tessellation Results
Evaluation shaders are required for tessellation to function. This shader runs on a per vertex basis on each vertex in a patch, but can determine how many vertices are in the patch and what the primitive ID for the vertex is. Additionally, the evaluation shader can also access the location of the new vertex within the patch as well as the tessellation levels. When complete, the evaluation shader outputs vertex attributes to be processed by the subsequent pipeline shaders.
Tessellation shaders are a great tool for enhancing geometry while incurring little additional overhead. They can significantly augment scene geometry while not affecting system bandwidth requirements or requiring significant increases in geometry storage. For more details, you can take a look at the OpenGL 4.0 specification or the ARB_tessellation_shader extension.