Texture mapping

Texture mapping^[1][2][3] is a method for mapping a texture on a computer-generated graphic. Texture here can be high frequency detail, surface texture, or color.

History

The original technique was pioneered by Edwin Catmull in 1974 as part of his doctoral thesis.^[4]

By 1983 work done by Johnson Yan, Nicholas Szabo, and Lish-Yaan Chen in their invention "Method and Apparatus for Texture Generation" provided for texture generation in real time where texture could be generated and superimposed on surfaces (curvilinear and planar) of any orientation. Texture patterns could be modeled suggestive of the real world material they were intended to represent in a continuous way and free of aliasing, ultimately providing level of detail and gradual (imperceptible) detail level transitions. Texture generating became repeatable and coherent from frame to frame and remained in correct perspective and appropriate occultation.^[5] Because the application of real time texturing was applied to early three dimensional flight simulator CGI systems, many of these techniques were later widely used in graphics computing and gaming and applications for years to follow as Texture was often the first prerequisite for realistic looking graphics.

Also in 1983, in a paper "Pyramidial Parametrics", Lance Williams, another graphics pioneer introduced the concept of mapping images onto surfaces to increase the realism of such images.^[6]

Texture mapping originally referred to diffuse mapping, a method that simply mapped

A texture map

They may have 1-3 dimensions, although 2 dimensions are most common for visible surfaces. For use with modern hardware, texture map data may be stored in

They usually contain

The way that samples (e.g. when viewed as pixels on the screen) are calculated from the texels (texture pixels) is governed by texture filtering. The cheapest method is to use the nearest-neighbour interpolation, but bilinear interpolation or trilinear interpolation between mipmaps are two commonly used alternatives which reduce aliasing or jaggies. In the event of a texture coordinate being outside the texture, it is either clamped or wrapped. Anisotropic filtering better eliminates directional artefacts when viewing textures from oblique viewing angles.

Texture streaming

Texture streaming is a means of using data streams for textures, where each texture is available in two or more different resolutions, as to determine which texture should be loaded into memory and used based on draw distance from the viewer and how much memory is available for textures. Texture streaming allows a rendering engine to use low resolution textures for objects far away from the viewer's camera, and resolve those into more detailed textures, read from a data source, as the point of view nears the objects.

Baking

As an optimization, it is possible to render detail from a complex, high-resolution model or expensive process (such as

Baking can be used as a form of

Rasterisation algorithms

Various techniques have evolved in software and hardware implementations. Each offers different trade-offs in precision, versatility and performance.

Affine texture mapping

Affine texture mapping linearly interpolates texture coordinates across a surface, and so is the fastest form of texture mapping. Some software and hardware (such as the original

For the case of rectangular objects, using quad primitives can look less incorrect than the same rectangle split into triangles, but because interpolating 4 points adds complexity to the rasterization, most early implementations preferred triangles only. Some hardware, such as the forward texture mapping used by the Nvidia NV1, was able to offer efficient quad primitives. With perspective correction (see below) triangles become equivalent and this advantage disappears.

For rectangular objects that are at right angles to the viewer, like floors and walls, the perspective only needs to be corrected in one direction across the screen, rather than both. The correct perspective mapping can be calculated at the left and right edges of the floor, and then an affine linear interpolation across that horizontal span will look correct, because every pixel along that line is the same distance from the viewer.

Perspective correctness

Perspective correct texturing accounts for the vertices' positions in 3D space, rather than simply interpolating coordinates in 2D screen space.^[13] This achieves the correct visual effect but it is more expensive to calculate.^[13]

To perform perspective correction of the texture coordinates ${\displaystyle u}$ and ${\displaystyle v}$, with ${\displaystyle z}$ being the depth component from the viewer's point of view, we can take advantage of the fact that the values ${\displaystyle {\frac {1}{z}}}$, ${\displaystyle {\frac {u}{z}}}$, and ${\displaystyle {\frac {v}{z}}}$ are linear in screen space across the surface being textured. In contrast, the original ${\displaystyle z}$, ${\displaystyle u}$ and ${\displaystyle v}$, before the division, are not linear across the surface in screen space. We can therefore linearly interpolate these reciprocals across the surface, computing corrected values at each pixel, to result in a perspective correct texture mapping.

To do this, we first calculate the reciprocals at each vertex of our geometry (3 points for a triangle). For vertex ${\displaystyle n}$ we have ${\displaystyle {\frac {u_{n}}{z_{n}}},{\frac {v_{n}}{z_{n}}},{\frac {1}{z_{n}}}}$. Then, we linearly interpolate these reciprocals between the ${\displaystyle n}$ vertices (e.g., using barycentric coordinates), resulting in interpolated values across the surface. At a given point, this yields the interpolated ${\displaystyle u_{i},v_{i}}$, and ${\displaystyle zReciprocal_{i}={\frac {1}{z_{i}}}}$. Note that this ${\displaystyle u_{i},v_{i}}$ cannot be yet used as our texture coordinates as our division by ${\displaystyle z}$ altered their coordinate system.

To correct back to the ${\displaystyle u,v}$ space we first calculate the corrected ${\displaystyle z}$ by again taking the reciprocal ${\displaystyle z_{correct}={\frac {1}{zReciprocal_{i}}}={\frac {1}{\frac {1}{z_{i}}}}}$. Then we use this to correct our ${\displaystyle u_{i},v_{i}}$: ${\displaystyle u_{correct}=u_{i}\cdot z_{i}}$ and ${\displaystyle v_{correct}=v_{i}\cdot z_{i}}$.^[14]

This correction makes it so that in parts of the polygon that are closer to the viewer the difference from pixel to pixel between texture coordinates is smaller (stretching the texture wider) and in parts that are farther away this difference is larger (compressing the texture).

Affine texture mapping directly interpolates a texture coordinate ${\displaystyle u_{\alpha }^{}}$ between two endpoints ${\displaystyle u_{0}^{}}$ and ${\displaystyle u_{1}^{}}$:

${\displaystyle u_{\alpha }^{}=(1-\alpha )u_{0}+\alpha u_{1}}$ where ${\displaystyle 0\leq \alpha \leq 1}$

Perspective correct mapping interpolates after dividing by depth ${\displaystyle z_{}^{}}$, then uses its interpolated reciprocal to recover the correct coordinate:

${\displaystyle u_{\alpha }^{}={\frac {(1-\alpha ){\frac {u_{0}}{z_{0}}}+\alpha {\frac {u_{1}}{z_{1}}}}{(1-\alpha ){\frac {1}{z_{0}}}+\alpha {\frac {1}{z_{1}}}}}}$

3D graphics hardware typically supports perspective correct texturing.

Various techniques have evolved for rendering texture mapped geometry into images with different quality/precision tradeoffs, which can be applied to both software and hardware.

Classic software texture mappers generally did only simple mapping with at most one lighting effect (typically applied through a lookup table), and the perspective correctness was about 16 times more expensive.

Restricted camera rotation

The

which allowed the appearance of greater freedom whilst using the same rendering technique.

Some engines were able to render texture mapped

Nova Logic's Voxel Space, and the engine for Outcast) via Bresenham-like incremental algorithms, producing the appearance of a texture mapped landscape without the use of traditional geometric primitives.^[15]

Subdivision for perspective correction

Every triangle can be further subdivided into groups of about 16 pixels in order to achieve two goals. First, keeping the arithmetic mill busy at all times. Second, producing faster arithmetic results.^[vague]

World space subdivision

For perspective texture mapping without hardware support, a triangle is broken down into smaller triangles for rendering and affine mapping is used on them. The reason this technique works is that the distortion of affine mapping becomes much less noticeable on smaller polygons. The

Sony PlayStation

made extensive use of this because it only supported affine mapping in hardware but had a relatively high triangle throughput compared to its peers.

Screen space subdivision

Software renderers generally preferred screen subdivision because it has less overhead. Additionally, they try to do linear interpolation along a line of pixels to simplify the set-up (compared to 2d affine interpolation) and thus again the overhead (also affine texture-mapping does not fit into the low number of registers of the

RISC

is much more suited).

A different approach was taken for

polygon normal

to achieve a more constant z but the effort seems not to be worth it.

Other techniques

Another technique was approximating the perspective with a faster calculation, such as a polynomial. Still another technique uses 1/z value of the last two drawn pixels to linearly extrapolate the next value. The division is then done starting from those values so that only a small remainder has to be divided^[17] but the amount of bookkeeping makes this method too slow on most systems.

Finally,

the Build engine

extended the constant distance trick used for Doom by finding the line of constant distance for arbitrary polygons and rendering along it.

Hardware implementations

Texture mapping hardware was originally developed for simulation (e.g. as implemented in the

flight simulation, texture mapping provided important motion and altitude cues necessary for pilot training not available on untextured surfaces. It was also in flight simulation applications, that texture mapping was implemented for real-time processing with prefiltered texture patterns stored in memory for real-time access by the video processor.^[18]

Modern

fixed function units called texture samplers, or texture mapping units, to perform texture mapping, usually with trilinear filtering or better multi-tap anisotropic filtering and hardware for decoding specific formats such as DXTn. As of 2016, texture mapping hardware is ubiquitous as most SOCs

contain a suitable GPU.

Some hardware combines texture mapping with hidden-surface determination in tile based deferred rendering or scanline rendering; such systems only fetch the visible texels at the expense of using greater workspace for transformed vertices. Most systems have settled on the Z-buffering approach, which can still reduce the texture mapping workload with front-to-back sorting.

Among earlier graphics hardware, there were two competing paradigms of how to deliver a texture to the screen:

• Forward texture mapping iterates through each texel on the texture, and decides where to place it on the screen.
• Inverse texture mapping instead iterates through pixels on the screen, and decides what texel to use for each.

Inverse texture mapping is the method which has become standard in modern hardware.

Inverse texture mapping

With this method, a pixel on the screen is mapped to a point on the texture. Each vertex of a

rendering primitive is projected to a point on the screen, and each of these points is mapped to a u,v texel coordinate

on the texture. A rasterizer will interpolate between these points to fill in each pixel covered by the primitive.

The primary advantage is that each pixel covered by a primitive will be traversed exactly once. Once a primitive's vertices are transformed, the amount of remaining work scales directly with how many pixels it covers on the screen.

The main disadvantage versus forward texture mapping is that the

swizzled texture

memory arrangement.

The linear interpolation can be used directly for simple and efficient affine texture mapping, but can also be adapted for perspective correctness.

Forward texture mapping

Forward texture mapping maps each texel of the texture to a pixel on the screen. After transforming a rectangular primitive to a place on the screen, a forward texture mapping renderer iterates through each texel on the texture, splatting each one onto a pixel of the

frame buffer

.

This was used by some hardware, such as the 3DO, the Sega Saturn and the NV1.

The primary advantage is that the texture will be accessed in a simple linear order, allowing very efficient caching of the texture data. However, this benefit is also its disadvantage: as a primitive gets smaller on screen, it still has to iterate over every texel in the texture, causing many pixels to be overdrawn redundantly.

This method is also well suited for rendering quad primitives rather than reducing them to triangles, which provided an advantage when perspective correct texturing was not available in hardware. This is because the affine distortion of a quad looks less incorrect than the same quad split into two triangles (see affine texture mapping above). The NV1 hardware also allowed a quadratic interpolation mode to provide an even better approximation of perspective correctness.

The existing hardware implementations did not provide effective

UV coordinate mapping, which became an important technique for 3D modelling and assisted in clipping

the texture correctly when the primitive goes over the edge of the screen. These shortcomings could have been addressed with further development, but GPU design has since mostly moved toward inverse mapping.

Applications

Beyond 3D rendering, the availability of texture mapping hardware has inspired its use for accelerating other tasks:

Tomography

It is possible to use texture mapping hardware to accelerate both the reconstruction of voxel data sets from tomographic scans, and to visualize the results.^[19]

User interfaces

Many user interfaces use texture mapping to accelerate animated transitions of screen elements, e.g.

Mac OS X

.

See also

• 2.5D
• 3D computer graphics
• Mipmap
• Materials system
• Parametrization
• Texture synthesis
• Texture atlas
• Texture splatting – a technique for combining textures
• Shader (computer graphics)

References

1. ^ Wang, Huamin. "Texture Mapping" (PDF). department of Computer Science and Engineering. Ohio State University. Archived from the original (PDF) on 2016-03-04. Retrieved 2016-01-15.
2. ^ "Texture Mapping" (PDF). www.inf.pucrs.br. Retrieved September 15, 2019.
3. ^ "CS 405 Texture Mapping". www.cs.uregina.ca. Retrieved 22 March 2018.
4. ^ Catmull, E. (1974). A subdivision algorithm for computer display of curved surfaces (PDF) (PhD thesis). University of Utah.
5. ^ "Method and Apparatus for Texture Generation".
6. doi:10.1145/964967.801126
   .
7. ^ Fosner, Ron (January 1999). "DirectX 6.0 Goes Ballistic With Multiple New Features And Much Faster Code". Microsoft.com. Archived from the original on October 31, 2016. Retrieved September 15, 2019.
8. ^ Hvidsten, Mike (Spring 2004). "The OpenGL Texture Mapping Guide". homepages.gac.edu. Archived from the original on 23 May 2019. Retrieved 22 March 2018.
9. ^ Jon Radoff, Anatomy of an MMORPG, "Anatomy of an MMORPG". radoff.com. August 22, 2008. Archived from the original on 2009-12-13. Retrieved 2009-12-13.
10. ^ Roberts, Susan. "How to use textures". Archived from the original on 24 September 2021. Retrieved 20 March 2021.{{cite web}}: CS1 maint: unfit URL (link)
11. ^ Blythe, David. Advanced Graphics Programming Techniques Using OpenGL. Siggraph 1999. (PDF) (see: Multitexture)
12. ^ Real-Time Bump Map Synthesis, Jan Kautz¹, Wolfgang Heidrichy² and Hans-Peter Seidel¹, (¹Max-Planck-Institut für Informatik, ²University of British Columbia)
13. ^
    Imagine Media
    . March 1996. p. 38.
14. ^ Kalms, Mikael (1997). "Perspective Texturemapping". www.lysator.liu.se. Retrieved 2020-03-27.
15. ^ "Voxel terrain engine", introduction. In a coder's mind, 2005 (archived 2013).
16. ISBN 1-57610-174-6 (PDF Archived 2007-03-11 at the Wayback Machine
    ) (Chapter 70, pg. 1282)
17. ^ US 5739818, Spackman, John Neil, "Apparatus and method for performing perspectively correct interpolation in computer graphics", issued 1998-04-14 
18. doi:10.1109/MCG.1985.276213. {{cite journal}}: External link in |ref= (help
    )
19. ^ "texture mapping for tomography".

Software

• TexRecon Archived 2021-11-27 at the Wayback Machine — open-source software for texturing 3D models written in C++

External links

• Introduction into texture mapping using C and SDL (PDF)
• Programming a textured terrain using XNA/DirectX, from www.riemers.net
• Perspective correct texturing
• Time Texturing Texture mapping with bezier lines
• Polynomial Texture Mapping Archived 2019-03-07 at the Wayback Machine Interactive Relighting for Photos
• 3 Métodos de interpolación a partir de puntos (in spanish) Methods that can be used to interpolate a texture knowing the texture coords at the vertices of a polygon
• 3D Texturing Tools