Spatiotemporal Interpolation


Matrix information signs in railway stations and trains often display scrolling text. The text usually appears slightly tilted or italicised, yet the matrix of light-emitting diodes (LEDs) is aligned vertically and horizontally. So why does the text appear tilted?
The answer lies partly in the electronics of the sign, and partly in the visual system. Each horizontal row of the matrix is updated sequentially by control circuitry, so the text display updates one row at a time. The demonstration above is a simulation of such a display. It consists of a 20 x 3 matrix of 'LEDs'. The 'letter' is a single column of lit LEDs which scrolls from side to side, but each horizontal row of LEDs is updated sequentially. Use the playback control to pause the movie, and use the keyboard's arrow keys to step through frames and inspect the animation sequence. When the animation is played, the scrolling column of lit LEDs appears to have a slight tilt, even though the matrix is aligned vertically and horizontally. Notice that the tilt is smaller than any available oblique alignment of LEDs in the matrix (any slight jerkiness in the motion is due to QuickTime's inability to display fast animations accurately).
The apparent tilt is created in the visual system. The visual response to each LED flash persists for a short time after the LED has flashed, so the sequence of flashes along each row appears as a smoothly moving light. The slight time delay in the updating of each row is translated into a slight spatial offset in the instantaneous perceived position of the light in each row, creating the apparent tilt. Another way to describe this effect is to say that the visual system interpolates between the discrete spatial and temporal samples provided by the LED matrix to create apparently smooth and continuously moving letters, hence 'spatiotemporal interpolation'. Inside the visual system, slight differences in timing are translated into slight differences in signalled spatial position.
Burr, D. C. (1979). Acuity for apparent vernier offset. Vision Research, 19(7), 835–837.
Fahle, M., & Poggio, T. (1981). Visual Hyperacuity: Spatiotemporal Interpolation in Human Vision. Proceedings of the Royal Society of London Series B, 213(1193), 451–477.
Morgan, M. J. (1980). Spatiotemporal filtering and the interpolation effect in apparent motion. Perception, 9(2), 161–74.
Nishida, S. Y. (2004). Motion-based analysis of spatial patterns by the human visual system. Current Biology, 14(10), 830-839.