Vision Science Calls for Perception-Friendly Displays

Abstract

Designing displays based on more than three light emitters requires neither new manufacturing techniques, nor hyperspectral cameras or signaling.

PEOPLE PERCEIVE SURFACE COLORS BASED ON THREE types of spectra: the spectral sensitivity of their own retinal cone cells; the spectral radiance of light arriving at their eyes from objects; and the average spectral radiance of light recently arriving at their eyes from the surrounding field of view. This is also true when viewed objects are images on a display, but in that case an added consideration is the spectral power distribution of its light emitters. This article discusses how using narrow-band, trichromatic displays can impair color reproduction accuracy, which is not a good way to increase color gamut. Fortunately, this problem can be greatly reduced by adding one or more emitters to the display design, without requiring a change to cameras or television signals.

Almost all current color displays are trichromatic (i.e., they use three emission light spectra having different peak wavelengths to form images that realistically reproduce the colors found in most natural scenes). Here, we focus on a key aspect of display emitters—their spectral power distribution, often portrayed in a graph of radiance versus wavelength.

Most display emitters have a peak wavelength (at which it is most intense) and an effective “bandwidth.” Previously (in IEC 61966-2-1/sRGB),¹ the bandwidths of the emitters have been approximately one-third of the visible spectrum, which in total spans from violet (∼400 nm) to deep red (∼680 nm).

Today, there is growing interest in expanding display gamut, in turn leading to the consideration of emitters that have much smaller bandwidths. This issue features examples of various spectral distributions. Unfortunately, they cause color distortion for many viewers.

So, why are manufacturers producing narrow-band, trichromatic displays? There are multiple reasons. For example, some projectors must use highly collimated lasers, which are naturally narrow band. Another reason is that narrow-band emitters can produce some colors that are more saturated than real-life objects. Sales representatives could demonstrate this extra vividness to customers, who might choose a display with that capability, if they were unaware of the potential problems that arise if the display uses only three narrowband emitters.

Also, because the problem described is not well known yet, it is understandable that not every researcher in the display industry would be aware of it. We hope this discussion will encourage manufacturers to learn more about these important issues.

Overall, we strongly believe that narrow-band, trichromatic displays are a poor approach for the display industry to embark upon. This does not mean that wide color gamut (WCG) displays are a poor idea. With proper ergonomic design, WCG displays can be an affordable, valuable addition to this industry. That requires more than three emitters, but importantly, no change would be required to the current standard HDTV signals, and these advanced displays could be made using existing, practical, display manufacturing methods.

Problems with Wide-Gamut, Trichromatic WCG Displays

A key point about human vision is that it is significantly affected by previous sights over various timeframes ranging from recent seconds to years.² Therefore, we should be concerned about color gamut—the size of the range of colors a display can produce. If synthetic content providers prioritize greater color gamut in displays, this could impair the user's appreciation of less-saturated real-life colors because of adaptation and metamerism in human vision. Furthermore, if manufacturers introduce trichromatic, narrow-band displays in an attempt to create more colorful images, many observers will experience poorer color fidelity.³

Understandably, manufacturers often compete with one another to achieve ever-increasing quality in their product specifications. Boosting color gamut is an example. Unfortunately, excessive exposure to WCG visual stimuli could cause adaptive responses in the human vision system that could change how people perceive the real world. Another side effect of narrow-band emitters is that different observers might see the same colors quite differently—an effect called metameric mismatch, which can be disturbing. Fortunately, these problems are avoidable.

Key Aspects of Color Perception

A common misunderstanding about human color perception is the notion that it mainly “happens” in the eye. That is inconsistent with a large, well-established body of scientific evidence. As the adage goes, the truth is that “we see with our brains, not with our eyes.” Optimizing color displays requires a deep understanding of that concept.

Notably, the information transmitted by the optic nerve to the brain bears little resemblance to the color sensations that people describe. Although the human color perception system is not fully understood, we have learned that the results it produces can be well-represented by numbers generated by formulas that were developed to fit experimental data by “color appearance modeling.”

Despite successes in modeling the human visual perception system, it is not yet fully understood exactly how it achieves these valuable results. An example could be a TV camera pointed at an object in a scene containing many other objects having a wide variety of spectral reflectance curves. Some of the light reflecting from each of those objects enters the camera lens, which sends a signal to a color display in another place with different surroundings. The TV industry has learned how to use the camera, intervening signal processing stages, and a display to do a wonderful thing. They yield an image of the object, whose perceived color appearance in the second location matches that of the object in the first location. This makes it possible for a camera to view an object in one lighting environment (for example, outdoors on a sunny day) and for the display to represent it in an “appearance-matching” image on a display in a very different visual environment (for example, indoors under much lower illuminance levels).

Unfortunately, most people have a limited understanding of human color perception and therefore do not realize that it produces valuable color information through highly sophisticated information processing. The primary net effect of the vision system, as a whole, is to estimate the spectral reflectance of objects in the visual field. This is remarkably challenging, but the vision system does it well. It suggests that there was an evolutionary advantage in having information about spectral reflectance, which drove the evolution of a color vision system that infers the spectral reflectance of surfaces from incoming spectral radiance information about objects and the overall scene.

Returning to the topic of displays, it is important to picture the entire system of an information display from end to end. Imagine a person viewing a colored surface (e.g., a paint sample) in a specified visual environment (e.g., a light box with a white interior). Vision psychologists have established that the brightness of that object depends not only on its radiance, but also on the average radiance of the surrounding scene. Put simply, the vision system does not “measure” radiance; instead, it infers reflectance by comparing an object's radiance to average radiance values, both in the present visual environment and the recent past.

Extending this to considering color, the human vision system infers the approximate spectral reflectance of an object averaged over three wavelength bands: short, medium, and long. (These are not directly equivalent to the cone fundamentals because of the processing of signals later in the vision system.⁴) It does so remarkably well by means of sophisticated processing of the information obtained from the eye, in part by constantly “looking around” the visual scene. The spectral reflectance in the short, medium, and long wavelength bands represents the “color” of an object. Generally speaking, this enables “color constancy”—people perceive a given object as having a single, stable color, even when viewed in a wide range of lighting conditions and environments. That is valuable, because lighting can change significantly in intensity and spectral power distribution, yet the appearance of a given object remains largely stable. This does not seem accidental and likely evolved because it aided survival.

Vision scientists have modeled this color constancy phenomenon in a framework that essentially relies on two measurements and a simple calculation. Roughly speaking, here is how the perceived color of an object can be approximately predicted in a fairly uniform lighting environment:

1. Aim a spectral radiance meter at the object and record the spectral data, S_R(λ).
2. Point a spectral irradiance meter at the object to determine the scene's average spectral data, S_I(λ).
3. Calculate the ratio R_E(λ) = S_R(λ)/S_I(λ) as an effective spectral reflectance assessment.
4. Summarize R_E(λ) by averaging it over three wavelength bands.

The correspondence between the perceived color and the three (long, medium, and short wavelengths) averages of R_E(λ) is remarkably consistent for people with normal color vision, which suggests that this was evolutionarily beneficial. Perhaps people used reflectance information to identify plants, assess water purity, or determine other people's health.

But how does the vision system achieve this useful result, which is often called color constancy? Conversely, can it be impaired? Unfortunately, that is the problem with displays that use three narrow-band emitters.

To summarize, shades of gray and color are determined almost entirely by the ratios of the intensity of light arriving at the eye after reflecting off a specific object, and by the average of the intensities arriving at the eye coming from the visual scene as a whole. This works really well over the 1,000-fold illuminance range spanning from dim indoor lighting to outdoor daylight. Some sketches can help clarify this. Fig. 1 illustrates a case without wavelength variation, so there are only shades of gray.

Fig 1

Three objects with differing reflectance values. For the left object, the “<” symbol means the object's radiance is much less than the scene average (about 1/10). The “=” symbol means they are about equal, and the “>” symbol means the object's radiance is much greater than average (about 10×). Most people say they appear as dark gray, gray, and light gray. These appearances are stable over roughly a 1,000-fold range of ambient illumination.

People mainly perceive intensity ratios, not intensity values. (This “relativistic” characteristic is true for many aspects of perception—not just vision—a phenomenon sometimes called Weber's Law.) Notably, there is a good reason for this; these ratios provide useful information about the object we are looking at. For example, is the surface coal, cement, or ice cream? This ratio-related information is largely independent of the overall scene illuminance, and therefore it provides people with useful, actionable information about the surfaces themselves.

Consider reflectance varying with wavelength. Most object colors can be created by varying three reflectance bands. Typically, these bands are ∼80-nm wide, centered (roughly) on 440 nm (for the “blue” band), 520 nm (the “green” band), and 600 nm (the “red” band). Interestingly, human cone cells do not reflect that picture; the sensitivity functions of the L and M cones overlap a lot. Because we see with our brains, where signal processing yields a simple, useful result, if two objects do not match in reflectance in each of those three bands, their colors will not match well. But if they do, they will. To summarize, the human color vision system is not a camera. It is an imaging three-band spectral reflectance evaluator.

Consider the representative array of three-band spectral reflectance combinations depicted in Fig. 2.

Fig 2

Surface appearances are largely because of three average reflectance values in the long, medium, and short visible wavelength bands. For each, the selected objects have 1/10, equal, or 10× the radiance of the background average, denoted as <, =, >. Here, there are 27 permutations, shown in seven subgroups with overall reflectance increasing from left to right.

The surface located at the right edge of the middle subgroup appears “orange” because it has a high reflectance in the long wavelength “red” band, medium in green, and low in blue. It is all about the ratios.

Counterintuitively, the perceived color of an object is every bit as dependent on the light coming from the overall visual scene as it is on the light coming from the object itself. Think of a piece of paper that looks white outdoors (under blue-white, 10,000 lux, daylight) and also indoors (under orange-white, 100 lux, evening illumination). The vision system continually and gradually recalibrates the multiple stages within the vision system, which ensures that most people see colors quite similarly, and that these perceptions remain stable over a person's life, even as their eyes slowly change.⁵

However, the valuable capability described has a critical limitation. The color accuracy depends on viewing spectral power distributions that vary smoothly with wavelength. If the light spectra are not reasonably smooth, significant color distortions can arise. As an example, color accuracy is greatly diminished when the illumination spectral power distribution contains a small number of narrow-band spectral peaks—a problem often called metameric mismatch.⁶

Implications of Vision Characteristics for Display Design

We already mentioned that the “relativistic” nature of color vision has big implications for color displays. With broad “RGB” bands, as is common with LCDs with broadband backlights, all the ideas presented previously work well, and color vision behaves essentially as described.

However, such displays cannot achieve all possible colors. To help understand this, we recommend reading another article in this issue.⁷ Here, we mainly explain why using three narrow-band display emitters is a poor idea for accurate color perception.

Additionally, a different kind of concern arises from the highly adaptive nature of the color vision perceptions system. Apparently, the system continuously monitors both the stimuli it receives and visual perceptions it yields. Based on that information, it gradually adjusts its operating parameters to optimize the visual system's performance. As a result, if a person observes a lot of high-gamut color information, the system's color contrast sensitivity diminishes, making the real world appear less colorful.

Experimental Tests: Concerns about Narrow-Band Trichromatic Displays

Recent studies showed that when two different displays having three narrow-band emitters are shown side by side, although the CIE-calculated chromaticity values of display whites are the same, they can look quite different to people even with normal color vision.⁸ (Although in the past, having two displays in the same room usually was restricted to TV showrooms, today it is commonplace for several people to be viewing their phones, laptops, and a TV simultaneously.)

Fig. 3 shows the experimental setting for an example display color-matching experiment. Two lighting fields were generated with different displays and shown to an observer in a dark room. Then the observer was asked to match the color appearance of two fields by adjusting the RGB values of the test display. After the adjustment, spectra of two fields, S_ref(λ) and S_test(λ), were measured.

Fig 3

Display color-matching experimental setup.

If the CIE standard observer properly represents the average observer, the CIE XYZ values calculated using S_ref(λ) and S_test(λ) should result in the same values. However, when the calculated CIE colorimetric values of the reference and the matched tristimulus values were compared, significant differences were observed.

Fig. 4 shows an analysis in which the XYZ matching data are converted to the CIE u’10 v’10 chromaticity diagram. “Reference” (blue circle) depicts the reference display chromaticity, and the red star represents the chromaticity of the color-matched test display based on the CIE standard observer. The red and blue arrows connect to the test display chromaticity selected to match the reference by each observer. Thus, the length of each red arrow depicts the difference between the observer-selected chromaticity values and that for the CIE standard observer.

Fig 4

Chromaticity values obtained from the experiment shown in Fig. 3.

Most of the display RGB matching experiments showed that the individual variation is largest for neutral colors and becomes smaller for chromatic colors. Also, the amount of individual variation is highly dependent on the display spectral characteristics. Narrow-band WCG RGB displays yield more variation.

It should be noted that the CIE 10-degree standard observer is based on experimental data from only 49 subjects, as published in CIE 170-1.⁹ It is also well-known that human cone fundamentals are affected by the size of field and age, and modest variations in the cone peak wavelengths. Thus, some degree of mismatch is expected under any circumstance. However, both theory and observations clearly show that mismatches of this type are significant in trichromatic, narrow-band emitter displays. In the future, it is likely that improvements in determining cone fundamentals will help guide overall progress in displays, considering the natural variation in cone fundamentals.

Design Implications for the Future

We have no fundamental objection, in principle, to the introduction of WCG displays, although there probably is little need for them. We recognize that in side-by-side comparison, viewers often may choose a WCG display if they do not understand the associated problems. For this reason, we strongly recommend that displays should be required to emit light that is less able to cause adaptive changes in color vision, at least some of which can be relatively long-lasting.^{10, 11}

A related consideration is that the light from displays, including smartphones, enters the surrounding personal environment to an ever-increasing extent, so “screen light” could distort the color appearance of nearby objects in the room or elsewhere. It seems reasonable to consider that the light output of screens should be taken into account when lighting designers ensure that the total light in a given location meets the color-rendering requirements for illumination. For example, the Illuminating Engineering Society recommends using the IESTM-30 Color Fidelity Index R_f (the improved version of the original CIE Ra), for assessing indoor illumination, and today it often is considered appropriate for R_f and R_a to exceed 90, on a scale where daylight and incandescent lights score 100.¹² In contrast, a trichromatic narrow-band display ranks far lower and would never be considered (on its own) to be a suitable source for illumination. Yet, increasingly, light from displays illuminates nearby objects. In short, using three narrow-band emitters in a display is a poor idea from multiple perspectives.

It should be relatively simple to develop a technical specification that ensures that a display does not distort the vision of normal observers, and that it cannot expose viewers to too much gamut for too long. This could apply to all displays and would be especially important for WCG displays. Today, it often is considered ideal for luminaires to have R_f values above 90, whereas daylight and incandescent lights have an R_f value of 100.¹² This suggests that a careful evaluation process should be carried out to establish an appropriate R_f target for the light emitted from displays. This seems important because, increasingly, display emissions illuminate nearby objects. Additionally, there could be a requirement that displays automatically down-regulate chroma as needed to avoid exceeding the gamut levels that are typically found in ordinary real-world environments, more than a specified amount per hour, which probably could be achieved with minimal impact on normal TV content. In short, using only three narrow-band emitters in a display is a poor idea from multiple perspectives.

More importantly, we recommend research on ways to minimize the overarching problem that narrow-band trichromatic displays cause significant color differences between diverse, but color-normal, individuals. Effectively, these displays cause a significant portion of the color-normal population to be made, effectively, slightly color blind when viewing the display. This can be avoided by not using trichromatic displays or by deliberately broadening emission spectra.

Instead of having only three emitters, it has been shown that by adding even one more emitter to a display can greatly reduce this problem, and using five or six narrow-band emitters essentially can eliminate it.³ Importantly, people sometimes have assumed that adding more emitters to a display would require more primaries in the color TV signal, but this is not the case. Standard HDTV signals encoded in (for example, ITU-R BT.2020) all the information needed to drive displays with many emitter wavelengths will require the development of control algorithms that ensure accurate color while minimizing metameric mismatch. This will enable them to produce exceedingly accurate color experiences for all color-normal observers, and such diversity tolerance should become a required standard for healthful information displays.

We also note that current displays lack the far-red and near-infrared wavelengths that are present in natural daylight, and recent research¹³ suggests that the absence of this portion of the spectrum may be an unhealthful characteristic. Currently, there is no standard way to summarize far-red or near-infrared radiation content. We recommend serious consideration of the possibility that one of the emitters of a multi-chromatic display could be a far-red and/or near-infrared source to help address this concern.

Conclusion

The increased display gamut made possible by using three narrow-band emitters is appealing, but it is a bad idea for three reasons. First, most real-world scenes do not need to be faithfully displayed. Second, trichromatic narrow-band displays can never look correct for all observers, so they needlessly discriminate against some normal viewers. Third, exposing people, especially children, to large amounts of saturated colors in synthetic or chroma-boosted content could cause incorrect, long-term adaptations within the human vision system that could diminish perception of subtle colors in real environments. Fortunately, a good alternative exists that addresses these concerns: designing displays based on more than three light emitters. Importantly, this does not require new manufacturing techniques, nor hyperspectral cameras or signaling. Furthermore, because high-chroma content might be harmful if watched excessively (especially for children), it could be beneficial to monitor incoming signals and subtly reduce gamut as required to prevent excessive chroma exposure using gamut-mapping algorithms in the display. With these design improvements, all viewers can have superb, vision-friendly color experiences, and color mismatches finally will be a thing of the past.

Acknowledgments

We thank Michael Webster of the University of Nevada, Reno, for his significant advice on the multitude of adaptive processes that exist, on different time scales, within the human color vision system.

Biographies

• Youngshin Kwak is a professor at the Ulsan National Institute of Science and Technology. She served as the CIE Division1 Vision and Color Director from 2016 to 2023. Kwak earned a PhD in color science from the University of Derby. She may be reached at [email protected].

  

• Lorne Whitehead is a professor at the University of British Columbia, Vancouver. He currently chairs the CIE Technical Committee 1-98, “A roadmap toward basing CIE colorimetry on cone fundamentals.” Whitehead received a PhD in physics from the University of British Columbia. He may be reached at [email protected].