Figure 2: Bayer CFA layout.
Each pixel repr e sents either
the red, blue, or green value
of the light incident at the
sensor, depending on arrange-
ment in the array. To get
all three elements at every lo-
cation, demosaicing must be
applied.
Color Channel Scaling
An unfortunate reali ty of color imaging is that there is no truth
in color. Generally, one cannot look at a color image and
know that it faithfully re pr es ents the color of the subject in ques-
tion at the time the image was taken. As a severe simplifica-
tion, one can think of an illuminating light as having an intrin-
sic color and the object it falls upon also having its own color.
These interact, and the light which the camera or eye receives, the
light reflected from the object, is an i ne xt r i cab l e combination of
the two.
What this leads to is that any object can look like any color, depending on
the light illuminating it. What we need is a reference point, something we
know should be a certain color (or more accurately, a certain chromatic-
ity), so that we can adjust the R, G, B values of th e pixel until i t is that
color. This compensates for th e color of the illuminating light and reveals
the “true” color of the object. Under the assumption that the same illu-
minant is lighting the entire scene, we can do the same balancing to every
pixel in the image. This is the process of white balancing. Essentially, we find a pixel we know should be
white (or gray), which we know should have RGB values all equal, and find the scaling factors necessary to
make them all equal.
Thus the problem reduces to simply finding two scalars which represent the relative scaling of two of the
color channels to the third. It is typical that the green channel is assumed to be the channel to which the
others are compared. These scalars also take into account the relative sensitivities of the different color
channels, e.g., the fact that the green filter of the CFA is much more transparent than the red or blue filters.
Thus, the green channel scalar is usually 1, while the othe rs are often > 1.
Camera Color Spaces
The number of dimens ion s of perceivable colors is three. This may seem obvious to those who are familiar
with this result (such as anyone who has ever heard that “any color can be made from a combination of
red, green, and blue light”), but the reduction from an infinite-dimensional sp ace to a three-dimensional one
is far from trivial. An almost-vector space may be construct e d for perceivable colors, leading to the use of
many familiar linear algebra operations
1
on what we refer to as “the color space”. The ‘almost-’ qualifier is
necessary b e cau se one cannot subtract color since there is no such thing as negative light (yet). Fortunately,
this st i ll leaves us with a convex cone of color which is quite amenable to linear algebra.
In t h e language of linear algeb ra, we repr es ent the color of a pixel with three coordin ate s (which we recognize
as the R, G, and B values associated with the pixel). Importantly, these are the p i x el ’ s color coordinates with
res pect to a particular basis. What is often not evident is that the basis generated by the physical sensors
of the digital camera is not the same as those of most displays. A full discuss ion of this topic is beyond the
scope of this document, but we will state simply that we presume the displayable output space to be that
defined as a common standard, sRGB[
10].
Thus, though we obtain a familiar RGB image after white balancing and demosaicing a CFA, its colors are
not those which t he com pu t er mon i t or ex pects. To correct for this, we must apply a linear transformation
(i.e., an app rop r iat e change of basis matrix) to the RGB-vector of each pixel in the image. It is descri bed
later how this matrix can b e found.
1
In fact, the man who essentially invented linear algebra, Hermann Grassman, was also noted for exploring the laws p er tain ing
to the perceptual equivalence of the color of various light sources. Treating colors as vectors is historically tied into the origins
of vector spaces themselves!
3