A recent thread about whether Blackmagic camera will have an antialiasing filter or not has gotten me thinking about the process of debayering/demosaicing in general and it's applications for different purposes. Currently topics discussed in that tread are related to whether or not BMCC will have an AA filter, what would be the consequences of having it or not and what is the purpose of such filter in general. I suggest reading that thread, lots of interesting stuff being written!
So, what is debayering / demosaicing?
Almost all still cameras and most video cameras these days have the so-called CMOS sensor. It means that light is captured on single sensor (as opposed to CCD systems which have separate sensor for each color) and to get color information out of it, a special filter, called Color Filter Array (CFA) is placed on the sensor. This filter has alternating pattern of red, green and blue sites in checkerboard (mostly) or some other pattern (hexagonal patterns are sometimes used). Because human eye is most sensitive to green light and thus green gives us lightness information, CFA has as much green sites as red and blue combined. In 2x2 checker, two diagonal patches would be green and other ones red and blue. Bayer pattern is essentially a hard-wired 4:2:2 subsampling schema that operates on RGB channels (in GRB order in this case).
CMOS sensor itself is basically black and white, it does not "see" color but only the intensity of light. So through filter we get intensities of green, red and blue light in a certain pattern. To get full RGB raster from this intensity map, debayering operation is performed. In it's simplest form, average values of adjascent sites can be calculated to make full image for each color. Simple averaging is easy but tends to give a very low quality image. For better results, "smarter" algorithms can be used that take into account value gradients, similarities and what not. A lot of these algorithms are patented and secret because among digital image makers, better image is worth a lot of money. Some information about different algorithms can be found on Wikipedia
What about BMCC
One topic that got me thinking about the difference between combating moire and aliasing on DSLRs and digital film cameras was the AA filter from company called Mosaic Engineering. They offer a filter that you can attach in front of Canon 7D or 5D sensor and get rid of ugly color moire issues. In the thread mentioned before, some people argued that not having such a filter in front of the BMCC sensor would turn up similar ugly patterns as on most DSLRs. I can't say that I totally get the logic behind this. DSLRs generally skip lines to pull HD image from 5-6K sensor without downscaling that is some expencive processing. Skipping lines makes images vulnerable to moire because when pattern frequency and line skipping frequency clash, moire jumps up.
In the Pool Shark clip from John Brawley, chair with double fibre mesh backing made people jump because "BMCC has moire!" although this pattern is visible for anyone who sees such chair in person. Fibre mesh on the back of chair is somewhat similar to sensor that skips elements. Some spots are "blind", some "see". When this blind-see pattern sees another high-contrast pattern, these blind-see spots can either line up or not. When they don't some spots see another pattern, some dont and new pattern is formed that has maximums at lined up parts and minimums where patterns "cancel" each other out. Like two sine waves summed. This happens with sensors that have gaps between image elements, whatever the reason for this is (skipping, gaps due to sensor construction etc).
BMCC sensor does not (?) have gaps between sensor elements, at least I hope so. Gapless sensor catches all light that falls on it and thus should effectively prevent most moire patterns. It is natural antialiasing where for example a very thin diagonal line leaves mark on all pixels it crosses, although it doesn't cover any of them entirely. The intensity of pixel gives us impression that line crosses it but only barely and thus resulting image does not show aliasing. Aliased image would be produced if we tested for stick only in the dead centers of elements. Most pixels would show nothing and some would show everythin, thus a jagged aliased image.
What happens with this naturally antialiased bayer image during demosaicing is totally another story and this is where most of the ugly colored 1-2 pixel wide edges appear. They are the result of incorrect calculations in predicting what RGB values in this point should be. From totally black and white source image projected on bayer sensor, demosaicing can produce a rather rainbow like result where edges shimmer with purple, yellow, blue and every other color imaginable.
New understanding!
During writing the above text, it clicked that even with gapless sensor, patterns can introduce moire because for example on one element, 100% value is captured but on other two only 50% each because light patch falls on both of them. This leads to bright values on every third element and half-bright on every other two and so a new pattern that didn't exist in projected image appears as moire
This is where AA filter comes to play and cuts off higher frequencies that produce most of such moire.
No comments:
Post a Comment