Q1Digital Image Processing

(a) Down Sampling and Up Sampling

Down sampling, also called subsampling, is the process of reducing the spatial resolution of a digital image by retaining only every Nth pixel, row, or column and discarding the rest. If an image of size M x N is down sampled by a factor of 2, the resulting image has M/2 x N/2 pixels, and the data volume is reduced to one quarter of the original. Down sampling is widely used in building image pyramids (multi-resolution representations used in compression and computer vision), generating thumbnails for quick previews, and reducing computational load for algorithms that operate at coarse resolution first and refine later. A critical requirement before down sampling is anti-alias filtering: the image must first be passed through a low-pass filter to remove high-frequency detail that cannot be represented at the lower sampling rate. If this step is skipped, high-frequency content folds back into lower frequencies and appears as spurious patterns known as aliasing artifacts, such as jagged edges or moire patterns. The Nyquist criterion governs this: the sampling rate after down sampling must be at least twice the highest frequency present in the pre-filtered image.

Up sampling is the reverse process: increasing the spatial resolution of an image by inserting new pixels between existing ones. Since these new pixel positions did not exist in the original signal, their values must be estimated from neighboring known pixels using an interpolation scheme. Nearest-neighbor interpolation simply copies the value of the closest known pixel, which is fast but produces blocky results. Bilinear interpolation computes a weighted average of the four nearest known pixels based on distance, giving smoother transitions. Bicubic interpolation uses a weighted average of the sixteen nearest pixels with a cubic polynomial kernel, producing the smoothest and sharpest results among the three but at higher computational cost. It is essential to understand that up sampling can never recover information that was lost during an earlier down sampling step; it only produces a plausible estimate of what the missing pixels might have been. The output is always smoother or blurrier than a genuine full-resolution capture of the same scene.

Worked Example

Consider a simple 4x4 grayscale image with pixel intensities arranged in a grid. To down sample this image to 2x2 by a factor of 2, we keep only the pixels at even row and column indices (row 0, row 2 and column 0, column 2), discarding the intervening rows and columns entirely. The resulting 2x2 image retains only 4 of the original 16 pixel values, so 12 pixel values (75 percent of the information) are permanently discarded. If we now attempt to up sample this 2x2 image back to 4x4 using bilinear interpolation, the interpolation engine inserts three new rows and three new columns of pixels whose values are computed as weighted averages of the 4 surviving corner pixels. The reconstructed 4x4 image will visually resemble the original in gross structure (overall brightness pattern) but will appear blurred and will have lost all the fine detail that existed in the discarded rows and columns of the original image. This demonstrates that down sampling followed by up sampling is a lossy round trip: the up sampled result is only an estimate, never an exact reconstruction of the original 4x4 image.

(b) Image Acquisition Using CCD Sensors

A Charge-Coupled Device (CCD) sensor is one of the most common image acquisition devices used to convert an optical image into a digital image. A CCD sensor consists of a two-dimensional array of photosensitive elements called photosites, each of which behaves as a tiny capacitor. When light (photons) from the scene falls on a photosite during the exposure interval, the photoelectric effect causes the photosite to generate free electrons in proportion to the intensity and duration of the incident light. These electrons accumulate as an electric charge in the capacitor, so at the end of the exposure period each photosite holds a charge packet whose magnitude is proportional to the local brightness of the scene at that point.

Once exposure ends, the accumulated charge must be read out and converted into pixel values. This is accomplished through a charge-coupled shift register mechanism: the charge packets in each row are shifted downward, one row at a time, into a horizontal shift register at the bottom of the array (or the charge is shifted through vertical shift registers associated with each column). The horizontal shift register then shifts the charge packets of that row, one photosite at a time, to a single output node. At the output node, an output amplifier (often a charge-to-voltage converter followed by an analog amplifier) converts each charge packet into a proportional analog voltage. This analog voltage signal is then fed to an analog-to-digital converter (ADC), which quantizes the continuous voltage into a discrete numerical pixel value (for example, an 8-bit value between 0 and 255). This entire sequential process is repeated row by row until the charge from every photosite in the array has been read out, converted, and assembled into a complete two-dimensional digital image.

The key architectural components of a CCD sensor are therefore: (1) the photosite array, which performs light-to-charge conversion, (2) vertical shift registers, which move charge column-wise toward the horizontal register, (3) the horizontal shift register, which moves an entire row of charge packets serially toward the output, and (4) the output amplifier and ADC stage, which converts each charge packet into a digital pixel value. Because the readout is inherently sequential (all photosites share a common output path), CCDs have historically offered very high image quality and low noise, at the cost of slower frame rates compared to CMOS sensors, which read out pixels in parallel.

It is worth noting that the entire CCD readout process must complete before the next exposure begins (or a mechanical/electronic shutter is used to block light during readout), because the charge-coupled shift register mechanism physically moves charge packets through the same silicon substrate that is still being exposed to incoming light; if new photons continued generating charge in the photosites while old charge was being shifted out, the transferred charge packets would pick up spurious extra charge from the pixels they pass through, causing a vertical smear artifact in the final image, especially visible when imaging very bright light sources. This is one reason CCD sensors are typically paired with either a mechanical shutter or a specialized frame-transfer or interline-transfer architecture, where a light-shielded storage area adjacent to each photosite allows the accumulated charge to be quickly moved out of the light-sensitive region before the relatively slower serial readout begins, decoupling the exposure and readout timing to some extent.

In comparison to CMOS sensors, which read out each pixel's charge independently and in parallel using dedicated per-pixel or per-column amplifiers, the CCD's serial, shared-output-amplifier readout architecture generally produces very uniform, low-noise images because all pixels are converted to voltage by the same single amplifier (avoiding pixel-to-pixel gain mismatches), which is why CCDs have historically been preferred in applications demanding the highest image quality, such as scientific and astronomical imaging and high-end scanners, even though this comes at the cost of slower overall readout speed and typically higher power consumption compared to CMOS alternatives.