Lesson 11: Smoothing and Gaussian Pyramids¶
Lesson 10 introduced Gaussian blur as one convolution kernel among several. Here we look at why blurring matters beyond just "softening" an image: it's the key ingredient that makes downsampling safe. That leads directly to the Gaussian pyramid — a stack of progressively smaller, blurrier versions of an image, used throughout computer vision for multi-scale analysis.
import numpy as np
import cv2
import matplotlib.pyplot as plt
Why downsampling needs blurring first¶
Naively shrinking an image by keeping every $k$-th pixel ("nearest-neighbor downsampling") can produce aliasing: fine periodic detail that oscillates faster than the new pixel spacing can represent folds into a completely different, fake low-frequency pattern.
We demonstrate with a photo of a brick wall — the repeating rows of bricks and mortar lines are exactly the kind of fine, regular detail that aliases badly.
brick_wall = cv2.imread('../img/brick_wall.jpg', cv2.IMREAD_GRAYSCALE)
factor = 4
naive_downsample = brick_wall[::factor, ::factor] # subsample directly
safe_downsample = cv2.GaussianBlur(brick_wall, (0, 0), sigmaX=factor / 2)[::factor, ::factor] # blur first
fig, axes = plt.subplots(1, 3, figsize=(10, 3.5))
axes[0].imshow(brick_wall, cmap='gray')
axes[0].set_title('Original')
axes[1].imshow(naive_downsample, cmap='gray')
axes[1].set_title('Naive: subsample only\n(aliased, noisy-looking)')
axes[2].imshow(safe_downsample, cmap='gray')
axes[2].set_title('Blur, then subsample\n(correctly soft)')
for ax in axes:
ax.axis('off')
plt.tight_layout()
plt.show()
Image source: publicdomainpictures.net
The naive version turns the regular brick pattern into fine, noise-like speckle — a classic aliasing artifact (the same effect that makes a car's wheels look like they're spinning backwards on camera). The blurred version stays recognizably a brick wall, because the mortar lines were smoothed away before subsampling, rather than being randomly kept or dropped pixel by pixel.
Choosing sigma and kernel size¶
Blurring correctly before downsampling needs two things to actually work: the right sigma, and a kernel large enough to represent that sigma. cv2.GaussianBlur's sigma sets how far the bell-curve weighting spreads: larger sigma averages over a wider neighborhood, removing finer detail. Concretely, blurring a single pixel with a 3x3 Gaussian kernel is nothing more than a weighted average of its 3x3 neighborhood: multiply each of the 9 neighboring pixel values by the matching kernel weight and sum. cv2.getGaussianKernel(3, sigma) gives the 1D weights; their outer product gives the 2D kernel.
rng = np.random.default_rng(0)
noise_img = rng.integers(0, 255, (20, 20)).astype(np.uint8) # single-channel, for a simple example
sigma = 1.0
k1d = cv2.getGaussianKernel(3, sigma)
kernel2d = k1d @ k1d.T
print('3x3 Gaussian kernel (sigma=1.0):\n', np.round(kernel2d, 3))
print('weights sum to:', kernel2d.sum())
y, x = 10, 10
neighborhood = noise_img[y - 1:y + 2, x - 1:x + 2].astype(np.float64)
manual = (kernel2d * neighborhood).sum()
blurred_full = cv2.GaussianBlur(noise_img, (3, 3), sigmaX=sigma)
print(f'\n3x3 neighborhood around pixel ({x}, {y}):\n{neighborhood}')
print(f'\nweighted average (manual): {manual:.2f}')
print(f'cv2.GaussianBlur pixel: {blurred_full[y, x]}')
The manual weighted average matches cv2.GaussianBlur's output for that pixel (up to integer rounding) — every blurred pixel really is just this 9-number dot product, repeated at every location in the image. Increasing sigma while keeping the kernel at 3x3 barely changes anything, though, because a 3x3 window only has room for 3 taps — to actually see a wider sigma's effect, the kernel has to grow too.
img = np.zeros((150, 150, 3), dtype=np.uint8)
cv2.circle(img, (75, 75), 55, (255, 120, 30), -1)
cv2.rectangle(img, (20, 20), (60, 60), (30, 200, 255), -1)
sigmas = [0, 1, 3, 8]
fig, axes = plt.subplots(1, len(sigmas), figsize=(12, 3.5))
for ax, s in zip(axes, sigmas):
# ksize=(0, 0) tells OpenCV to pick a kernel size automatically, big enough for this sigma
blurred = img if s == 0 else cv2.GaussianBlur(img, (0, 0), sigmaX=s)
ax.imshow(blurred)
ax.set_title(f'sigma = {s}')
ax.axis('off')
plt.tight_layout()
plt.show()
Kernel size has to keep up with sigma¶
ksize=(0, 0) above told OpenCV to auto-size the kernel for each sigma. What happens if the kernel is fixed too small instead? cv2.getGaussianKernel always renormalizes its weights to sum to 1, no matter how few taps it's given — so a too-small kernel doesn't just clip the Gaussian's tails, it silently reshapes the whole kernel into something close to a plain box average, throwing away the bell-curve shape entirely.
sigma = 5.0
k3 = cv2.getGaussianKernel(3, sigma).ravel()
print(f'a 3-tap kernel for sigma={sigma}, renormalized to sum to 1:', np.round(k3, 3))
print('(nearly identical to a plain 3-tap box average [0.33, 0.33, 0.33] -- the wide bell curve got squashed away)')
too_small = cv2.GaussianBlur(img, (3, 3), sigmaX=sigma) # kernel far too small for this sigma
auto = cv2.GaussianBlur(img, (0, 0), sigmaX=sigma) # OpenCV auto-sizes the kernel to fit sigma
fig, axes = plt.subplots(1, 3, figsize=(9, 3.5))
for ax, im, title in zip(axes, [img, too_small, auto],
['Original', 'ksize=3\n(too small for sigma=5)', 'ksize=0\n(auto-sized for sigma=5)']):
ax.imshow(im)
ax.set_title(title, fontsize=9)
ax.axis('off')
plt.tight_layout()
plt.show()
print(f'mean abs difference, too-small vs. correctly-sized kernel: {np.abs(too_small.astype(int) - auto.astype(int)).mean():.2f}')
The two results are visibly different — ksize=3 barely blurs the image at all, despite asking for sigma=5, since a 3-tap window simply can't represent a bell curve that wide. This is exactly why the pyramid code below (and the aliasing fix above) passes ksize=(0, 0): letting OpenCV choose a kernel wide enough for the requested sigma, rather than risking a silently-too-small one.
Building a Gaussian pyramid¶
A Gaussian pyramid repeats "blur, then downsample by 2" over and over, producing a stack of images each half the width and height of the previous one. Each level is a properly anti-aliased, coarser view of the same scene — not just a smaller crop.
def gaussian_pyramid(image, num_levels, sigma=1.0):
pyramid = [image]
current = image
for _ in range(num_levels - 1):
blurred = cv2.GaussianBlur(current, (0, 0), sigmaX=sigma)
current = blurred[::2, ::2]
pyramid.append(current)
return pyramid
photo = np.zeros((256, 256, 3), dtype=np.uint8)
photo[:] = (40, 40, 40)
cv2.circle(photo, (128, 128), 90, (255, 120, 30), -1)
cv2.rectangle(photo, (30, 30), (110, 110), (30, 200, 255), -1)
pyramid = gaussian_pyramid(photo, num_levels=5)
fig, axes = plt.subplots(1, len(pyramid), figsize=(13, 3))
for ax, level in zip(axes, pyramid):
ax.imshow(level)
ax.set_title(f'{level.shape[1]}x{level.shape[0]}', fontsize=9)
ax.axis('off')
plt.tight_layout()
plt.show()
Comparison to OpenCV's built-in cv2.pyrDown¶
OpenCV provides cv2.pyrDown, which does the same blur-then-downsample idea using a fixed, carefully designed 5-tap binomial kernel (approximating a Gaussian) instead of an arbitrary sigma. The output sizes match ours exactly; pixel values are close but not identical, since the kernels differ slightly.
cv_pyramid = [photo]
current = photo
for _ in range(4):
current = cv2.pyrDown(current)
cv_pyramid.append(current)
for ours, cvs in zip(pyramid, cv_pyramid):
assert ours.shape == cvs.shape
diff = np.abs(ours.astype(int) - cvs.astype(int))
print(f'{ours.shape[1]:>4}x{ours.shape[0]:<4} mean abs diff = {diff.mean():.2f}')
Pyramids lose information¶
Downsampling is not reversible: upsampling a lower pyramid level back to the original size (cv2.pyrUp) cannot recover detail that blurring/subsampling discarded. This gap between an upsampled coarse level and the original is exactly what a Laplacian pyramid captures at each level — a topic for a future lesson — but we can already see the information loss directly.
level1 = pyramid[1] # 128x128, one pyrDown from the original
reconstructed = cv2.pyrUp(level1) # back up to 256x256
diff = cv2.absdiff(photo, reconstructed)
fig, axes = plt.subplots(1, 3, figsize=(9, 3.5))
for ax, im, title in zip(axes, [photo, reconstructed, diff],
['Original', 'pyrDown then pyrUp', 'Difference (lost detail)']):
ax.imshow(im)
ax.set_title(title, fontsize=9)
ax.axis('off')
plt.tight_layout()
plt.show()
print(f'mean absolute reconstruction error = {diff.astype(np.float64).mean():.2f} gray levels')
The sharp edges of the circle and square come back soft and shifted-looking — the fine detail was permanently discarded when the image was blurred and subsampled, and pyrUp (interpolation) can only guess, not restore it.
Exercise¶
- Repeat the brick-wall experiment with
factor = 2instead of4. Does naive subsampling still show visible aliasing? Why might a smaller downsampling factor alias less? - Gaussian pyramids are used for coarse-to-fine search (e.g. quickly finding an approximate object location at a small pyramid level, then refining at larger levels). Why would skipping the blur step and just subsampling directly break this strategy?
- Build a 6-level pyramid of a real photo-like image and measure how many levels it takes before the image becomes too small to recognize any shape at all. What real-world resolution would that correspond to for, say, a 1920x1080 photo?