Project 2 - Fun with Filters and Frequencies

General Overview

The goal of this project was to implement techniques for image filters and fiddling around with frequencies for creating hybrid images or blending images. Using gaussian filters, I could filter out the higher frequencies of an image, which helped me sharpen them. I also implemented gaussian and laplacian stacks to help blend images together, using a mask.

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

Part 1.1 Filtered Image

Above are the finite difference filters in the x and y directions. Let the image be M. Then, the partial derivative in the x direction is G_x = M_x * D_x. The partial derivative in the y direction is G_y = M_y * D_y. To get the gradient magnitude of the image, we simply do G_m = sqrt(G_x^2 + G_y^2). So, the gradient magnitude is just the Euclidean distance of the two partial derivatives. Finally, to turn this into an edge image, we binarize G_m by thresholding it, where we only accept values above a threshold. The results are displayed below:

original image
Image 1
no modifications to image
G_x
Image 2
partial derivative in x of image
G_y
Image 3
partial derivative in y of image
G_m
Image 4
gradient magnitude of image
G_m, binarized
Image 5
threshold = 0.25

Part 1.2: Derivative of Gaussian (DoG) Filter

First, we try the blur then redo the previous methods. For this, we first apply the gaussian filter to the original image. Afterwards, we apply the same methods as before. Compared to the results from part 1.1, these images are significantly smoother. This can be especially seen in the partial derivative images, where they feel less sharp. Also, the gradient magnitude is less noisy, and the binarized image is much more smooth. Below are the results:

blurred original image
Image 1
gaussian filter blurred image
G_x
Image 2
partial derivative in x of blurred image
G_y
Image 3
partial derivative in y of blurred image
G_m
Image 4
gradient magnitude of blurred image
G_m, binarized
Image 5
threshold = 0.09

Now, we try the second method by creating the DoG filters. As seen below, the results are the same as the previous method. Both methods in part 1.2 produce the same results at the end, because convolution is communitive. So, whether we are applying the gaussian filter to smooth the original image or to the finite difference operators, we will eventually get the same result. Below are the results:

original image
Image 1
no effects applied
D_x convolved
Image 2
result after D_x convolved with Gaussian kernel
D_y convolved
Image 3
result after D_y convolved with Gaussian kernel
G_x
Image 4
partial derivative in x using DoG
G_y
Image 5
partial derivative in y using DoG
G_m
Image 6
gradient magnitude using DoG
G_m, binarized
Image 7
threshold = 0.09/div>

Part 2: Fun with Frequencies

Part 2.1: Image Sharpening

In total, I have sharpened three images. Each row reflects the progression of each image as it is sharpened. Below are the results:

taj.jpg
Image 1
alpha = 2
Image 2
alpha = 4
Image 3
alpha = 6
Image 3
light.jpg
Image 1
alpha = 2
Image 2
alpha = 4
Image 3
alpha = 6
Image 3
dog.png
Image 1
alpha = 2
Image 2
alpha = 4
Image 3
alpha = 6
Image 3

As seen above, as alpha increased, each image got gradually sharper. I wanted to experiment with the light and dog images, because when I took the light image in Japan, I thought it was too blury. For the dog images, it had a lot of fur, so sharpening the image will provide more contrast to these details. Interestingly, for the light image, because overall my photo quality was too blurry, it mainly captured the edge features in the middle. This location in Tokyo is where the light through the clouds was shining on. So, as alpha increased, this region got highlighted more and more. The dog's fur showed created contrast and sharpness, as expected. My favorite image was the dog's progression, and as I thought its fur was already pretty clear, I chose this image for the evaluation. Below are the results from blurring the dog image, then re-sharpening it:

dog.png
Image 1
original image before effects
blurred dog
Image 2
low filter of original image
sharpened image
Image 3
added high filters repeatedly to sharpen

Part 2.2: Hybrid Images

This section of the project was to replicate the SIGGRAPH paper on hybrid images. The idea is when viewed from close and far distances, the image would look drastically different. This illusion is created by combining the low frequencies of one image with the high frequencies of another. Below are the results:

Derek's image
Image 1
Nutmeg's image
Image 2
Derek + Nutmeg
Image 3
Babak's image
Image 1
Bamman's image
Image 2
Failed hybrid
Image 3
dog facing left
Image 1
dog facing right
Image 2
left + right
Image 3

I could not find a way to make Babak and Bamman's image work, mainly because of Babak's facial features. While tuning the code, I could see Bamman's head structure when viewing the image from far away. However, Baback's eyebrows would always appear instead of Bamman's glasses. Also because Baback's image has darker colors, they tended to dominate the hybrid image. My favorite result was of the puppy looking either right or left. Below is the fourier frequency analysis of the images:

left.jpg
Image 1
frequency of image facing left
right.jpg
Image 2
frequency of image facing right
flip.png
Image 3
frequency of hybrid image
left.jpg
Image 1
frequency of left image after low-pass-filtered
right.jpg
Image 2
frequency of right image after high-pass-filtered

Bells and Whistles: I used color to enhance the effects of the hybrid images. It seems to be working much better for the low-frequency component. This is because the high-frequency components captures details such as edges, meaning that most pixels aren't captured. So, since the low pass filter lets much more pixels through, the majority of the color would be then from the low-frequency image.

Part 2.3: Gaussian and Laplacian Stacks

This section of the project is setting up the stacks in order to replicate the oraple result. Below are the recreated results for the laplacian stack levels, which are very similar to the original paper's:

Part 1.1 Filtered Image

Part 2.4: Multiresolution Blending

In this section, I added up the combined laplacian stack levels to create a blended image. Below are the original two images, the mask applied, and the resulting blend:

apple.jpeg
Image 1
orange.jpeg
Image 2
mask applied (vertical split)
Image 3
oraple!
Image 4
pizza 1
Image 1
pizza 2
Image 2
mask applied (vertical split)
Image 3
half-half pizza
Image 4
lego storm trooper
Image 1
background (airplane wing)
Image 2
mask applied (stormtrooper figure)
Image 3
figure on wing
Image 4

Reflection

It was really fun to implement these filters and manipulate the frequency of the images. What I enjoyed most was making the oraple, because this combined the other techniques I learned about filters and low vs high frequencies to make a seamless image.