This project came from a computer vision class where we built an image stitching pipeline using Harris corners and SIFT descriptors. I’m keeping the original code mostly intact to show how I approached it at the time, but I’ve added some thoughts below on what I’d do differently now.

The goal was to align and blend multiple overlapping images into a panorama, using classic computer vision techniques (no deep learning). This post goes over what I implemented, what I’ve learned since, and what I’d change if I revisited it today.

What This Project Does

Detects corners in each image using Harris corner detection
Computes SIFT descriptors around those corners
Matches features between image pairs
Estimates the homography using RANSAC
Warps and blends images into a stitched result
Supports stitching multiple images in a sequence

Here’s what the stitched output looks like:

What I Did (Original Approach)

At the time, my focus was on getting the pipeline working and matching the spec. I handled corner detection with cv2.cornerHarris, filtered out low responses, then extracted SIFT descriptors.

A_gray = cv2.cvtColor(A, cv2.COLOR_BGR2GRAY)
A_corners = cv2.cornerHarris(A_gray, blockSize=2, ksize=3, k=0.05)
# Using Harris corner detector on grayscale images A and B
keypoints = [cv2.KeyPoint(x, y, radius) for x, y in corner_locs]
_, descriptors = sift.compute(A_gray, keypoints)

I matched descriptors using OpenCV’s brute-force matcher with cross-checking, took the top 200 matches, and then used RANSAC to compute a homography.

bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True)
matches = bf.match(desc1, desc2)
matches = sorted(matches, key=lambda x: x.distance)

# Estimate homography from matched keypoints using RANSAC
H, mask = cv2.findHomography(dst_pts, src_pts, cv2.RANSAC, 0.3)

After computing the homography, I warped one image into the other's frame and blended them together. For multiple images, I ran all pairwise combinations, selected the pairs with the most inliers, and stitched them in that order. It worked, though it was a bit rigid.

warped = cv2.warpPerspective(new_img, H, canvas_size)
mask = (warped > 0)
panorama[mask] = warped[mask]

What I’d Do Differently Now

Since submitting this assignment, I’ve learned a few things that would make the whole pipeline more reliable and scalable. If I were to revisit this, here’s how I’d improve it:

Preprocessing

I didn’t smooth the grayscale image before running Harris detection, which can cause noisy responses. I’d apply a Gaussian blur (σ=2) beforehand now.
Harris should run on float32, but I passed in the raw grayscale output.

Keypoints and Descriptors

Instead of manually detecting corners and computing descriptors, I’d just use SIFT’s detectAndCompute() which handles both in a consistent way.
It’s more robust to scale and rotation than manually sampling Harris corners.

Matching

Rather than using brute-force matching with cross-checking and slicing the top 200 matches, I’d switch to KNN matching with Lowe’s ratio test. It does a better job at filtering out bad matches.
I’d also add a sanity check to make sure there are at least 4 good matches before trying to compute a homography.

Multi-Image Stitching

Instead of planning out the full stitching sequence at the start, I’d do it recursively—merge the best pair, then re-run matching with the merged image and the rest.
That would be more adaptive and flexible, especially for larger sets of images.

Blending

Right now, I just overwrite the overlap with the warped image. It works, but it creates hard seams. I’d do linear blending in the overlapping region or look into OpenCV’s multi-band blending for smoother transitions.

Why I’m Keeping It As-Is

Even though I know how to make it better now, I’m keeping this version mostly untouched for two reasons:

It shows where I was at the time. I think it’s important to document progress honestly.
The code still works, and I learned a lot from writing it—even if I’d approach it differently today.

There’s value in being able to look back at your work and recognize areas for improvement. That’s part of the learning process.

Final Thoughts

This was one of the first projects where I had to take a full vision pipeline from scratch to output. It’s far from perfect, but it got me thinking critically about matching, transformation, and how to go from theory to implementation.

If you want to check out the code or try it yourself, here’s the GitHub repo:
GitHub - image-stitching-harris-sift

Stitching Images with Harris Corners and SIFT: A Homework Retrospective

Table of contents