Description
1 Assignment description
Image stitching is a technique to combine a set of images into a larger image by registering, warping, resampling
and blending them together. A popular application for image stitching is creation of panoramas. Generally
speaking, there are two classes of methods for image stitching, direct methods and feature-based methods. In
this assignment, we will implement a feature-based method. It contains three parts: i) feature detection and
matching, ii) homography estimation and iii) blending.
2 Assignment details
2.1 Feature detection and matching
Given two input images, detect SIFT features from them. And then establish the feature correspondence
between SIFT features in the two input images using Brute-Force matcher. In OpenCV, there is a tutorial for
SIFT https://docs.opencv.org/3.4/da/df5/tutorial_py_sift_intro.html, and a tutorial for Brute-Force
matcher https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html.
This part is corresponding to extract and match feature function in the provided skeleton code:
def e x t r a c t a n d m a t c h f e a t u r e ( img 1 , img 2 , r a t i o t e s t = 0. 7 ):
”””
: param img 1 : i n p u t image 1
: param img 2 : i n p u t image 2
: param r a t i o t e s t : r a t i o f o r t h e r o b u s t n e s s t e s t
: r e t u r n l i s t p a i r s m a t c h e d k e y p o i n t s : a l i s t o f p a i r s o f matched p o i n t s :
[ [ [ p1x , p1y ] , [ p2x , p2y ] ] ]
”””
l i s t p a i r s m a t c h e d k e y p o i n t s = [ ]
# t o be c om ple te d . . . .
return l i s t p a i r s m a t c h e d k e y p o i n t s
You need to do the following things in this function:
1) extract SIFT feature from image 1 and image 2,
2) use a bruteforce search to find pairs of matched features: for each feature point in img 1, find its best
matched feature point in img 2,
3) apply ratio test to select the set of robust matched points.
2.2 Homography estimation
The 2D image transformations (translation, rotation, scale, affine and perspective) can be represented as homography. Therefore, we can estimate the best homography by the matched pairs of features. Also, we employ
RANSAC algorithm to eliminate the ”bad” matches. This can make our program more robust. In this section,you need to implement the RANSAC algorithm to find a robust homography between two input images
using the feature correspondence.
This part is corresponding to find homography ransac function in the provided skeleton code:
1
5 – Assignment 2: Image Stitching 2
def fi n d h om o g r a p h y r a n s a c ( l i s t p a i r s m a t c h e d k e y p o i n t s ,
t h r e s h o l d r a t i o i n l i e r s =0.85 ,
t h r e s h o l d r e p r o j e c t i o n e r r o r =3,
m ax num t ri al =1000):
”””
: param l i s t p a i r s m a t c h e d k e y p o i n t s :
a l i s t o f p a i r s o f matched p o i n t s : [ [ [ p1x , p1y ] , [ p2x , p2y ] ] , . . . ]
: param t h r e s h o l d r a t i o i n l i e r s :
t h r e s h o l d on t h e r a t i o o f i n l i e r s ove r t h e t o t a l number o f samples ,
a c c e p t t h e e s t im a t e d homography i f r a t i o i s h i g h e r than t h e t h r e s h o l d
: param t h r e s h o l d r e p r o j e c t i o n e r r o r :
t h r e s h o l d o f r e p r o j e c t i o n e r r o r ( measured as e u c l i d e a n d i s t a n c e , in
p i x e l s ) t o de te rm ine w he t her a sample i s i n l i e r or o u t l i e r
: param m ax n um t r i al :
t h e maximum number o f t r i a l s t o t a k e sample and do t e s t i n g t o f i n d
t h e b e s t homography m a t r ix
: r e t u r n b e s t H : t h e b e s t found homography m a t r ix
”””
be s t H = None
# t o be c om ple te d . . .
return be s t H
2.3 Blending
Having the estimated homography, we can warp the second image to align with the first image and then
blend them together to get a a single panorama image. For warping, we employ inverse warping and bilinear
resampling.
This part is corresponding to warp blend image function in the provided skeleton code:
def w a rp blend im a ge ( img 1 , H, img 2 ) :
”””
: param img 1 : t h e o r i g i n a l f i r s t image
: param H: e s t im a t e d homography
: param img 2 : t h e o r i g i n a l second image
: r e t u r n img panorama : r e s u l t i n g panorama image
”””
img panorama = None
# t o be c om ple te d . . .
return img panorama
You need to do the following things in this function:
1) warp image img 2 using the homography H to align it with image img 1 (using inverse warping and bilinear
resampling),
2) stitch image img 2 to image img 1 and apply average blending to blend the two images into a single
panorama image.
5 – Assignment 2: Image Stitching 3
3 Submission guidelines
3.1 Marks
* extract and match feature: 40 points
* find homography ransac: 30 points
* warp blend image: 30 points
3.2 Put personal information in your source code
In all your source files, type your full name and student ID, just like:
#
# CSCI3290 Compu ta t ional Imag ing and V i s i on ∗
# −−− D e cl a r a t i o n −−− ∗
# I d e c l a r e t h a t t h e a s s ignmen t he re s u bm i t t e d i s o r i g i n a l e x c e p t f o r s o u rc e
# m a t e r i a l e x p l i c i t l y ackn owle dge d . I a l s o ackn owle dge t h a t I am aware o f
# U n i v e r s i t y p o l i c y and r e g u l a t i o n s on h o n e s ty in academ ic work , and o f t h e
# d i s c i p l i n a r y g u i d e l i n e s and p r o c e d u r e s a p p l i c a b l e t o b r e a c h e s o f such p o l i c y
# and r e g u l a t i o n s , as c o n t a i n e d in t h e w e b s i t e
# h t t p : / /www . cuhk . edu . hk / p o l i c y / ac a dem ic h one s ty / ∗
# Ass ignmen t 2
# Name :
# S t u de n t ID :
# Email Addr :
#
3.3 Late submission penalty
If you submit your solution after the due date, 10 marks will be deducted per day, and the maximal deduction
is 30 marks even you delay more than 3 days. But there are hard deadlines as we need time to grade and submit
grade. The hard deadline is 29 April 2020. After this day, we will not grade your assignment.
