vismatch.base_matcher

class vismatch.base_matcher.BaseMatcher(device='cpu', **kwargs)

Bases: Module

This serves as a base class for all matchers. It provides a simple interface for its sub-classes to implement, namely each matcher must specify its own __init__ and _forward methods. It also provides a common image_loader and homography estimator

Parameters:

device (str)

device: str

skip_ransac: bool

ransac_iters: int

ransac_conf: float

ransac_reproj_thresh: float

property name: str

static load_image(path, resize=None, rot_angle=0)

load image from filesystem and return as tensor. Optionally rotate and resize.

Parameters:

• path (str | Path) – path to image on filesystem

• resize (int | tuple, optional) – size to resize img, either single value for square resize or tuple of (H, W). Defaults to None.

• rot_angle (float, optional) – CCW rotation angle in degrees. Defaults to 0.

Returns:

image as tensor (C x H x W)

Return type:

torch.Tensor

rescale_coords(pts, h_orig, w_orig, h_new, w_new)

Rescale kpts coordinates from one img size to another

Parameters:

• pts (np.ndarray | torch.Tensor) – (N,2) array of kpts

• h_orig (int) – height of original img

• w_orig (int) – width of original img

• h_new (int) – height of new img

• w_new (int) – width of new img

Returns:

(N,2) array of kpts in original img coordinates

Return type:

np.ndarray

compute_ransac(matched_kpts0, matched_kpts1)

Process matches into inliers and the respective Homography using RANSAC.

Parameters:

• matched_kpts0 (np.ndarray) – matching kpts from img0

• matched_kpts1 (np.ndarray) – matching kpts from img1

Returns:

(3 x 3) homography matrix from img0 to img1. Can be None if no homography is found inlier_kpts0 (np.ndarray): inlier kpts in img0 inlier_kpts1 (np.ndarray): inlier kpts in img1

Return type:

H (np.ndarray)

forward(img0, img1)

Run matching pipeline on two images. All sub-classes implement this interface.

Parameters:

• img0 (torch.Tensor | np.ndarray | str | Path | Image.Image) – image as (3, H, W) array in [0, 1] range, path, or PIL Image

• img1 (torch.Tensor | np.ndarray | str | Path | Image.Image) – image as (3, H, W) array in [0, 1] range, path, or PIL Image

Returns:

result dict with keys:

• num_inliers (int): number of inliers after RANSAC, i.e. len(inlier_kpts0)

• H (np.ndarray): (3 x 3) homography matrix to map matched_kpts0 to matched_kpts1

• all_kpts0 (np.ndarray): (N0 x 2) all detected keypoints from img0

• all_kpts1 (np.ndarray): (N1 x 2) all detected keypoints from img1

• all_desc0 (np.ndarray): (N0 x D) all descriptors from img0

• all_desc1 (np.ndarray): (N1 x D) all descriptors from img1

• matched_kpts0 (np.ndarray): (N2 x 2) keypoints from img0 that match matched_kpts1 (pre-RANSAC)

• matched_kpts1 (np.ndarray): (N2 x 2) keypoints from img1 that match matched_kpts0 (pre-RANSAC)

• inlier_kpts0 (np.ndarray): (N3 x 2) filtered matched_kpts0 that fit the H model (post-RANSAC)

• inlier_kpts1 (np.ndarray): (N3 x 2) filtered matched_kpts1 that fit the H model (post-RANSAC)

Return type:

dict

extract(img)

Extract keypoints and descriptors from a single image.

Parameters:

img (torch.Tensor | np.ndarray | str | Path | Image.Image) – image as (3, H, W) array in [0, 1] range, path, or PIL Image

Returns:

result dict with keys:

• all_kpts0 (np.ndarray): (N, 2) detected keypoints

• all_desc0 (np.ndarray): (N, D) descriptors

Return type:

dict

static get_empty_array_if_none(array)

Parameters:

array (ndarray | None)

Return type:

ndarray

static check_types(matched_kpts0, matched_kpts1, all_kpts0, all_kpts1, all_desc0, all_desc1)

Check that objects are of accepted types (nd.array, torch.tensor or None)

static check_shapes(matched_kpts0, matched_kpts1, all_kpts0, all_kpts1, all_desc0, all_desc1)

Check that objects have appropriate shapes, e.g. keypoints should have shape (N, 2)

class vismatch.base_matcher.EnsembleMatcher(matcher_names=[], device='cpu', **kwargs)

Bases: BaseMatcher

Parameters:

• matcher_names (list[str])

• device (str)