Trimaps in Image Matting

Author: Imran Kocabiyik

Image without background

Image matting is a difficult problem. Therefore, many image matting algorithms require a trimap as an additional input apart from a three-channel RGB image to make the problem less difficult.

Trimaps

Trimap is a one-channel map representing the absolute background, foreground, and unknown region. It is primarily human-annotated. However, some supervised and unsupervised machine learning solutions to generate them. Nevertheless, the quality might be comparable with human annotation.

Example Trimap

Below is a sample image and a corresponding trimap. Image and Trimap

Sample Application Here is the foreground selection tool interface of the GIMP image editing program.

GIMP Foreground Selection

Generating Trimap From Alpha Mask

To generate a trimap from a segmentation mask, you can use a simple technique called dilation and erosion. First, dilate the mask to expand the regions of foreground and background pixels. Then, erode the mask to shrink the regions of foreground and background pixels. The resulting image will be the trimap, with the foreground and background pixels represented by white and black pixels, respectively, and the unknown pixels represented by gray pixels. Here is a function that extracts trimap from the alpha channel (Python).

import numpy as np
import cv2 as cv

def generate_trimap(alpha):
   k_size = random.choice(range(2, 5))
   iterations = np.random.randint(5, 15)
   kernel = cv.getStructuringElement(cv.MORPH_ELLIPSE, (k_size, k_size))
   dilated = cv.dilate(alpha, kernel, iterations=iterations)
   eroded = cv.erode(alpha, kernel, iterations=iterations)
   trimap = np.zeros(alpha.shape, dtype=np.uint8)
   trimap.fill(128)

   trimap[eroded >= 255] = 255
   trimap[dilated <= 0] = 0

   return trimap

Generate Trimaps from Images

Trimap annotation is time-consuming, and users don't like to spend time on it. So to build a trimap-free solution, we need to drop it or generate it with other methods. There are two main approaches:

  1. Generating trimaps with unsupervised machine learning techniques
  2. Generating binary masks with supervised ML models and generating trimap with morphological operations in computer vision.

Scribble as an Alternative to the Trimap

A less time-consuming alternative is a scribble. It takes less effort to annotate the image.