# Alpha matting vs image segmentation

Segmentation decides which pixels belong to the subject. Alpha matting decides how much each pixel belongs to it. The difference is invisible on a product with a sharp outline and obvious on hair, fur, glass and smoke.

## Binary mask vs alpha matte

A segmentation mask has two values per pixel: foreground or background. An alpha matte has a value between 0 and 1: 0 is background, 1 is foreground, anything in between is a pixel that mixes both.

```python
# C: composited pixel, F: foreground colour, B: new background colour
# alpha is in [0, 1] for matting, and only 0 or 1 for a binary mask
C = alpha * F + (1 - alpha) * B
```

With a binary mask the in-between pixels are rounded to 0 or 1. On a hair strand that gives a hard, jagged edge or a halo of the old background. Many matting methods take a [trimap](https://withoutbg.com/tech/trimap) (foreground, background, unknown) as a hint; automatic background removal models predict the matte without one.

## How edge quality is measured

[Bench v0](https://withoutbg.com/benchmarks/open-background-removal.md) compares each model's predicted alpha matte with a ground-truth matte on real photos, both scored at a 1024 px shorter side (never upscaled). All three are lower-is-better:

- **MAE**: mean absolute alpha error over every pixel. Large flat areas dominate this average.
- **MGE**: error between the Sobel gradient magnitudes of prediction and ground truth. It reacts to edges that are too hard, too soft or misplaced.
- **Band MAE**: MAE only inside a 10 px band (at 1024 px) around the matte edge.

For current scores of every model (MAE, MGE, band MAE and ranks) see the [models leaderboard](https://withoutbg.com/models.md) and the [benchmark](https://withoutbg.com/benchmarks/open-background-removal.md). The HTML page shows the same scores sorted by edge error, with the router disclosure for the withoutBG pipelines, which use BiRefNet for some images.

## FAQ

**What is the difference between alpha matting and image segmentation?** Segmentation labels every pixel as foreground or background. Alpha matting predicts an opacity between 0 and 1 per pixel, so a hair strand or a glass edge can be partly foreground.

**Why do binary masks look wrong on hair, fur and glass?** At those edges one pixel mixes foreground and background colour. A binary mask must call it one or the other, which leaves a jagged edge or a halo of the old background on a new backdrop.

**What do MAE, MGE and band MAE measure?** MAE is the mean alpha error over all pixels, MGE compares Sobel gradient magnitudes (edge sharpness and placement), and band MAE is MAE within a 10 px band around the matte edge.

## Related

- [Bench v0 benchmark](https://withoutbg.com/benchmarks/open-background-removal)
- [Open background removal models](https://withoutbg.com/models)
- [Trimaps in image matting](https://withoutbg.com/tech/trimap)
- [Alpha matting evaluation benchmark](https://withoutbg.com/tech/alpha-matting-evaluation-benchmark)
