# withoutBG Pro Model Self-Host: API for AI coding assistants

You are integrating a self-hosted background-removal HTTP service. This file is the
complete contract. The running container serves the same text at `GET /llms.txt`
and the machine-readable spec at `GET /openapi.json`.

Claude Code: add `@AGENTS.md` to the project's `CLAUDE.md`. Cursor, Codex and others
read `AGENTS.md` from the repo root.

## Facts

- Base URL: `http://<host>:8000` (default `http://localhost:8000`). Plain HTTP.
- No authentication. Do not send an API key or `Authorization` header.
- `GET /ready` returns `200 {"status":"ready"}` once the model is loaded and `503`
  before (loading takes up to a minute after start). Wait for it before the first request.
- Each container processes one image at a time; other requests queue. Use a client
  timeout of at least 120 s.
- Input: JPEG, PNG, WebP or anything Pillow reads, any size. EXIF orientation is
  applied. Output has the input's size (after orientation).
- All result images are PNG. The cutout is RGBA. The matte is a black-and-white PNG
  (stored as RGB, all three channels equal): read one channel as alpha 0..255.

## Which endpoint

| Client | Endpoint |
|---|---|
| Server code with image bytes, wants a PNG back | `POST /v1/remove-background` (raw bytes or multipart) |
| JSON-only (browser, workflow tool, serverless) | `POST /v1/remove-background/base64` |
| Smart Brush editor | either; `/base64` returns `matte` with every call |

## `POST /v1/remove-background`

Query: `output=cutout` (default, transparent PNG) or `output=matte`. Any other value: 422.

Body, one of:

1. Raw image bytes, `Content-Type: image/jpeg` (or `image/png`, ...). Response: PNG.
2. `multipart/form-data`: `image` (required file), optional files `keep_mask`,
   `remove_mask`, `matte`. Response: PNG.
3. `application/json`: `{"image", "keep_mask"?, "remove_mask"?, "matte"?}`, each a
   data URL (`data:image/png;base64,...`) or raw base64. Response JSON
   (`output` is ignored):
   `{"processed": "data:image/png;base64,...", "alphaMatte": "data:image/png;base64,...", "latencyMs": int, "routeCategory": str|null, "routePipeline": str|null}`
   `processed` is the cutout, `alphaMatte` the matte.

PNG responses carry headers `X-Latency-Ms`, `X-Route-Category`, `X-Route-Pipeline`.

## `POST /v1/remove-background/base64`

Request JSON:

| Field | Type | Required | Meaning |
|---|---|---|---|
| `image` | string | yes | Base64 image (a `data:` URL is also accepted) |
| `keep_mask` | string | no | Base64 Smart Brush Keep mask |
| `remove_mask` | string | no | Base64 Smart Brush Remove mask |
| `matte` | string | no | Base64 matte from a previous response; needs a mask |
| `output` | `"cutout"` \| `"matte"` | no | What `image` holds in the response (default `cutout`) |

Response JSON:
`{"image": "<base64 PNG>", "matte": "<base64 PNG>", "latencyMs": int, "routeCategory": str|null, "routePipeline": str|null}`.
Plain base64, no `data:` prefix. `matte` is always present.

## Smart Brush (Keep / Remove)

Fixes a result. Send the original photo plus one or both masks:

- Masks are images exactly the size of the photo after EXIF orientation. Do not
  resize or crop them. A size mismatch returns 422.
- A pixel is marked when its luminance is 128 or more after compositing onto black:
  white strokes on black, or white strokes on transparent, both work.
- `keep_mask`: marked parts must stay. `remove_mask`: marked parts become
  transparent. Where both mark a pixel, Remove wins.
- Edits are object-level: one short stroke on an object keeps or removes all of it.
- `matte` (optional): the matte the server returned last for this photo. With it,
  earlier edits are kept and only the new strokes apply. Without it, the photo is
  matted from scratch and then edited. `matte` without any mask: 422.
- The server keeps no state. Send the photo with every call.

Editor loop: show cutout → user paints strokes → send photo + strokes + last
`matte` → show new cutout, keep the new `matte` → repeat.

## Errors

| Status | When | Body |
|---|---|---|
| 400 | Not an image, empty body, missing multipart `image` | binary: `{"error": "..."}`; JSON: `{"detail": "..."}` |
| 422 | Bad mask or matte (not an image, wrong size), `matte` without a mask | binary: `{"error": "..."}`; JSON: `{"detail": "..."}` |
| 422 | Invalid `output`, JSON body missing `image` | `{"detail": [ ...validation errors... ]}` |
| 503 | Model still loading | `{"error": "Model not ready"}` or `{"detail": "Model not ready"}` |

Retry only 503 (wait a few seconds). 400 and 422 fail the same way on retry.

## Other endpoints

- `GET /health`: liveness, `{"status":"ok","model":"withoutbg-pro-self-host","version":"..."}`.
- `GET /docs`: Swagger UI (works offline). `GET /openapi.json`: OpenAPI 3 spec.
- `GET /v1/licenses`: license summary and upstream components.

## Python (requests)

```python
import base64, requests

BASE = "http://localhost:8000"

# Cutout PNG from raw bytes
with open("photo.jpg", "rb") as f:
    r = requests.post(f"{BASE}/v1/remove-background", data=f.read(),
                      headers={"Content-Type": "image/jpeg"}, timeout=180)
r.raise_for_status()
open("cutout.png", "wb").write(r.content)

# Base64 + Smart Brush edit, keeping the matte for the next edit
b64 = lambda p: base64.b64encode(open(p, "rb").read()).decode()
first = requests.post(f"{BASE}/v1/remove-background/base64",
                      json={"image": b64("photo.jpg")}, timeout=180).json()
edit = requests.post(f"{BASE}/v1/remove-background/base64", json={
    "image": b64("photo.jpg"),
    "remove_mask": b64("remove.png"),
    "matte": first["matte"],
}, timeout=180).json()
open("edited.png", "wb").write(base64.b64decode(edit["image"]))
```

## Node 18+ (fetch)

```js
import { readFile, writeFile } from "node:fs/promises";

const BASE = "http://localhost:8000";
const res = await fetch(`${BASE}/v1/remove-background`, {
  method: "POST",
  headers: { "Content-Type": "image/jpeg" },
  body: await readFile("photo.jpg"),
  signal: AbortSignal.timeout(180_000),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
await writeFile("cutout.png", Buffer.from(await res.arrayBuffer()));
```

## curl

```bash
curl -s -H 'Content-Type: image/jpeg' --data-binary @photo.jpg \
  localhost:8000/v1/remove-background -o cutout.png
curl -s -F image=@photo.jpg -F remove_mask=@remove.png \
  localhost:8000/v1/remove-background -o fixed.png
```

## Do not

- Do not add authentication headers, API keys or a `refine` parameter: none exist.
- Do not resize masks or the matte to a different size than the photo.
- Do not send many requests in parallel to one container expecting speedup; scale
  with more containers behind a load balancer (health check: `GET /ready`).
- Do not expose the service to the internet without your own reverse proxy (TLS,
  auth, body size limit, timeout of 120 s or more).
