How AI Background Removal Works (and Why Ours Runs in Your Browser)

Not long ago, removing a background meant tracing round the subject in Photoshop with the pen tool, which could easily eat half an hour per photo. Now it takes a couple of seconds. Here's what happens in those seconds, without the math, and how this site manages to do it inside your browser.

The task: separating subject from background

In computer vision, splitting an image into meaningful regions is called segmentation. Background removal is a specific kind of it: for each pixel, decide whether it belongs to the main subject or to the background. The result is a mask, a greyscale image the same shape as the photo where white means "keep", black means "remove" and grey means "partly".

Older tools used simple rules: remove everything close to a chosen color (the way green screens work), or follow strong edges. Those rules break down with real-world photos, where the background has many colors and the subject's edges are soft. Modern tools use neural networks that have learned what subjects look like from many thousands of example photos paired with hand-made masks.

Salient object detection

The models used here answer a question you answer without thinking: what is this photo of? Researchers call it salient object detection, finding the object that naturally draws the eye. A salient object model doesn't need to know that something is a shoe or a cat. It only needs to find the thing that stands out.

This site uses models from the U²-Net family, published by Xuebin Qin and colleagues in 2020 ("U²-Net: Going Deeper with Nested U-Structure for Salient Object Detection"). The name describes its shape. A U-Net first shrinks the image step by step to understand the big picture, then grows it back up to recover fine detail, passing information across at each level. U²-Net nests smaller U-shaped blocks inside the big one, so it can look at both coarse shapes and fine edges at every stage.

What happens when you drop in a photo

1. Resize for the model. The network works on a small square version of your photo, 320 × 320 pixels. Working small keeps it fast enough to run on a phone.
2. Normalize. Pixel colors are converted into the numeric range the model was trained on.
3. Run the network. The model outputs a value for every pixel of that small square: how likely it is to be part of the main subject.
4. Clean up the mask. The values are rescaled into a 0 to 255 mask, and small detached blobs that clearly aren't part of the subject are dropped.
5. Scale the mask up, following your photo's edges. The mask is enlarged back to your photo's original size with an edge-aware filter, which uses the full photo to move the outline onto the real boundary between subject and background instead of just stretching it. Your photo itself is never shrunk, so the download keeps every pixel of the original resolution. (See why cutouts get blurry edges.)
6. Apply it as alpha. The mask becomes the image's alpha channel, its transparency. If you choose a background color, the subject is layered over that color. (More on alpha in transparent PNG explained.)

Two models, two trade-offs

You get a choice of two:

• Standard uses U²-Netp, a lightweight version of the network that is only a few megabytes. It loads almost instantly and works on any kind of subject: products, pets, cars, people.
• Better model uses a full-size U²-Net trained specifically to segment people, compressed to 8-bit numbers ("INT8 quantization") so it downloads faster. It's a bigger one-time download and is noticeably better on portraits, but it's specialized, so use Standard for objects.

Why it runs in your browser

Most online background removers upload your photo to a server, run the model there and send the result back. Here, it's the other way round: your browser downloads the model once, and the work runs on your own device using ONNX Runtime Web, an open-source engine that executes neural networks with WebAssembly.

Doing it this way has three upsides:

• Privacy. Your photo is never sent to a server, so there's nothing for us to store, leak or reuse. That matters for passport scans, family photos and unreleased products.
• No queue and no limits. Your device does the work, so there are no credits and no waiting behind other users.
• It keeps working. After the model has loaded, processing doesn't depend on a fast connection.

The catch is that speed depends on your device. A recent laptop usually needs a second or two; an older phone can take several. The very first photo also waits for the model to download.

Where AI still struggles

Once you know how the model works, its weak spots make sense:

• Fine hair and fur. The mask is predicted at 320 × 320. Edge-aware enlarging recovers a lot of the outline, but single strands finer than the small mask can show may still be lost.
• Low contrast. If subject and background look alike, the model has little to go on.
• Several subjects. Salient object detection looks for the main thing, so secondary objects may be kept or dropped unpredictably.
• Transparent objects. Glass shows the background through itself, so no mask can fully separate them.

Most of these can be improved by how you take the photo. Our 10 tips for cleaner cutouts cover what to do.