Tissue masking & tiling

Before any embedding or gene-expression model runs, two preprocessing steps prepare the slide:

Tissue masking — a whole-slide image is mostly background. Tissue segmentation produces a binary mask so background is discarded before the expensive feature step.
Tiling — the slide is too large to process at once, so it is split into small, model-sized tiles. Only tiles that overlap tissue are kept.

Resolutions differ by stage: tissue segmentation runs at 8 µm/pixel (coarse, 512×512 tiles), while embeddings and gene predictions run at 0.5 µm/pixel (224×224 tiles). The mask is generated once and reused to filter the fine-resolution grid.

The Inference pipeline does both steps for you. Reach for the manual API below only when you need masks or tiles as standalone artifacts — for QC, custom filtering, or your own pipeline.

The quick path: the Inference pipeline

Inference masks tissue, caches the result, and reuses it automatically for later embed/predict calls.

from bioptimus.inference import Inference
from bioptimus.models.types import Models
from bioptimus import utils

infer = Inference(
    model_name=Models.H1,
    api_url="http://localhost:8080",
    tissue=True,
    mask_threshold=0.5,
    output_path="/data/output",
    experiment="preprocessing-demo",
    run=1,
)

mask = infer.tissue(wsi_path)   # uint8 (H, W) array; cached to <workspace>/tissue/<slide>.png

The mask is cached to <workspace>/tissue/<slide>.png and reused on the next tissue(), embed(), or predict() call. Visualize it against the slide thumbnail with the built-in helper:

mask_png = "/data/output/preprocessing-demo/run_1/tissue/<slide>.png"
utils.plot_slide_and_mask(wsi_path, mask_png, threshold=0.5)

Slide thumbnail alongside its binary tissue mask — Tissue mask (right) beside the slide thumbnail (left), TCGA-LUAD TCGA-75-7027. Background is discarded before feature extraction. Representative example.

The manual path: masks and tiles as artifacts

1. Generate a tissue mask

Build a mask provider from the tissue-seg endpoint. It is reusable across slides — it generates a fresh mask per slide.

from bioptimus.io.wsi import WSI
from bioptimus.models.backbones import Backbone
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask

tissue_backbone = Backbone("tissue-seg", base_url="http://localhost:8080")
provider = TiledTissueMask(model=BioptimusTissueMaskModel(tissue_backbone))

wsi = WSI(wsi_path)
mask = provider.generate(wsi)   # uint8 (H, W) in {0, 1} over the tissue bounding box at 8 µm/pixel

Already have masks on disk? Use them directly instead of the endpoint. Non-zero pixels mean tissue, and files are matched to slides by filename stem (PNG, JPEG, TIFF, BMP, or .npy):

from bioptimus.preprocess.wsi.provider.precomputed import PrecomputedTissueMask

provider = PrecomputedTissueMask(mask_dir="masks/", suffix=".png")

Bring your own segmentation (custom mask provider)

Need a different segmentation model — for example one that is robust to marker-pen artifacts, or one you already run in-house? Subclass TissueMaskProvider and return your own mask. TileExtractor and the Inference pipeline depend only on this interface, so any provider — algorithmic, model-based, or a wrapper around an external tool — plugs in unchanged.

import numpy as np

from bioptimus.io.wsi.interface import WSIReader
from bioptimus.observability import log_failures
from bioptimus.preprocess.wsi.provider.base import TissueMaskProvider


class CustomMaskProvider(TissueMaskProvider):
    """Wraps an external segmentation model as a tissue mask provider."""

    def __init__(self, segmenter):
        self._segmenter = segmenter

    @log_failures
    def generate(self, reader: WSIReader, *, show_progress: bool = True) -> np.ndarray:
        # Run your own segmentation and return a uint8 (H, W) mask, 1 = tissue.
        # The mask must cover the same extent the extractor scales it against:
        # the slide's tissue-bounded region, reader.dimensions(bounded=True).
        mask = self._segmenter.run(reader.path)
        return (mask > 0).astype(np.uint8)

What the mask must be. A 2-D uint8 (H, W) array where 1 = tissue. Two rules keep it aligned with the tiling grid:

Resolution is flexible. The mask does not need to match the slide’s pixel dimensions or any specific µm/pixel — TileExtractor resizes it (nearest-neighbour) to the reference level before filtering. A mask produced at 2 µm/pixel therefore aligns correctly against a 0.5 µm/pixel tile grid.
Extent must match, not just aspect ratio. The mask is stretched onto the slide’s tissue-bounded region — reader.dimensions(bounded=True). For slides with no scan-area metadata this equals the full slide, so a full-slide mask is correct. For slides that do carry bounds metadata (e.g. some scanner formats), bounded=True is a crop; a mask spanning the whole slide would be squashed into that crop and misalign. Produce the mask over the same region your segmenter sees, and compare reader.dimensions(bounded=True) with reader.dimensions(bounded=False) if unsure.

As a guard, if the mask’s aspect ratio differs from the target region’s by more than 5%, a MASK_ASPECT_MISMATCH warning is emitted so a transposed or wrong-slide mask does not pass silently.

Use the provider one of two ways: generate the masks up front, save them to a directory, and point Inference at that directory with tissue=False (shown next), or pass it straight to TileExtractor(mask_provider=...) when tiling manually (see Tile the slide below). To confirm alignment visually, save the mask and overlay it on the slide thumbnail with utils.plot_slide_and_mask (see the quick path above).

Hybrid pipeline: local mask, remote embeddings

Because masking and embedding are independent stages, you can produce tissue masks locally with a custom provider while sending the surviving tiles to a remote endpoint (e.g. AWS SageMaker) for embeddings. This is a two-phase workflow: first generate and save the masks, then run inference against them with tissue=False so the built-in tissue-seg model does not recompute over your masks. Phase 1 — generate the masks with your custom provider and save them by slide stem:

from pathlib import Path

from PIL import Image

from bioptimus.io.wsi import WSI

provider = CustomMaskProvider(my_local_segmenter)
wsi_dir, mask_dir = Path("/data/wsis"), Path("/data/masks")
mask_dir.mkdir(parents=True, exist_ok=True)

for wsi_path in wsi_dir.glob("*.svs"):
    reader = WSI(str(wsi_path))
    mask = provider.generate(reader)              # runs YOUR segmentation; uint8 (H, W)
    # Save named by slide stem; *255 keeps it visible and round-trips through (loaded > 0).
    Image.fromarray(mask * 255).save(mask_dir / f"{wsi_path.stem}.png")

Phase 2 — run the pipeline against those masks, with embeddings on the remote endpoint:

from bioptimus.inference import Inference
from bioptimus.models import Models

infer = Inference(
    model_name=Models.H1,
    backend="aws",                 # remote embeddings on SageMaker
    endpoint_name="my-endpoint",
    wsi_dir="/data/wsis",
    mask_dir="/data/masks",        # your custom masks, paired to slides by filename stem
    tissue=False,                  # do NOT recompute masks with the built-in tissue-seg model
)
infer.find()                       # auto-pairs each slide with its mask by stem
infer.embed()

mask_dir pairs each mask to its slide by filename stem via the built-in PrecomputedTissueMask provider, and tissue=False ensures your masks are used as-is. The embedding backend (backend="aws") is chosen independently of how the mask is produced.

Inference over a slide is atomic — if any tile fails after retries, that slide’s output is discarded. The Cohort workflow resumes per slide, so a single failure does not restart the whole cohort.

2. Tile the slide

A TileSpec defines the tile geometry; TileExtractor builds the grid, filters it against the mask, and exports the tiles. Passing mask_provider lets the extractor generate the per-slide mask during fit.

from bioptimus.io.wsi import MPP, MeasurementUnit
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.extraction.wsi.tile_extraction import TileExtractor

spec = TileSpec(
    size=(224, 224),
    stride=(224, 224),
    resolution=MPP(0.5),
    unit=MeasurementUnit.PIXELS,
)

extractor = TileExtractor(tile_spec=spec, mask_provider=provider, mask_threshold=0.5)
tiles = extractor.fit_extract(wsi)   # keeps only tiles with >= 50% tissue

extractor.save("tiles/", image_format="png", workers=4)   # writes tile images
extractor.to_csv("tiles/")                                # writes tiles/<slide>.csv
print(f"{len(tiles)} tiles kept")

to_csv writes one row per retained tile:

Column	Description
`slide_name`	Slide filename stem.
`top`, `left`	Tile top-left location in absolute slide coordinates.
`width`, `height`	Tile size.
`unit`	Measurement unit (`PIXELS` or `UM`).
`resolution_type`, `resolution_value`	Resolution the tile is defined at (e.g. `mpp` / `0.5`).
`tissue_ratio`	Fraction of the tile covered by tissue.

save writes each tile image as {slide}_x{left}_y{top}_w{width}_h{height}_{res}_mask_r{ratio}.png.

Tuning knobs

Knob	Where	Effect
`mask_threshold`	`Inference`, `TileExtractor`	Minimum tissue fraction (0–1) to keep a tile. Lower keeps more border tiles; `None` disables filtering.
`size` / `stride`	`TileSpec`	Tile size and step. Equal size and stride give a non-overlapping grid; a smaller stride overlaps tiles.
`resolution` / `unit`	`TileSpec`	Physical resolution of tiles. Bioptimus feature models expect 224×224 at 0.5 µm/pixel.
`max_concurrency`	`TiledTissueMask.generate`	Concurrent requests to the tissue-seg endpoint (default 64).
`workers`	`TileExtractor.save`	Threads for writing tile images (default 4).

For end-to-end inference you rarely call these directly — the Inference pipeline handles masking and tiling and streams results to disk. Use the manual API for QC, custom region extraction, or building your own pipeline. To read regions, thumbnails, and slide metadata, see WSI processing.

For guidance on which of these knobs actually affect throughput (and by how much), see Performance tuning.

Overview

Get Started

Preprocessing

Workflows

Troubleshooting

Reference

The quick path: the Inference pipeline

The manual path: masks and tiles as artifacts

1. Generate a tissue mask

Bring your own segmentation (custom mask provider)

Hybrid pipeline: local mask, remote embeddings

2. Tile the slide

Tuning knobs

​The quick path: the Inference pipeline

​The manual path: masks and tiles as artifacts

​1. Generate a tissue mask

​Bring your own segmentation (custom mask provider)

​Hybrid pipeline: local mask, remote embeddings

​2. Tile the slide

​Tuning knobs

The quick path: the Inference pipeline

The manual path: masks and tiles as artifacts

1. Generate a tissue mask

Bring your own segmentation (custom mask provider)

Hybrid pipeline: local mask, remote embeddings

2. Tile the slide

Tuning knobs