# H-Optimus (H1) tile embedding
Source: https://docs.bioptimus.com/api-reference/embedding/h-optimus-h1-tile-embedding
/api-reference/openapi.json post /api/embed/h1
Compute H1 tile embeddings. Input: a 224×224 tile at 0.5 MPP. Output: a 1536-dimensional embedding vector in `output`.
# M-Optimus tile embedding
Source: https://docs.bioptimus.com/api-reference/embedding/m-optimus-tile-embedding
/api-reference/openapi.json post /api/embed/m-optimus
Compute M-Optimus tile embeddings. Input: a 224×224 tile at 0.5 MPP. Output: an embedding vector in `output`.
# Bioptimus Inference Server
Source: https://docs.bioptimus.com/api-reference/introduction
The REST API served by the Bioptimus Model Server, and the SageMaker dispatch endpoint.
The Bioptimus Model Server exposes a JSON REST API. The same `ModelRequest` / `ModelResponse` schema is used everywhere; what differs is how you reach it:
* **On-premise** — path-based endpoints (`/api/embed/h1`, `/api/predict/m-optimus`, …) on the container (default port 8080). Interactive Swagger UI is served at `/docs`, and the live schema at `/openapi.json`.
* **AWS SageMaker** — a single `/invocations` endpoint that accepts a `SageMakerRequest` (a `ModelRequest` plus `model_name` and `mode` for dispatch).
* **SDK** — the [Bioptimus SDK](/guides/get-started/sdk) wraps both and is the recommended interface for whole-slide inference.
The model endpoints (embedding, prediction, metadata) are listed in the **Endpoints** section, generated from the server's OpenAPI schema. The endpoints below are not part of that schema:
| Endpoint | Method | Purpose |
| -------------- | ------ | ---------------------------------------------- |
| `/invocations` | POST | SageMaker dispatch (adds `model_name`, `mode`) |
| `/ping` | GET | Health check |
| `/bioptimus/` | GET | Service discovery |
## Authentication
The Model Server REST API requires no authentication. Deploy it on a private network and restrict access at the network layer. On AWS, the SageMaker endpoint is protected by IAM (requests are SigV4-signed); on Hugging Face, model access is gated.
## Errors
All endpoints return JSON. Common statuses:
| Status | Meaning |
| ------ | ------------------------------------------------------------------------------ |
| `200` | Success |
| `422` | Validation error — malformed or missing fields (FastAPI `HTTPValidationError`) |
| `500` | Inference error |
| `503` | Model still loading, or GPU unavailable (see `/ping` states) |
A `422` body follows the FastAPI shape: `{"detail": [{"loc": [...], "msg": "...", "type": "..."}]}`.
# M-Optimus gene sets
Source: https://docs.bioptimus.com/api-reference/prediction/m-optimus-gene-sets
/api-reference/openapi.json get /api/metadata/m-optimus
Returns the ordered input and output gene sets used by the M-Optimus model.
# M-Optimus spatial gene expression prediction
Source: https://docs.bioptimus.com/api-reference/prediction/m-optimus-spatial-gene-expression-prediction
/api-reference/openapi.json post /api/predict/m-optimus
Predict spatial gene expression from a tile and optional bulk RNA counts. Input: a 224×224 tile at 0.5 MPP, plus an optional `bulk_rna` vector aligned to the model's input gene set (see GET /api/metadata/m-optimus). When `bulk_rna` is omitted, a zero vector is used (H&E-only mode). Output: predicted expression for each output gene in `output`.
# Tissue segmentation
Source: https://docs.bioptimus.com/api-reference/prediction/tissue-segmentation
/api-reference/openapi.json post /api/predict/tissue-seg
Generate a tissue segmentation mask for a tile. Input: a 512×512 tile at 8.0 MPP. Output: a flattened binary mask (0/1) of length H×W in `output`.
# Health check
Source: https://docs.bioptimus.com/api-reference/service/health-check
/api-reference/openapi.json get /ping
Returns 200 with the loaded model list when the server is ready. Returns 503 with a status reason while loading or if the GPU/CUDA context is unavailable.
# SageMaker dispatch endpoint
Source: https://docs.bioptimus.com/api-reference/service/sagemaker-dispatch-endpoint
/api-reference/openapi.json post /invocations
SageMaker-compatible endpoint. Accepts a SageMakerRequest (ModelRequest plus `model_name` and `mode`) to route to the correct model and mode. Used by the SDK's AWS backend.
# Service discovery
Source: https://docs.bioptimus.com/api-reference/service/service-discovery
/api-reference/openapi.json get /bioptimus/
Returns links to the interactive docs and the OpenAPI schema.
# Deployment overview
Source: https://docs.bioptimus.com/deployment/overview
Choose the right way to access Bioptimus models.
Bioptimus models are available three ways. The on-premise and SageMaker options share the same JSON API and are both supported by the [Bioptimus SDK](/guides/get-started/sdk), so your inference code is portable between them.
Managed cloud endpoints from AWS Marketplace. Best for production and scale.
Self-hosted container. Best for data residency and air-gapped environments.
Direct H-Optimus weights for non-commercial academic research.
## Which should I choose?
| Consideration | AWS & SageMaker | On-premise | Hugging Face |
| ---------------------------- | ------------------------------ | ---------------------- | ---------------------- |
| Time to first inference | Minutes | Moderate (infra setup) | Minutes |
| Data leaves your environment | Within your AWS account/region | No | No |
| Commercial use | Yes | Yes | **No** — academic only |
| Models | H-Optimus, M-Optimus | H-Optimus, M-Optimus | H-Optimus only |
| Best for | Cloud-native production | Regulated / air-gapped | Academic research |
For data residency and PHI considerations, see [Security & compliance](/documentation/resources/security-compliance).
# AWS & SageMaker
Source: https://docs.bioptimus.com/deployment/platforms/aws-sagemaker
Deploy a Bioptimus model from AWS Marketplace to a SageMaker endpoint.
Subscribe to a model on [AWS Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-buy.html), deploy it as a [SageMaker endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html), and run inference either with the [Bioptimus SDK](/guides/get-started/sdk) (recommended) or the raw SageMaker runtime API.
## 1. Subscribe and configure
If your Bioptimus contact sent you a [**private offer**](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-private-offers.html), accept it via the link they provided before continuing. After acceptance the model appears under **Manage your subscriptions**, and the remaining steps are identical.
Subscribe to the model. In **Manage your subscriptions**, open your model subscription and click **Configure**.
You will be asked to select the model access interface — **SageMaker AI** (Python SDK) or **CLI** — based on your preference and use case.
Select your **region** and **inference mode** (e.g. a [real-time inference endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) or [batch transform jobs](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html)). This guide spawns a simple real-time inference endpoint.
Copy the [**Model Package ARN**](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-buy.html) shown in the right-hand pane of the configuration page — make sure you copy the one for your chosen region.
The region must be identical across every command and in the Model Package ARN. Mixing regions is the most common cause of deployment failures.
## 2. Deploy an endpoint
Deploy with the Python SDK (recommended) or the AWS CLI.
```python theme={null}
import sagemaker
from sagemaker import ModelPackage
session = sagemaker.Session()
role = ""
model = ModelPackage(role=role,
model_package_arn="",
sagemaker_session=session)
predictor = model.deploy(initial_instance_count=1,
instance_type="ml.g5.xlarge",
endpoint_name="bioptimus-prod",
inference_ami_version="al2-ami-sagemaker-inference-gpu-2")
```
AWS walks you through these steps from the configuration page. The commands below are provided in case you prefer to run them yourself. For full option details, see the [`aws sagemaker` command reference](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/index.html).
First, [install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).
Find your **SSO start URL** and **SSO region** by following the *Access Keys* link on the AWS landing page for your role, then [configure an IAM Identity Center (SSO) profile](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html):
```bash theme={null}
aws configure sso
```
Answer the prompts:
1. **Session name** — anything you like (e.g. your username).
2. **SSO start URL** — the URL you obtained above.
3. **SSO region** — the region you obtained above.
4. **SSO registration scopes** — leave blank; it defaults to `sso:account:access`.
5. Authorize access in the browser page that opens.
6. Select the account to use.
7. Select the **`AWSAdministratorAccess`** role (required for the next steps).
8. **CLI default client Region** — leave blank.
9. **CLI default output format** — leave blank.
10. **CLI profile name** — a name to reuse this profile later.
If subsequent steps fail, activate the profile explicitly:
```bash theme={null}
export AWS_PROFILE=
```
Create (or reuse) a [SageMaker execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html):
```bash theme={null}
role_name="AmazonSageMaker-ExecutionRole-AWSMarketplace"
aws iam create-role \
--role-name "${role_name}" \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "sagemaker.amazonaws.com" },
"Action": "sts:AssumeRole"
}
]
}'
aws iam attach-role-policy \
--role-name "${role_name}" \
--policy-arn "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
aws iam put-role-policy \
--role-name "${role_name}" \
--policy-name "AmazonSageMaker-ExecutionPolicy" \
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::sagemaker"]
},
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
"Resource": ["arn:aws:s3:::sagemaker/*"]
}
]
}'
```
Store the role ARN in a variable for the next steps:
```bash theme={null}
execution_role_arn="arn:aws:iam::$(aws sts get-caller-identity --query 'Account' --output text):role/${role_name}"
```
Use the **Model Package ARN** for your region. [Network isolation](https://docs.aws.amazon.com/sagemaker/latest/dg/mkt-algo-model-internet-free.html) is required for Marketplace model packages. The names below are arbitrary and work for either package (H-Optimus or M-Optimus) — choose your own.
```bash theme={null}
model_name="bioptimus-model"
endpoint_name="bioptimus-endpoint"
aws sagemaker create-model \
--model-name "${model_name}" \
--execution-role-arn "${execution_role_arn}" \
--primary-container ModelPackageName="" \
--enable-network-isolation \
--region
```
Create the endpoint configuration (instance type and count), then the endpoint itself. The generous timeouts give the large model weights time to download and load before SageMaker's startup health check — without them, endpoint creation can fail.
Set `` to a [supported SageMaker inference AMI](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ProductionVariant.html) for your region (e.g. `al2-ami-sagemaker-inference-gpu-2`), or omit the field to let SageMaker pick the default.
```bash theme={null}
aws sagemaker create-endpoint-config \
--endpoint-config-name "${endpoint_name}-config" \
--production-variants '[{
"VariantName": "AllTraffic",
"ModelName": "'"${model_name}"'",
"InitialInstanceCount": 1,
"InstanceType": "ml.g5.xlarge",
"InitialVariantWeight": 1.0,
"InferenceAmiVersion": "al2-ami-sagemaker-inference-gpu-2",
"ModelDataDownloadTimeoutInSeconds": 1800,
"ContainerStartupHealthCheckTimeoutInSeconds": 1800
}]' \
--region
aws sagemaker create-endpoint \
--endpoint-name "${endpoint_name}" \
--endpoint-config-name "${endpoint_name}-config" \
--region
```
Endpoint creation takes about 5–10 minutes. Check the status from the CLI:
```bash theme={null}
aws sagemaker describe-endpoint \
--endpoint-name "${endpoint_name}" \
--region \
--query 'EndpointStatus'
```
Or open **SageMaker AI** in the AWS console and go to *Deployment & Inference > Endpoints*. The endpoint is ready once its status is `InService`.
## 3. Run inference with the Bioptimus SDK (recommended)
The Bioptimus SDK's AWS backend routes through the SageMaker `/invocations` endpoint and adds the `model_name` dispatch field for you.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.H1,
backend="aws",
endpoint_name="bioptimus-prod",
region_name="",
)
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.M_OPTIMUS,
backend="aws",
endpoint_name="bioptimus-prod",
region_name="",
)
```
For whole-slide inference (tiling, tissue masking, bulk RNA, output formats), see the [Bioptimus SDK](/guides/get-started/sdk).
## 4. Or call the runtime API directly
A SageMaker request is a `ModelRequest` plus `model_name` and `mode` fields. See the [API reference](/api-reference/introduction) for the full schema.
## 5. Clean up
Endpoints incur charges while running. Delete the endpoint when finished.
```python theme={null}
predictor.delete_endpoint()
```
Delete the endpoint, its configuration, and the model:
```bash theme={null}
aws sagemaker delete-endpoint --endpoint-name "${endpoint_name}" --region
aws sagemaker delete-endpoint-config --endpoint-config-name "${endpoint_name}-config" --region
aws sagemaker delete-model --model-name "${model_name}" --region
```
If you created the IAM execution role just for this deployment, delete it as well:
```bash theme={null}
role_name="AmazonSageMaker-ExecutionRole-AWSMarketplace"
aws iam delete-role-policy \
--role-name "${role_name}" \
--policy-name "AmazonSageMaker-ExecutionPolicy"
aws iam detach-role-policy \
--role-name "${role_name}" \
--policy-arn "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
aws iam delete-role --role-name "${role_name}"
```
## Reference notebooks
End-to-end examples: [`h1-jumpstart`](https://github.com/bioptimus/h1-jumpstart) (H-Optimus) and [`m-jumpstart`](https://github.com/bioptimus/m-jumpstart) (M-Optimus).
# Hugging Face
Source: https://docs.bioptimus.com/deployment/platforms/hugging-face
Load H-Optimus-1 weights directly for non-commercial academic research.
For academic users, H-Optimus-1 weights are available directly on Hugging Face and can be loaded with `timm` — no server or AWS account required.
**Non-commercial academic use only.** H-Optimus-1 on Hugging Face is released under **CC-BY-NC-ND 4.0**. Commercial use, sale, or monetization (including models trained on its outputs) is prohibited without prior approval. For commercial use, deploy via [AWS](/deployment/platforms/aws-sagemaker) or [on-premise](/deployment/platforms/on-premise), or contact Bioptimus.
## 1. Request access
Access is gated. On [huggingface.co/bioptimus/H-optimus-1](https://huggingface.co/bioptimus/H-optimus-1), accept the terms using your **institutional email** (it must match your Hugging Face account email to be approved).
## 2. Load the model and extract features
H-Optimus-1 expects 224×224 images extracted at 0.5 microns per pixel.
```python theme={null}
from huggingface_hub import login
import torch
import timm
from torchvision import transforms
login() # token from https://huggingface.co/settings/tokens
model = timm.create_model(
"hf-hub:bioptimus/H-optimus-1",
pretrained=True, init_values=1e-5, dynamic_img_size=False,
)
model.to("cuda").eval()
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(
mean=(0.707223, 0.578729, 0.703617),
std=(0.211883, 0.230117, 0.177517),
),
])
tile = transforms.ToPILImage()(torch.rand(3, 224, 224))
with torch.autocast(device_type="cuda", dtype=torch.float16):
with torch.inference_mode():
features = model(transform(tile).unsqueeze(0).to("cuda"))
assert features.shape == (1, 1536)
```
Mixed precision (`autocast`) is recommended for faster inference. The normalization constants above are specific to H-Optimus-1 — use them as shown.
## When to use a different option
M-Optimus and commercial licensing are available via AWS Marketplace.
The Bioptimus SDK handles tiling and tissue masking against a deployed server.
# On-premise
Source: https://docs.bioptimus.com/deployment/platforms/on-premise
Load, run, and verify the Bioptimus Model Server container in your own environment.
The Bioptimus Model Server runs H-Optimus and M-Optimus entirely within your infrastructure — keeping data on your hardware for residency, compliance, or air-gapped requirements. It serves the same JSON API as the [SageMaker deployment](/deployment/platforms/aws-sagemaker), and the [Bioptimus SDK](/guides/get-started/sdk) works against both.
Two package variants are available. Each image contains the relevant model **plus** the tissue segmentation model:
| Package | Models | Key endpoints |
| ------- | ----------------------------------- | ------------------------------------------------------------------------------------------ |
| **H1** | H-Optimus (`h1`), tissue-seg | `POST /api/embed/h1`, `POST /api/predict/tissue-seg` |
| **M** | M-Optimus (`m-optimus`), tissue-seg | `POST /api/embed/m-optimus`, `POST /api/predict/m-optimus`, `POST /api/predict/tissue-seg` |
## Prerequisites
| Requirement | Detail |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Docker | Version 20.10 or later |
| NVIDIA Container Toolkit | Required for GPU inference ([install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)) |
| GPU | NVIDIA GPU with **CUDA Compute Capability 8.6** (e.g. A10G-class, equivalent to SageMaker `ml.g5.xlarge`). A CUDA GPU is **required** — the container exits if none is detected. |
| CPU architecture | **x86-64** — CC 8.6 + x86-64 runs out of the box; other architectures require recompilation. |
| GPU access | Direct passthrough (`--gpus all`) |
| OS | Linux with an NVIDIA GPU |
| Disk | Enough to hold the image (weights are baked in) |
The REST API currently requires **no authentication** — run the server on a private network and restrict access at the network layer. See the [API reference](/api-reference/introduction#authentication).
## 1. Load the container image
You receive the container as a compressed archive plus a `sha256` checksum. Verify integrity, then load it into Docker.
```bash theme={null}
sha256sum -c bioptimus-h1-onpremise-v1.0.1.sha256
docker load < bioptimus-h1-onpremise-v1.0.1.tar.gz
docker images bioptimus-h1-onpremise
```
```bash theme={null}
sha256sum -c bioptimus-m-onpremise-v1.0.1.sha256
docker load < bioptimus-m-onpremise-v1.0.1.tar.gz
docker images bioptimus-m-onpremise
```
## 2. Start the container
The server is self-contained — all weights and assets are baked into the image, so no volume mounts are required.
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
bioptimus-h1-onpremise:v1.0.1 serve
```
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
bioptimus-m-onpremise:v1.0.1 serve
```
Models load at startup. Follow the logs until the server is ready:
```bash theme={null}
docker logs -f bioptimus-server 2>&1 | grep -m1 "models ready"
```
## 3. Air-gapped install
For environments with no outbound network, transfer the `.tar.gz` archive via your approved process, then run `sha256sum -c`, `docker load`, and `docker run` exactly as above. No registry access is needed — the image is self-contained.
## 4. Verify the deployment
```bash H-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
```bash M-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["m-optimus", "tissue-seg"]}
```
A 503 `{"status": "loading"}` means models are still initialising — retry. See the [API reference](/api-reference/introduction) for all health states.
Open `http://localhost:8080/docs` for a Swagger UI to explore endpoints and try calls. Service discovery is at `http://localhost:8080/bioptimus/`.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
model = Backbone(Models.H1, base_url="http://localhost:8080")
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
model = Backbone(Models.M_OPTIMUS, base_url="http://localhost:8080")
```
For whole-slide inference, see the [Bioptimus SDK](/guides/get-started/sdk).
## Managing the container
```bash theme={null}
docker stop bioptimus-server # stop (preserves state)
docker start bioptimus-server # restart
docker logs bioptimus-server # view logs
docker rm -f bioptimus-server # remove
```
## Environment variables
Set with `-e` at start:
| Variable | Default | Description |
| -------------- | --------------- | ----------------------------------------------- |
| `PORT` | `8080` | Server port |
| `LOG_LEVEL` | `INFO` | Log level (`DEBUG`, `INFO`, `WARNING`, `ERROR`) |
| `SM_MODEL_DIR` | `/opt/ml/model` | Model artefact directory (used by SageMaker) |
## Troubleshooting
Container not running or still starting. Check `docker ps` and `docker logs bioptimus-server`.
Models are still loading. Wait for the `models ready` log line.
The GPU is no longer available, or the CUDA context was corrupted. Check `nvidia-smi` on the host and restart the container.
Common causes: no CUDA GPU detected (a GPU is required), port conflict, or insufficient memory. Run `docker logs bioptimus-server` for the error.
The server auto-batches concurrent requests (max batch size 32). Reduce the number of concurrent SDK requests to lower peak memory use.
# Requirements
Source: https://docs.bioptimus.com/deployment/requirements
Prerequisites for each deployment option.
Prerequisites depend on how you deploy:
An IAM role with `AmazonSageMakerFullAccess`, plus permission to subscribe to AWS Marketplace products.
An active subscription to the relevant model package ([H-Optimus](https://aws.amazon.com/marketplace/pp/prodview-cuad7l27fobx4) / M-Optimus) to obtain the Model Package ARN.
A SageMaker execution environment and sufficient quota for `ml.g5.xlarge` instances in your region. Outside SageMaker, set the execution role ARN explicitly (`get_execution_role()` is unavailable).
| Requirement | Detail |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Docker | Version 20.10 or later |
| NVIDIA Container Toolkit | Required for GPU inference ([install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)) |
| GPU | NVIDIA GPU with **CUDA Compute Capability 8.6** (e.g. A10G-class). A 24 GB GPU — equivalent to SageMaker `ml.g5.xlarge` — is a good baseline. A CUDA GPU is **required** (container exits without one). |
| CPU architecture | **x86-64** (CC 8.6 + x86-64 runs out of the box; other architectures require recompilation) |
| GPU access | Direct passthrough via the NVIDIA Container Toolkit (`--gpus all`) |
| OS | Linux with an NVIDIA GPU |
| Disk | Enough to hold the image archive (weights are baked in) |
See the [on-premise guide](/deployment/platforms/on-premise) for the full install flow.
| Requirement | Detail |
| ----------- | --------------------------------------------------------------------------- |
| License | CC-BY-NC-ND 4.0 — **non-commercial academic research only** |
| Access | Gated; accept the terms on Hugging Face using an institutional email |
| Python | `timm`, `torch`, `torchvision`, `huggingface_hub`; a CUDA GPU for inference |
See the [Hugging Face guide](/deployment/platforms/hugging-face).
## Python SDK & dependencies
The [Bioptimus SDK](/guides/get-started/sdk) handles WSI reading, tiling, tissue masking, and concurrent dispatch for both the on-premise and SageMaker backends. The SageMaker reference notebooks pin:
```bash theme={null}
pip install sagemaker==2.254.1 boto3==1.42.2
```
# Validation & acceptance testing
Source: https://docs.bioptimus.com/deployment/validation
Verify a deployment produces correct, expected outputs.
Use these checks after deploying ([on-premise](/deployment/platforms/on-premise) or [SageMaker](/deployment/platforms/aws-sagemaker)) to confirm the service is healthy and producing expected results.
## 1. Service health
```bash H-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
```bash M-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["m-optimus", "tissue-seg"]}
```
## 2. Model availability (SDK)
`available_backbones()` lists the models the Bioptimus SDK can construct (read from its bundled configs) — it does not query the server. The `/ping` response in step 1 is the source of truth for what the deployed endpoint actually serves.
```python theme={null}
from bioptimus.models.backbones import Backbone
print(Backbone.available_backbones())
# ['h1', 'm-optimus', 'tissue-seg']
```
## 3. Output shape checks
| Model | Expected output length |
| --------------------------- | ------------------------------------------------------ |
| H-Optimus (`/api/embed/h1`) | 1536 |
| M-Optimus embedding | 1536 |
| M-Optimus prediction | number of output genes (see `/api/metadata/m-optimus`) |
| Tissue segmentation | H×W (262,144 for a 512×512 tile) |
# Welcome
Source: https://docs.bioptimus.com/documentation/introduction
Deploy and build with foundation models for biology — across histology, transcriptomics, and genomics.
Bioptimus builds foundation models that learn the dynamics of human biology across scales — from cell to tissue to organ. This documentation covers how to access, deploy, and build with our models on AWS and on-premise.
## How it works
Bioptimus models are pretrained on large, diverse histology data, so their representations transfer to many tasks without training from scratch. Point the SDK at a slide and a model, and it tiles the slide, masks out background, runs the model, and writes results to disk. For precise definitions, see the [Glossary](/documentation/resources/glossary).
A scanned H\&E slide can be **billions of pixels** — too large to process at once — so the pipeline starts from the slide and splits it into small tiles, each processed independently.
![H&E whole-slide image thumbnail]()
Slides are mostly background. The pipeline tiles coarsely (512×512 at 8 µm/px) and runs [tissue segmentation](/documentation/models/tissue-segmentation) to produce a tissue map, keeping only tissue-bearing tiles — cutting cost before the expensive feature step.
![Slide thumbnail alongside its binary tissue mask]()
Tissue tiles (224×224 at 0.5 µm/px) are embedded by [H-Optimus](/documentation/models/h-optimus) into a **1536-d feature vector** each. Principal components of these embeddings reveal the dominant axes of morphology — tumor, stroma, and immune compartments.
![Spatial heatmaps of the first three principal components of tile embeddings]()
[M-Optimus](/documentation/models/m-optimus) produces the same 1536-d embeddings **and** predicts spatial gene expression directly from each tile (optionally informed by bulk RNA) — a molecular readout without a spatial assay.
![Spatial gene-expression panel predicted from H&E by M-Optimus]()
## The models
A vision foundation model for histology. Extracts tile-level features from H\&E whole slide images.
A multimodal, multi-scale model (M-Optimus-1) that predicts spatial gene expression from routine H\&E, refined with bulk RNA.
Both models are trained on data from STELA, our data engine:
A multi-institutional data engine generating the deeply profiled, clinically linked patient data our models train on.
## Where to get each model
A quick map of which model is available on which channel, and where to start.
| Model | AWS & SageMaker | On-premise | Hugging Face |
| ----------------------- | ----------------------------------------------- | -------------------------------------------- | ------------------------------------------------ |
| **H-Optimus** | ✅ [Deploy](/deployment/platforms/aws-sagemaker) | ✅ [Deploy](/deployment/platforms/on-premise) | ✅ [Academic](/deployment/platforms/hugging-face) |
| **M-Optimus** | ✅ [Deploy](/deployment/platforms/aws-sagemaker) | ✅ [Deploy](/deployment/platforms/on-premise) | — |
| **Tissue segmentation** | ✅ (bundled) | ✅ (bundled) | — |
Managed endpoints from AWS Marketplace.
Self-hosted container for full data control.
H-Optimus weights for non-commercial academic use.
## Where to start
Deploy a model and get embeddings back in minutes. For ML engineers and data scientists.
See how teams use our models for biomarker discovery, indication expansion, and trial design.
Data handling, residency, and deployment options for regulated environments.
Bioptimus models are for **research use** and are not approved medical devices. See [Responsible use](/documentation/resources/responsible-use) for intended use and your responsibilities.
# Benchmarks
Source: https://docs.bioptimus.com/documentation/models/benchmarks
Independent benchmark results for Bioptimus pathology models.
Bioptimus models are evaluated on independent, peer-reviewed benchmarks. The two most relevant for pathology foundation models are **PathBench** (HKUST) and **HEST** (Harvard; Jaume et al. 2025).
## PathBench
PathBench is a multi-task, multi-organ benchmark for pathology foundation models. Bioptimus reports H-Optimus-1 as the top-ranked model on the overall rank score (lower is better).
| Model | Overall rank score |
| --------------- | ------------------ |
| **H-Optimus-1** | **6.06** |
| Virchow2 | 6.34 |
| H-Optimus-0 | 6.86 |
| UNI2 | 7.10 |
| mSTAR | 7.65 |
**Rankings are task- and organ-dependent.** H-Optimus-1 ranks first overall and leads on several organs (e.g. lung and colorectal); other models lead on specific tasks. Always link to the source so readers can see the full picture.
Sources: [PathBench paper (arXiv:2505.20202)](https://arxiv.org/abs/2505.20202) · [bioptimus.com](https://www.bioptimus.com)
## HEST
HEST (Harvard; Jaume et al. 2025) measures how well a model predicts **gene expression from histology** across nine organs. The metric is Pearson's correlation coefficient (higher is better). As of May 2026, H-Optimus-1 is the top-ranked image-only model on HEST, and M-Optimus-1 surpasses it by adding multimodal training.
| Model | HEST (avg. Pearson r) |
| --------------- | --------------------- |
| **M-Optimus-1** | **0.440** |
| **H-Optimus-1** | **0.423** |
Figures are reported in the M-Optimus-1 report (M-Optimus-1 vs. H-Optimus-1 on HEST). The full per-model HEST leaderboard is maintained by the HEST authors — see the source for the complete table and methodology.
Bioptimus's write-up of the HEST evaluation and methodology.
## At scale
* 100+ papers published using Bioptimus models
* Used by 16 of the top 20 pharma companies
* 1,000+ clinical practices · 1M+ total model downloads
Peer-reviewed work and case studies in the Bioptimus Knowledge Base.
# H-Optimus
Source: https://docs.bioptimus.com/documentation/models/h-optimus
A vision foundation model for histology feature extraction.
H-Optimus is a vision transformer foundation model for histology. It extracts tile-level **feature vectors (embeddings)** from H\&E whole slide images — the input to downstream tasks such as mutation prediction, survival analysis, and tissue classification. H-Optimus produces features; it does not predict molecular signal directly (for that, see [M-Optimus](/documentation/models/m-optimus)).
**H-Optimus-1** is current — a 1.1B-parameter vision transformer (ViT-g/14) trained with self-supervised learning on billions of tiles from over 1 million slides of more than 800,000 patients, spanning 50+ organs, 3 scanner types, and 4,000+ clinical centers. As of May 2026 it ranks **#1** on the public PathBench leaderboard and is the top image-only model on HEST. **H-Optimus-0** (2024) is the previous generation, open-source under Apache 2.0. See [Benchmarks](/documentation/models/benchmarks) and the [Hugging Face model details](/deployment/platforms/hugging-face).
![Spatial heatmaps of the first three principal components of tile embeddings]()
## Specifications
| Property | Value |
| -------------------- | --------------------------------------- |
| Task | Tile embedding |
| Input | 224×224 RGB tile at 0.5 µm/px |
| Output | 1536-dimensional feature vector |
| On-prem endpoint | `POST /api/embed/h1` |
| SageMaker dispatch | `model_name: "h1"`, `mode: "embedding"` |
| Recommended instance | `ml.g5.xlarge` |
## How it's used
Run [tissue segmentation](/documentation/models/tissue-segmentation) to keep tissue-bearing tiles.
Embed 224×224 tiles to get a 1536-d vector each (the [SDK](/guides/get-started/sdk) tiles and dispatches for you).
Aggregate embeddings for slide-level prediction, retrieval, or classification.
## Deploy
H-Optimus runs on AWS SageMaker, on-premise, or Hugging Face (academic). See the [Deployment overview](/deployment/overview) for the full comparison and setup steps.
## Guides
Extract embeddings and visualize morphology.
Run whole-slide inference with the Bioptimus SDK.
# M-Optimus
Source: https://docs.bioptimus.com/documentation/models/m-optimus
A multimodal, multi-scale foundation model that predicts spatial gene expression from histology.
M-Optimus is a **multimodal, multi-scale** foundation model. The current generation, **M-Optimus-1 (M1)**, learns across three biological layers at once — H\&E pathology, bulk RNA-seq, and spatial transcriptomics — to build a unified representation of a patient across tissue and molecular scales.
Its headline capability is **predicting spatial gene expression directly from a routine H\&E tile**, across up to 6,002 genes, optionally refined with bulk RNA-seq — recovering an expensive molecular readout from a low-cost slide. M-Optimus also produces the same 1536-d tile embeddings as [H-Optimus](/documentation/models/h-optimus). It is trained on proprietary multimodal cohorts, powered by the [STELA](https://www.bioptimus.com/stela) data engine.
## What makes it different
Where H-Optimus stops at features, M-Optimus adds a **prediction** head over a defined set of output genes. Provide a tile and (optionally) a bulk RNA vector, and it returns predicted expression per output gene — which you can render as spatial heatmaps. Because it is multimodal at both training and inference, it can ingest H\&E alone or H\&E + bulk RNA, with no retraining needed to benefit from the extra modality.
![EPCAM overlay predicted from H&E: with bulk RNA vs image only]()
## Performance
Results below are from the [M-Optimus-1 report](https://www.bioptimus.com); rankings and metrics are task- and dataset-dependent.
| Result | Finding |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| Spatial gene expression from H\&E | **+60%** vs. DeepSpot, a leading H\&E→ST model (\~30% from proprietary pretraining data, \~30% from the multimodal method) |
| Adding bulk RNA at inference | **+4%** over H\&E-only, with no retraining (late-binding) |
| HEST (gene expression from histology) | **0.440** avg. Pearson vs. **0.423** for H-Optimus-1 |
| Classification (mean AUC, 9 tasks) | **0.664** vs. **0.661** for H-Optimus-1 — matches the image-only SOTA while adding molecular prediction |
| Generalist vs. specialists | **+8%** on colon and head & neck vs. indication-specific models; strong zero-shot generalization to unseen tissues (e.g. kidney, skin) |
## Modes
* **Endpoint:** `POST /api/predict/m-optimus`
* **Input:** 224×224 tile at 0.5 µm/px, plus optional `bulk_rna` aligned to the model's input gene set. Omitting bulk RNA uses a zero vector (H\&E-only mode).
* **Output:** predicted expression for each output gene.
* Gene sets are available at `GET /api/metadata/m-optimus`.
* **Endpoint:** `POST /api/embed/m-optimus`
* **Input:** 224×224 tile at 0.5 µm/px.
* **Output:** 1536-d tile feature vector.
The [SDK](/guides/get-started/sdk) aligns and reorders your bulk RNA to the model's input gene set automatically. See [Add bulk RNA](/guides/workflows/spatial-transcriptomics#3-add-bulk-rna-multimodal-prediction) for the full input contract and transform pipeline.
## Specifications
| Property | Value |
| -------------------- | --------------------------------------------------------------------------- |
| Tasks | Spatial gene-expression prediction (spot-level); tile embedding |
| Genes | Up to 6,002 (model's defined output set; see `GET /api/metadata/m-optimus`) |
| Input | 224×224 RGB tile at 0.5 µm/px (+ optional bulk RNA) |
| Bulk RNA input | TPM-normalized expression values, Ensembl gene IDs |
| Bulk RNA transforms | Server-side: `log1p` on input, `expm1` on output |
| Embedding | 1536-d |
| SageMaker dispatch | `model_name: "m-optimus"` |
| Recommended instance | `ml.g5.xlarge` |
## What you can build
M-Optimus turns routine slides into a molecular map for translational research:
Surface candidate biomarkers and molecular signatures from H\&E at scale.
Enrich and stratify cohorts; analyze legacy trial slides retrospectively.
## Access & deploy
M-Optimus is available now by request. [Contact Bioptimus](https://www.bioptimus.com/contact) to discuss access, then deploy on AWS SageMaker or on-premise — see the [Deployment overview](/deployment/overview) for setup steps.
M-Optimus is not available on Hugging Face. For academic-only H\&E feature extraction, use [H-Optimus on Hugging Face](/deployment/platforms/hugging-face).
## Guides
Predict spatial gene expression end to end.
Run a model over many slides.
# Tissue segmentation
Source: https://docs.bioptimus.com/documentation/models/tissue-segmentation
Separate tissue from background before feature extraction.
The tissue segmentation model identifies tissue regions in a slide, letting you discard background tiles before running [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus). It is bundled with both model packages.
## Specifications
| Property | Value |
| -------------------- | -------------------------------------------------------------------- |
| Task | Binary tissue mask |
| Input | 512×512 RGB tile at 8.0 µm/px |
| Output | Flattened binary mask of length 262,144 (512×512), values 0.0 or 1.0 |
| SageMaker dispatch | `model_name: "tissue-seg"`, `mode: "prediction"` |
| Recommended instance | `ml.g5.xlarge` |
Run segmentation at the coarse 8.0 µm/px resolution first; it is cheap relative to per-tile feature extraction at 0.5 µm/px and can dramatically reduce the number of tiles you embed.
## Reshaping the output
The endpoint returns a flat array. Reshape it to a 512×512 mask:
```python theme={null}
import numpy as np
mask = np.array(result["output"]).reshape(512, 512)
```
## Deploy
Tissue segmentation has no separate deployment — it is **bundled with both the H-Optimus and M-Optimus packages** and served from the same endpoint. Deploy either model (see the [Deployment overview](/deployment/overview)) and `tissue-seg` is available alongside it.
## Using it in a pipeline
In practice you rarely call this endpoint directly — the Bioptimus SDK's tissue mask provider generates and caches masks for you, then filters tiles before feature extraction or prediction. See [Spatial transcriptomics](/guides/workflows/spatial-transcriptomics) and the [Bioptimus SDK](/guides/get-started/sdk).
Once masks are generated, `bioptimus.utils.plot_slide_and_mask` renders the mask next to the slide thumbnail for a quick sanity check:
```python theme={null}
from bioptimus import utils
record = cohort[0]
utils.plot_slide_and_mask(record.wsi_path, record.mask_path, threshold=0.5)
```
![Slide thumbnail alongside its binary tissue mask]()
# Quickstart
Source: https://docs.bioptimus.com/documentation/quickstart
Pick your model and platform, then run your first inference.
Three steps: deploy a model, connect the Bioptimus SDK, and run your first inference.
Not sure which model? **M-Optimus-1** predicts spatial gene expression and also produces embeddings; **H-Optimus-1** is the lighter, feature-only model. See [M-Optimus](/documentation/models/m-optimus) and [H-Optimus](/documentation/models/h-optimus).
## 1. Deploy your model
Deploy on the platform that fits your use case — each guide has the full steps:
| Platform | Best for | Guide |
| ------------------- | --------------------------------------- | ---------------------------------------------------- |
| **AWS & SageMaker** | Production pipelines and scale | [Deploy →](/deployment/platforms/aws-sagemaker) |
| **On-premise** | Data residency, air-gapped sites | [Deploy →](/deployment/platforms/on-premise) |
| **Hugging Face** | Non-commercial academic, H-Optimus only | [Load weights →](/deployment/platforms/hugging-face) |
We recommend a real-time endpoint on `ml.g5.xlarge` (or the on-premise container) — the models are compiled for that GPU architecture (NVIDIA A10G, CUDA Compute Capability 8.6).
## 2. Connect the Bioptimus SDK
Point the [Bioptimus SDK](/guides/get-started/sdk) at your deployment (use `Models.H1` or `Models.M_OPTIMUS`):
```python On-premise theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url="http://localhost:8080")
```
```python AWS SageMaker theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
**Academic (Hugging Face) users** load H-Optimus-1 directly with `timm` instead of the Bioptimus SDK — see [Hugging Face](/deployment/platforms/hugging-face).
## 3. Get your first embedding
Send a single 224×224 tile through the connected model to confirm the full path works:
```python H-Optimus theme={null}
import numpy as np
from PIL import Image
from bioptimus.inference.schemas import ModelRequest
# Replace the blank tile with a real 224×224 H&E tile at 0.5 µm/px.
tile = Image.fromarray(np.zeros((224, 224, 3), dtype=np.uint8))
request = ModelRequest(image_data=tile, slide_name="demo",
x=0, y=0, width=224, height=224, patch_idx=0)
response = model.embed(request)
print(len(response.output)) # 1536
```
```python M-Optimus theme={null}
import numpy as np
from PIL import Image
from bioptimus.inference.schemas import ModelRequest
# Replace the blank tile with a real 224×224 H&E tile at 0.5 µm/px.
tile = Image.fromarray(np.zeros((224, 224, 3), dtype=np.uint8))
request = ModelRequest(image_data=tile, slide_name="demo",
x=0, y=0, width=224, height=224, patch_idx=0)
response = model.predict(request) # spatial gene expression for the tile
print(len(response.output)) # one value per output gene
```
For whole-slide inference — tiling, tissue masking, and writing results — see the [Bioptimus SDK](/guides/get-started/sdk).
## Next steps
Whole-slide inference: tiling, tissue masking, bulk RNA, and output formats.
End-to-end examples: [`h1-jumpstart`](https://github.com/bioptimus/h1-jumpstart) (H-Optimus) and [`m-jumpstart`](https://github.com/bioptimus/m-jumpstart) (M-Optimus).
# FAQ
Source: https://docs.bioptimus.com/documentation/resources/faq
Common questions about deploying and using Bioptimus models.
## Getting started
**M-Optimus** predicts spatial gene expression and also produces embeddings; **H-Optimus** is the lighter, feature-only model; run **tissue segmentation** first to filter background. See [M-Optimus](/documentation/models/m-optimus) and [H-Optimus](/documentation/models/h-optimus).
224×224 tiles at 0.5 µm/px for embeddings and M-Optimus prediction; 512×512 at 8.0 µm/px for tissue segmentation.
## Deployment & infrastructure
Two ways: an **AWS SageMaker endpoint**, or a **containerized Docker image for on-premise**. Both serve the same API. See [Deployment](/deployment/overview).
The on-premise server ships as a Docker image loaded via `docker load` and run with the NVIDIA Container Toolkit.
*TBD — the documented deployment is a single Docker container; confirm orchestration support.*
**x86-64** with an NVIDIA GPU of **CUDA Compute Capability 8.6** (e.g. A10G-class) runs the container out of the box. Other architectures require recompilation. A 24 GB GPU (equivalent to SageMaker `ml.g5.xlarge`) is a good baseline.
GPU access is direct passthrough via the NVIDIA Container Toolkit (`--gpus all`).
## Data security & privacy
Data stays entirely in your own environment (your AWS account, or your on-prem hardware). **Bioptimus does not see your inputs, outputs, or usage.** See [Security & compliance](/documentation/resources/security-compliance).
Indicative (to be confirmed): disease area / task, slide volumes, functions called, and crash/bug signals — no slide images or patient data.
## Model & SDK usage
No. H-Optimus returns the **CLS token** of size **1536**; M-Optimus returns its MLP output. The embedding type is fixed.
A 1536-dimensional float vector per tile (H-Optimus and M-Optimus embedding mode). See the [API reference](/api-reference/introduction).
Customer-managed **SageMaker** endpoints and **on-premise** containers; H-Optimus is also on **Hugging Face** for academic use.
Install the provided Python wheel (or via a package index once confirmed). See the [Bioptimus SDK](/guides/get-started/sdk).
## Commercial & support
Three tiers: **Tier 1 — Co-Designed**, **Tier 2 — Supported**, and **Tier 3 — Self-Service**.
*TBD — contact Bioptimus to discuss evaluation options.*
Delete SageMaker endpoints and batch models when finished — endpoints bill while running.
# Glossary
Source: https://docs.bioptimus.com/documentation/resources/glossary
Key terms used across Bioptimus documentation.
A quick reference for technical and non-technical readers.
## Products & interfaces
| Term | Meaning |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Bioptimus SDK** | The Python client package. Handles WSI reading, tiling, tissue masking, bulk RNA alignment, and concurrent dispatch, against either the on-premise server or a SageMaker endpoint. See the [Bioptimus SDK](/guides/get-started/sdk). |
| **`Inference` (pipeline)** | The high-level, one-object pipeline (config + `tissue()`/`embed()`/`predict()`/`run()`/`save_config()`), with cached masks, a structured workspace, and reproducible config. Its `api_url` argument is the on-premise server URL — the same value as `Backbone`'s `base_url`. See the [Inference pipeline guide](/guides/get-started/inference-pipeline). |
| **`SlideInference`** | The lower-level, per-slide inference class (`Backbone` + mask provider + writer) for explicit control. |
| **`Cohort`** | A batch manifest of slides (and optional bulk RNA) that is the single source of truth for an experiment. Supports late-binding bulk RNA. |
| **`AWSClient`** | The low-level per-tile client for a SageMaker endpoint; injects `model_name`/`mode` into each request. |
| **`OutputFormat`** | The output writer format — Zarr (default), HDF5, or NPZ. |
| **Model Server / API** | The FastAPI inference server (shipped in the on-premise container) that exposes the REST endpoints (`/api/embed/h1`, `/api/predict/m-optimus`, …) and a Swagger UI at `/docs`. See the [API reference](/api-reference/introduction). |
| **Backbone** | The Bioptimus SDK factory class used to obtain a model client, e.g. `Backbone(Models.H1, backend="remote", base_url=...)` or `backend="aws"`. Takes a `Models` enum member (`Models.H1`, `Models.M_OPTIMUS`); the companion tissue model uses the string `"tissue-seg"`. |
| **Inference endpoint** | A running model service that accepts tile requests and returns outputs — an on-premise container or a SageMaker endpoint. |
| **SageMaker** | AWS's managed ML hosting service used for Bioptimus cloud deployment; the Bioptimus SDK reaches it via the `/invocations` dispatch endpoint. |
| **Model package** | The on-premise container variant — **H1** (H-Optimus + tissue-seg) or **M** (M-Optimus + tissue-seg). |
## Model & backend identifiers
The same model is referred to by a product name, a version, an SDK enum, and a server/endpoint id depending on context. This table is the canonical mapping.
| Model | Current version | SDK identifier | Server id | Endpoints | Hugging Face |
| ----------------------- | --------------- | ---------------------------------- | ------------ | ------------------------------------------------ | ----------------------- |
| **H-Optimus** | H-Optimus-1 | `Models.H1` (`"h1"`) | `h1` | `/api/embed/h1` | `bioptimus/H-optimus-1` |
| **M-Optimus** | M-Optimus-1 | `Models.M_OPTIMUS` (`"m-optimus"`) | `m-optimus` | `/api/embed/m-optimus`, `/api/predict/m-optimus` | — |
| **Tissue segmentation** | — | `"tissue-seg"` (string) | `tissue-seg` | `/api/predict/tissue-seg` | — |
The Bioptimus SDK reaches a deployment through a `backend` argument. The product term and the code value differ:
| Platform (product term) | SDK `backend` | Key connection args |
| --------------------------------- | ------------- | -------------------------------------------------- |
| On-premise container | `"remote"` | `base_url` (`api_url` on the `Inference` pipeline) |
| AWS SageMaker | `"aws"` | `endpoint_name`, `region_name` |
| In-process (local GPU, no server) | `"local"` | `model_dir` / `checkpoint` |
## Models & outputs
| Term | Meaning |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Foundation model (FM)** | A large model pre-trained on vast data that produces general-purpose representations reused across many downstream tasks. |
| **H-Optimus** | Bioptimus's histology foundation model; outputs tile embeddings. Current version: H-Optimus-1. |
| **M-Optimus** | Bioptimus's multimodal model; predicts spatial gene expression from histology (and optional bulk RNA), and also outputs embeddings. |
| **Embedding** | A numeric feature vector summarizing a tile. H-Optimus returns the **CLS token** of size **1536**. |
| **1536-d** | The dimensionality of H-Optimus and M-Optimus tile embeddings — a 1536-number feature vector per tile. |
| **CLS token** | The transformer's summary output used as the tile embedding. H-Optimus returns the 1536-d CLS token; M-Optimus returns its MLP output. The embedding type is fixed (not user-selectable). |
| **Spatial gene expression** | Gene expression mapped to locations across a slide. M-Optimus predicts this from H\&E tiles. |
| **Tissue segmentation (`tissue-seg`)** | A companion model (bundled in both packages) that produces a binary tissue/background mask. |
## Imaging & data
| Term | Meaning |
| --------------------------- | ---------------------------------------------------------------------------------------------------- |
| **WSI (whole slide image)** | A digitized microscope slide — often gigapixel-scale — split into tiles for processing. |
| **H\&E** | Hematoxylin and eosin, the standard tissue stain in routine pathology. |
| **Tile (patch)** | A fixed-size crop of a WSI. Embeddings use 224×224; tissue segmentation uses 512×512. |
| **MPP (microns per pixel)** | Physical image resolution. Embeddings use 0.5 µm/px; tissue segmentation uses 8.0 µm/px. |
| **Bulk RNA-seq** | Aggregate (non-spatial) gene expression for a sample. Optional input to M-Optimus prediction. |
| **Ensembl gene ID** | Standardized gene identifier (e.g. `ENSG00000000003`) used in bulk RNA inputs and gene-set metadata. |
| **Zarr / HDF5 / NPZ** | Output file formats the Bioptimus SDK writes per-slide results to. |
## Infrastructure
| Term | Meaning |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **CUDA Compute Capability** | An NVIDIA GPU architecture version. Compute Capability **8.6** on **x86-64** runs the container out of the box; other architectures require recompilation. |
| **NVIDIA Container Toolkit** | Enables GPU access inside Docker containers (`--gpus all`). |
| **`ml.g5.xlarge`** | The recommended SageMaker instance (a single 24 GB A10G-class GPU). |
## Benchmarks
| Term | Meaning |
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
| **PathBench** | A multi-task, multi-organ pathology FM benchmark (HKUST). H-Optimus-1 ranks first overall. |
| **HEST** | A benchmark for predicting gene expression from histology (Harvard; Jaume et al. 2025), scored by Pearson correlation. |
| **MIL (multiple instance learning)** | A method that aggregates many tile-level features into a slide-level prediction. |
# Responsible use
Source: https://docs.bioptimus.com/documentation/resources/responsible-use
Intended use, regulatory status, license terms, responsibilities, and disclaimer.
## Intended use
Bioptimus models are intended solely for non-clinical research use, including research workflows in biomarker discovery, spatial biology, and trial design.
They are not intended for, and must not be used for:
* clinical diagnosis;
* medical decision-making or patient management;
* patient treatment;
* development or use as a medical device, companion diagnostic, or other regulated product;
* any clinical, diagnostic, therapeutic, or other regulated use.
## Regulatory status
Bioptimus models have not been reviewed, approved, cleared, certified, or authorized by any regulatory authority for clinical, diagnostic, therapeutic, or other regulated use.
Bioptimus makes no representation or warranty that the models comply with requirements applicable to regulated products or uses in any jurisdiction.
## Your responsibilities
By accessing or using Bioptimus models, you are responsible for:
* complying with all applicable laws, regulations, and industry standards, including those relating to privacy, data protection, patient data, and healthcare or life sciences regulation;
* ensuring that your use of the models is permitted under your applicable license or commercial agreement with Bioptimus;
* independently evaluating and validating model outputs for your specific use case before relying on them.
## License terms
The terms governing your use of Bioptimus models depend on how you access them:
* **Via Hugging Face** — H-Optimus-1 is distributed under [CC-BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial academic use only. Commercial use under this license is not permitted.
* **Via Bioptimus API or platform** — use is governed by your commercial license agreement with Bioptimus.
## Disclaimer
Bioptimus models are provided "as is" and "as available," without warranties of any kind, express or implied, including any implied warranties of merchantability, fitness for a particular purpose, non-infringement, accuracy, or reliability.
Bioptimus does not warrant that the models will be uninterrupted, error-free, secure, or suitable for any particular use, nor that any outputs will be accurate, complete, or fit for your purposes.
To the maximum extent permitted by applicable law, Bioptimus disclaims liability for any losses, claims, damages, or decisions — including indirect, consequential, or incidental damages — arising out of or related to use of the models or their outputs.
## Data attribution
The example figures throughout this documentation were generated from **open-access [TCGA](https://www.cancer.gov/tcga) data** (whole-slide images and RNA-seq) obtained from the [NCI Genomic Data Commons](https://portal.gdc.cancer.gov/). TCGA data is open access, individually non-identifiable, and carries no restrictions on use in publications or presentations. In line with TCGA's request, we acknowledge:
> The results shown here are in whole or part based upon data generated by the TCGA Research Network: [https://www.cancer.gov/tcga](https://www.cancer.gov/tcga).
The figures are **real model outputs** on individual public slides (e.g. TCGA-LUAD case `TCGA-75-7027`), shown as representative examples of model behavior. They are not performance benchmarks, and outputs vary by slide, tissue type, and model variant.
# Drug target & biomarker discovery
Source: https://docs.bioptimus.com/documentation/use-cases/biomarker-discovery
Predict drug target distribution, immune microenvironment, and stromal architecture from routine histology slides.
Routine H\&E slides carry far more signal than the standard read extracts. Bioptimus models predict the spatial distribution of drug targets, the immune microenvironment, and stromal architecture directly from slides you already have — no new assays required to begin.
## What it enables
* Turn existing H\&E archives into a discovery substrate for targets and biomarkers
* Characterize the tumor microenvironment spatially, not just globally
* Generate hypotheses before committing to expensive spatial or molecular assays
## How it works
Run [H-Optimus](/documentation/models/h-optimus) over slide tiles to produce tile-level embeddings, or [M-Optimus](/documentation/models/m-optimus) when molecular data is available.
Train lightweight downstream models on the embeddings to predict target expression, microenvironment features, or architecture.
Use tile coordinates to render spatial maps of the predicted signal.
Deploy a model and extract your first features.
# Diagnostics & spatial biology
Source: https://docs.bioptimus.com/documentation/use-cases/diagnostics-spatial-biology
Integrate histology, spatial transcriptomics, and genomics in a single pipeline.
The standard workup leaves signal on the table. Bioptimus models integrate histology with spatial transcriptomics and genomics in one pipeline, reaching resolution the routine read does not.
## What it enables
* Combine modalities that are usually analyzed in isolation
* Surface clinically actionable biology at higher resolution
* Streamline multimodal workflows into a single feature pipeline
## How it works
Use [M-Optimus](/documentation/models/m-optimus) to jointly represent histology and molecular inputs.
Feed slides and molecular vectors through one inference path (see the [request schema](/api-reference/introduction)).
Map predictions back to slide coordinates for spatial interpretation.
# Indication expansion
Source: https://docs.bioptimus.com/documentation/use-cases/indication-expansion
Identify the patient subsets and indications where a therapeutic is most likely to succeed.
Your drug works — but for which patients, and in which other diseases? Bioptimus models score archival cohorts across disease, stage, and mechanism of action to surface the patient subsets whose spatial biology matches what your drug needs.
## What it enables
* Screen thousands of archived slides from completed trials as a biological engine
* Prioritize new indications grounded in spatial and molecular biology
* Focus enrollment on subsets most likely to respond
## How it works
Run [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus) across archived slides at scale with [cohorts](/guides/workflows/cohort).
Build downstream classifiers that score cohorts by disease, stage, and mechanism of action.
Identify patient subsets and indications with the strongest signal.
# Treatment response & trial design
Source: https://docs.bioptimus.com/documentation/use-cases/treatment-response
Discover multimodal biomarker signatures that distinguish responders from non-responders.
Two patients, same diagnosis, same standard biomarkers — different spatial biology and different outcomes. Bioptimus models discover multimodal signatures that separate responders from non-responders, even from the small cohorts typical of early-phase trials.
## What it enables
* Build response-predictive signatures from limited early-phase data
* Support patient enrichment strategies before late-stage commitment
* Inform trial design with biology the standard biomarker panel misses
## How it works
Extract features with [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus).
Train downstream models that distinguish responders from non-responders.
Use the signature to inform enrollment and trial design decisions.
# Inference pipeline
Source: https://docs.bioptimus.com/guides/get-started/inference-pipeline
Configure the whole pipeline with one object — tissue masking, embeddings, predictions, and reproducible workspaces.
The `Inference` pipeline replaces the multi-step manual pipeline (`Backbone` + mask provider + `SlideInference` + writer) with a single, reusable object. The constructor is the pipeline config; the methods do the work. It caches tissue masks, organizes outputs into a structured workspace, and can serialize its config for reproducibility.
## Configure the pipeline
Pick your backend and model — the rest of the guide follows the same steps.
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.H1,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="h1-embedding-demo",
run=1,
description="H1 embeddings with tissue masking",
)
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.M_OPTIMUS,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="m-optimus-demo",
run=1,
description="M-Optimus gene expression with tissue masking",
)
```
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.H1,
backend="aws",
endpoint_name="h-optimus",
region_name="us-east-1",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="h1-embedding-demo",
run=1,
workers=1, # ml.g5.xlarge has a single GPU
description="H1 embeddings with tissue masking",
)
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.M_OPTIMUS,
backend="aws",
endpoint_name="m-optimus",
region_name="us-east-1",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="m-optimus-demo",
run=1,
workers=1, # ml.g5.xlarge has a single GPU
description="M-Optimus gene expression with tissue masking",
)
```
## Run the pipeline
```python H-Optimus theme={null}
infer.tissue(wsi_path) # generate (or load cached) tissue mask
result_path = infer.embed(wsi_path) # -> /h1/embeddings/.zarr
```
```python M-Optimus theme={null}
infer.tissue(wsi_path) # generate (or load cached) tissue mask
result_path = infer.predict(wsi_path) # -> /m-optimus/predictions/.zarr
```
The tissue mask is cached to `/tissue/.png` and reused automatically on later calls.
If the first call raises a connection error, the on-premise server behind `api_url` isn't reachable yet. Verify it with `requests.get(f"{api_url}/ping", timeout=5)` (a healthy server returns `{"status": "ok", ...}`) and see [Server not responding?](/guides/get-started/sdk#connecting-to-a-model) for how to start it.
## Reproducibility
The config is auto-saved to `/config.yaml` on first use. Reconstruct the exact pipeline later:
```python theme={null}
infer.save_config()
infer2 = Inference.from_workspace(workspace_dir) # same workspace; cached masks reused
```
The same object works across multiple slides — call `infer.embed([s1, s2])` (or `infer.predict([s1, s2])` for M-Optimus) rather than creating a new `Inference` per slide. For full cohorts with bulk RNA, see [Cohorts](/guides/workflows/cohort).
## Workspaces & variants
Your workspace is the directory where all outputs are written. It's resolved from the config you pass:
```text theme={null}
//run_//
```
Only `output_path` is required; `experiment`, `run`, and `variant` are appended when set:
| Segment | Config field | Appended when |
| --------------- | ------------- | ----------------- |
| `` | `output_path` | always (required) |
| `` | `experiment` | set |
| `run_` | `run` | set |
| `` | `variant` | set |
`variant` (a string) adds a final sub-folder so you can keep multiple runs side by side — useful for experiments (e.g. comparing mask thresholds or model versions) without overwriting earlier outputs:
```python theme={null}
infer = Inference(model_name=Models.H1, api_url="http://localhost:8080",
output_path="/data/output", experiment="tcga_coad", run=1,
variant="mask_0.5")
# -> /data/output/tcga_coad/run_1/mask_0.5/
```
A populated workspace looks like:
```text H-Optimus theme={null}
//run_//
config.yaml
manifest.yaml
tissue/.png
h1/embeddings/.zarr
```
```text M-Optimus theme={null}
//run_//
config.yaml
manifest.yaml
tissue/.png
m-optimus/predictions/.zarr
```
# Bioptimus SDK
Source: https://docs.bioptimus.com/guides/get-started/sdk
The Bioptimus Python SDK — installation and the two ways to use it.
The Bioptimus Python SDK runs whole-slide inference against either an [on-premise server](/deployment/platforms/on-premise) or a [SageMaker endpoint](/deployment/platforms/aws-sagemaker). It handles WSI reading, tiling, tissue masking, bulk-RNA alignment, and concurrent dispatch.
The Bioptimus SDK talks to whichever model is deployed at the endpoint you connect to. `Backbone.available_backbones()` lists the models the Bioptimus SDK *knows how to build* (`h1`, `m-optimus`, `tissue-seg`) — to see what a server actually has loaded, check its [`/ping`](/api-reference/introduction) response.
## Installation
```bash theme={null}
pip install bioptimus-sdk
```
```bash theme={null}
pip install bioptimus_sdk--py3-none-any.whl
```
## Two ways to use it
One object configures the whole pipeline. Caches tissue masks, organizes outputs into a workspace, and is reproducible. Best for most users and for cohorts.
`Backbone` + `SlideInference` give explicit, per-slide control over the model client, mask provider, and writer.
## Connecting to a model
The `Backbone` factory is the low-level client used by both layers.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg']
model = Backbone(Models.H1, backend="remote", base_url="http://localhost:8080")
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg']
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url="http://localhost:8080")
```
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.H1, backend="aws",
endpoint_name="h-optimus", region_name="us-east-1",
)
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1",
)
```
For M-Optimus, gene sets are fetched from the server automatically (`model.input_gene_names`, `model.output_gene_names`).
Before constructing a `Backbone` (or an `Inference` pipeline), confirm the on-premise server is reachable:
```python theme={null}
import requests
requests.get("http://localhost:8080/ping", timeout=5).json()
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
If this raises `ConnectionError` / connection refused or times out, nothing is serving at that URL:
* **Not started** — launch the container, mapping port 8080, then wait for models to load. See [On-premise deployment](/deployment/platforms/on-premise):
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 serve
```
* **`503 {"status": "loading"}`** — models are still initialising; wait and retry.
* **Wrong URL/port** — `base_url` (and the pipeline's `api_url`) must be the server's host and port, with **no** trailing `/ping`.
For SageMaker there is no `/ping`: confirm the endpoint is `InService` and that `endpoint_name` / `region_name` are correct.
## Guides
One-object pipeline, workspaces, reproducible config.
Multi-slide cohorts and late-binding bulk RNA.
M-Optimus gene-expression prediction end to end.
Extract embeddings and visualize morphology.
Read slides: levels, MPP, regions, thumbnails.
Load Zarr/HDF5/NPZ and overlay genes and masks.
## Output formats
| Format | Extension | Notes |
| ------------------- | --------- | ------------------------------------------ |
| `OutputFormat.ZARR` | `.zarr` | Default. Directory store, memory-efficient |
| `OutputFormat.HDF5` | `.h5` | Single file, memory-efficient |
| `OutputFormat.NPZ` | `.npz` | Accumulates in memory, compressed on close |
Every output file contains the same contents:
* **Datasets:** `outputs`, `coords`, `tissue_ratios`, `thumbnail`, `tissue_mask` — plus `input_gene_names` and `output_gene_names` for M-Optimus.
* **Metadata attributes:** `slide_name`, `tile_size`, `stride`, `mpp`, `slide_dimensions`, `slide_dimensions_at_mpp`, `num_tiles`.
See [Visualizing results](/guides/get-started/visualizing-results) to load and plot them.
# Visualizing results
Source: https://docs.bioptimus.com/guides/get-started/visualizing-results
Load Zarr/HDF5/NPZ outputs and overlay gene expression or tissue masks.
Inference outputs are written as Zarr (default), HDF5, or NPZ — all three carry the same datasets under the same keys.
Figures in this documentation are real model outputs on open-access **TCGA** slides (e.g. `TCGA-75-7027`), shown as representative examples — not benchmarks. See [Responsible use → Data attribution](/documentation/resources/responsible-use#data-attribution).
## Load an output file
Load a Zarr output with the built-in helper; it returns the arrays and metadata as a dict.
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
preds, coords = data["outputs"], data["coords"]
gene_names = data.get("gene_names") # M-Optimus gene predictions only; None for embeddings
```
`load_zarr_output` returns the following datasets (plus a `metadata` dict). Optional entries are present only when the model and pipeline produce them:
| Key | Shape | Description |
| --------------- | -------------- | ------------------------------------------------------------------------------------ |
| `outputs` | `(n_tiles, D)` | Per-tile embeddings (`D` = 1536) or gene predictions (`D` = number of output genes). |
| `coords` | `(n_tiles, 2)` | Pixel `(x, y)` location of each tile on the slide. |
| `gene_names` | `(D,)` | Output gene identifiers (M-Optimus only). |
| `thumbnail` | `(H, W, 3)` | Downsampled RGB slide overview. |
| `tissue_mask` | `(H, W)` | Boolean tissue-vs-background mask. |
| `tissue_ratios` | `(n_tiles,)` | Fraction of each tile covered by tissue. |
Metadata attributes include `slide_dimensions`, `slide_dimensions_at_mpp`, `tile_size`, `mpp`, and `num_tiles`. HDF5 (`.h5`) and NPZ (`.npz`) outputs expose the same keys via `h5py` and `np.load(allow_pickle=True)`.
## Plot with the built-in helpers
The helpers below visualize **M-Optimus gene predictions** and require the `gene_names` returned above. For H-Optimus tile embeddings (no `gene_names`), see [Tile embeddings & PCA](/guides/workflows/embeddings-pca).
`bioptimus.utils` ships the plotting helpers used in the [getting-started notebooks](https://github.com/bioptimus/m-jumpstart). Heatmap overlays and single-gene overlays are all covered by the [spatial gene panel](#spatial-gene-panel) helper below — pass a single-entry `GENE_PANEL` for one gene — so there's no need to hand-roll matplotlib.
### Spatial gene panel
Overlay one or more genes on the slide thumbnail in one call.
```python theme={null}
GENE_PANEL = {
"EPCAM": "ENSG00000119888", # epithelium / tumor
"CD3E": "ENSG00000198851", # T-cells
"CD8A": "ENSG00000153563", # cytotoxic T-cells
"MKI67": "ENSG00000148773", # proliferation
"COL1A1": "ENSG00000108821", # stroma
"SFTPC": "ENSG00000168484", # alveolar (lung)
}
utils.plot_gene_panel_overlay(preds, coords, gene_names, GENE_PANEL,
wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Spatial gene-expression panel overlaid on a slide thumbnail]()
### Image-only vs. bulk-RNA-guided
Compare the same gene predicted with and without bulk RNA context.
```python theme={null}
gene_idx = gene_names.index("ENSG00000119888") # EPCAM
utils.plot_gene_overlay_comparison(
preds_bulk, coords_bulk, preds_image_only, coords_image_only,
gene_idx=gene_idx, wsi_path=wsi_path, gene_name="EPCAM",
label_a="With bulk RNA", label_b="Without bulk RNA (image only)",
)
```
![Side-by-side EPCAM overlay: with bulk RNA vs image only]()
### Highest- and lowest-expressing tiles
Pull the tiles driving a gene's prediction for quick visual QC.
```python theme={null}
utils.plot_top_gene_tiles(preds, coords, gene_idx=gene_idx,
wsi_path=wsi_path, gene_name="EPCAM",
n_top=6, n_bottom=6)
```
![Grid of tiles with highest and lowest predicted EPCAM expression]()
# WSI processing
Source: https://docs.bioptimus.com/guides/reference/wsi-processing
Read whole-slide images: pyramid levels, MPP, regions, and thumbnails.
The Bioptimus SDK's `WSI` factory provides uniform access to whole-slide images, automatically selecting the best available backend.
**Supported backends** (auto-selected, in order of preference):
* **CuCIM** — GPU-accelerated reading, used when available.
* **OpenSlide** — CPU fallback, broad format support.
**Supported formats** include `.svs`, `.tiff`, `.tif`, `.ndpi`, `.vms`, `.vsi`, `.scn`, `.mrxs`, and `.jp2` (subject to the active backend).
## Open a slide
```python theme={null}
from bioptimus.io.wsi import WSI, Level, MPP, Magnification, MeasurementUnit
wsi = WSI("/data/wsi/tcga_coad.svs")
print(wsi.level_count) # pyramid levels (0 = highest resolution)
print(wsi.level_dimensions(Level(0))) # base dimensions
print(wsi.mpp) # microns per pixel at level 0
```
## Properties
Slides carry rich scanning metadata. Access it through `props`, and inspect the pyramid with `level_dimensions` and `level_downsample`.
```python theme={null}
props = wsi.props
print(props.SCANNER) # scanning device
print(props.OBJECTIVE_POWER) # objective magnification (e.g. 40)
print(props.MPP_X, props.MPP_Y)
# Per-level pyramid structure:
for level in range(wsi.level_count):
dims = wsi.level_dimensions(Level(level))
ds = wsi.level_downsample(Level(level))
print(f"Level {level}: {dims.width}x{dims.height} (downsample {ds:.1f}x)")
```
## Bounded vs. unbounded
Scanners capture the whole slide, but tissue occupies only part of it. `bounded=True` returns the tissue area; `bounded=False` returns the full slide.
```python theme={null}
tissue = wsi.dimensions(bounded=True) # tissue area only
full = wsi.dimensions(bounded=False) # entire slide
```
## Read a region
Specify resolution three ways — by pyramid `Level`, by physical `MPP`, or by `Magnification` — and choose pixel or micrometer units.
```python theme={null}
# By pyramid level, size in pixels (Level 2 ≈ 16× downsample), tissue-bounded:
region_px = wsi.read_region(location=(1625, 0), size=(1200, 1200),
resolution=Level(2),
measurement_unit=MeasurementUnit.PIXELS, bounded=True)
# The same window requested in micrometers (physical units):
region_um = wsi.read_region(location=(1625, 0), size=(599.0, 599.0),
resolution=Level(2),
measurement_unit=MeasurementUnit.UM, bounded=True)
img = region_px.image # PIL image
print(region_px.image.size, region_px.resolution)
```
![Region read in pixels versus micrometers, side by side]()
`MPP ≈ 10 / magnification` (40× ≈ 0.25 MPP, 20× ≈ 0.5 MPP, 10× ≈ 1.0 MPP). Bioptimus embeddings use 0.5 MPP; tissue segmentation uses 8 MPP.
## Thumbnails & associated images
```python theme={null}
thumb = wsi.get_thumbnail(size=(512, 512), bounded=False) # PIL image
images = wsi.associated_images # {'thumbnail','label','macro'}
macro = images["macro"] # slide-level macro photograph
props = wsi.props # scanner, objective, MPP_X/Y, ...
```
`size` is an upper bound: the aspect ratio is preserved, so the longer side is fit to the requested dimension and the shorter side scaled down proportionally. This slide is slightly taller than wide, so requesting `(512, 512)` yields 486×512 px rather than an exact 512×512.
![Whole-slide image thumbnail]()
![Macro associated image of the whole slide]()
For inference you don't usually call the reader directly — the [Inference pipeline](/guides/get-started/inference-pipeline) and `SlideInference` handle tiling. Use the reader for QC, custom region extraction, or building your own pipelines.
# Cohorts
Source: https://docs.bioptimus.com/guides/workflows/cohort
Run a model over a cohort of slides with shared tissue masks and a structured workspace.
A `Cohort` is the single source of truth for a multi-slide experiment. You build it from a directory of WSIs (or a manifest CSV), run a model over it, and outputs are organized into a reproducible workspace. Cohorts can also hold bulk RNA for multimodal prediction — see [Spatial transcriptomics](/guides/workflows/spatial-transcriptomics#3-add-bulk-rna-multimodal-prediction).
`m-jumpstart` includes a cohort batch-processing example.
## 1. Build a cohort
```python theme={null}
from bioptimus.data.cohort import Cohort
cohort = Cohort.from_directories(wsi_dir="/data/wsi/tcga_mini_coad")
```
```python theme={null}
from bioptimus.data.cohort import Cohort
# One row per slide. Required column: `wsi_id` (filename stem or full name).
# Optional: `patient_id`, `bulk_rna_id`, `timepoint`, `wsi_path`, `bulk_rna_path`.
# Pass `wsi_dir=` / `bulk_rna_dir=` to resolve paths from IDs, or
# `columns={...}` to map custom column names.
cohort = Cohort.from_csv("/data/cohort.csv", wsi_dir="/data/wsi/tcga_mini_coad")
```
```python theme={null}
print(cohort.summary())
print(cohort.wsi_ids)
print(cohort[0].available_modalities) # e.g. ['image']
```
## 2. Run a model over the cohort
Create one `Inference` for the cohort, then run it. Tissue masks are cached and resume automatically, so re-running only processes what's missing. Pick your backend below (only the `common` dict differs), then your model.
```python theme={null}
common = dict(
api_url="http://localhost:8080",
tissue=True, mask_threshold=0.5,
output_path="/data/output", experiment="tcga_coad", run=1,
workers=5,
)
```
```python theme={null}
common = dict(
backend="aws", endpoint_name="bioptimus-prod", region_name="us-east-1",
tissue=True, mask_threshold=0.5,
output_path="/data/output", experiment="tcga_coad", run=1,
workers=1, # ml.g5.xlarge has a single GPU; concurrent requests can OOM
)
```
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(model_name=Models.H1, cohort=cohort, variant="mini", **common)
infer.tissue() # shared masks; only computes what's missing
infer.run(mode="embed") # H-Optimus produces embeddings
infer.report() # status summary
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(model_name=Models.M_OPTIMUS, cohort=cohort, variant="mini", **common)
infer.tissue() # shared masks; only computes what's missing
infer.run(mode="embed")
infer.run(mode="predict") # image-only (see Spatial transcriptomics for bulk RNA)
infer.report() # status summary
```
Each output is tagged with the modalities used (e.g. `["image"]`).
## 3. Extract & save tiles (optional)
Independent of inference — useful for QC or external pipelines:
```python theme={null}
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
from bioptimus.io.wsi.factory import WSI
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
# Reuse the model's tile geometry (224×224 @ 0.5 µm/px) so tiles match inference.
tile_spec = Backbone(Models.H1, base_url="http://localhost:8080").model_spec.tile_spec
extractor = TileExtractor(tile_spec=tile_spec, mask_threshold=0.5)
with WSI(wsi_path) as reader:
extractor.fit_extract(reader)
extractor.save(tile_dir, image_format="png", workers=4)
extractor.to_csv(csv_dir)
```
# Tile embeddings & PCA
Source: https://docs.bioptimus.com/guides/workflows/embeddings-pca
Extract tile embeddings (H-Optimus or M-Optimus) and visualize morphology with PCA.
Both H-Optimus and M-Optimus produce a 1536-dimensional embedding per tile (the CLS token). Principal components of these embeddings reveal the dominant axes of morphological variation — often corresponding to tissue vs. background, tumor vs. stroma, and immune-rich vs. immune-poor regions.
`h1-jumpstart` — a full, runnable H-Optimus embedding example.
`m-jumpstart` — the equivalent for M-Optimus.
## 1. Configure the model
Embeddings are available from both models. Import once, then pick your backend and model — the rest of the guide is identical.
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
```
```python H-Optimus theme={null}
API_URL = "http://localhost:8080"
model = Backbone(Models.H1, backend="remote", base_url=API_URL)
tissue = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python M-Optimus theme={null}
API_URL = "http://localhost:8080"
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url=API_URL)
tissue = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python H-Optimus theme={null}
model = Backbone(Models.H1, backend="aws",
endpoint_name="h-optimus", region_name="us-east-1")
tissue = Backbone("tissue-seg", backend="aws",
endpoint_name="h-optimus", region_name="us-east-1")
```
```python M-Optimus theme={null}
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
tissue = Backbone("tissue-seg", backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
```python theme={null}
mask_provider = TiledTissueMask(model=BioptimusTissueMaskModel(backbone=tissue))
```
## 2. Extract embeddings
```python theme={null}
from bioptimus.inference.inference import SlideInference as Inference
from bioptimus.inference.writers import OutputFormat
inferrer = Inference(wsi_path="/data/wsi/tcga_coad.svs", model=model,
mask_provider=mask_provider, mask_threshold=0.5)
result_path = inferrer.predict(output_path="/data/output/tcga_coad.zarr",
output_format=OutputFormat.ZARR,
max_concurrency=256, mode="embed")
```
On a single-GPU SageMaker endpoint (`ml.g5.xlarge`), lower `max_concurrency` (e.g. 32) to avoid out-of-memory from concurrent tile requests.
## 3. PCA
Load the embeddings and fit PCA in one call — `bioptimus.utils.plot_pca_scatter` fits, plots, and returns the scores plus the fitted PCA (explained-variance ratios are printed):
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
embeddings, coords = data["outputs"], data["coords"] # (N, 1536), (N, 2)
scores, pca = utils.plot_pca_scatter(embeddings, n_components=3,
title="Embeddings PCA", cmap="plasma")
```
![Scatter plot of the first two principal components of tile embeddings]()
## 4. Spatial visualization
`plot_spatial_pca` maps the PC scores back onto tile coordinates as per-PC spatial heatmaps — each component tends to isolate a distinct tissue compartment:
```python theme={null}
utils.plot_spatial_pca(scores, coords, components=[0, 1, 2])
```
![Three spatial heatmaps, one per principal component, overlaid on tile coordinates]()
The same embeddings feed downstream models (classification, clustering, retrieval, MIL). For the high-level workflow with caching and workspaces, use the [Inference pipeline](/guides/get-started/inference-pipeline) with `infer.embed(path)`.
# Tissue masking & tiling
Source: https://docs.bioptimus.com/guides/workflows/preprocessing
Preprocess a slide before inference: segment tissue to drop background, then split it into model-sized tiles.
Before any embedding or gene-expression model runs, two preprocessing steps prepare the slide:
1. **Tissue masking** — a whole-slide image is mostly background. [Tissue segmentation](/documentation/models/tissue-segmentation) produces a binary mask so background is discarded before the expensive feature step.
2. **Tiling** — the slide is too large to process at once, so it is split into small, model-sized tiles. Only tiles that overlap tissue are kept.
Resolutions differ by stage: tissue segmentation runs at **8 µm/px** (coarse, 512×512 tiles), while embeddings and gene predictions run at **0.5 µm/px** (224×224 tiles). The mask is generated once and reused to filter the fine-resolution grid.
The [Inference pipeline](/guides/get-started/inference-pipeline) does both steps for you. Reach for the manual API below only when you need masks or tiles as standalone artifacts — for QC, custom filtering, or your own pipeline.
## The quick path: the Inference pipeline
`Inference` masks tissue, caches the result, and reuses it automatically for later `embed`/`predict` calls.
```python theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
from bioptimus import utils
infer = Inference(
model_name=Models.H1,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="preprocessing-demo",
run=1,
)
mask = infer.tissue(wsi_path) # uint8 (H, W) array; cached to /tissue/.png
```
The mask is cached to `/tissue/.png` and reused on the next `tissue()`, `embed()`, or `predict()` call. Visualize it against the slide thumbnail with the built-in helper:
```python theme={null}
mask_png = "/data/output/preprocessing-demo/run_1/tissue/.png"
utils.plot_slide_and_mask(wsi_path, mask_png, threshold=0.5)
```
![Slide thumbnail alongside its binary tissue mask]()
## The manual path: masks and tiles as artifacts
### 1. Generate a tissue mask
Build a mask provider from the tissue-seg endpoint. It is reusable across slides — it generates a fresh mask per slide.
```python theme={null}
from bioptimus.io.wsi import WSI
from bioptimus.models.backbones import Backbone
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
tissue_backbone = Backbone("tissue-seg", base_url="http://localhost:8080")
provider = TiledTissueMask(model=BioptimusTissueMaskModel(tissue_backbone))
wsi = WSI(wsi_path)
mask = provider.generate(wsi) # uint8 (H, W) in {0, 1} over the tissue bounding box at 8 µm/px
```
Already have masks on disk? Use them directly instead of the endpoint. Non-zero pixels mean tissue, and files are matched to slides by filename stem (PNG, JPEG, TIFF, BMP, or `.npy`):
```python theme={null}
from bioptimus.preprocess.wsi.provider.precomputed import PrecomputedTissueMask
provider = PrecomputedTissueMask(mask_dir="masks/", suffix=".png")
```
### 2. Tile the slide
A `TileSpec` defines the tile geometry; `TileExtractor` builds the grid, filters it against the mask, and exports the tiles. Passing `mask_provider` lets the extractor generate the per-slide mask during `fit`.
```python theme={null}
from bioptimus.io.wsi import MPP, MeasurementUnit
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
spec = TileSpec(
size=(224, 224),
stride=(224, 224),
resolution=MPP(0.5),
unit=MeasurementUnit.PIXELS,
)
extractor = TileExtractor(tile_spec=spec, mask_provider=provider, mask_threshold=0.5)
tiles = extractor.fit_extract(wsi) # keeps only tiles with >= 50% tissue
extractor.save("tiles/", image_format="png", workers=4) # writes tile images
extractor.to_csv("tiles/") # writes tiles/.csv
print(f"{len(tiles)} tiles kept")
```
`to_csv` writes one row per retained tile:
| Column | Description |
| ------------------------------------- | ------------------------------------------------------- |
| `slide_name` | Slide filename stem. |
| `top`, `left` | Tile top-left location in absolute slide coordinates. |
| `width`, `height` | Tile size. |
| `unit` | Measurement unit (`PIXELS` or `UM`). |
| `resolution_type`, `resolution_value` | Resolution the tile is defined at (e.g. `mpp` / `0.5`). |
| `tissue_ratio` | Fraction of the tile covered by tissue. |
`save` writes each tile image as `{slide}_x{left}_y{top}_w{width}_h{height}_{res}_mask_r{ratio}.png`.
## Tuning knobs
| Knob | Where | Effect |
| --------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------- |
| `mask_threshold` | `Inference`, `TileExtractor` | Minimum tissue fraction (0–1) to keep a tile. Lower keeps more border tiles; `None` disables filtering. |
| `size` / `stride` | `TileSpec` | Tile size and step. Equal size and stride give a non-overlapping grid; a smaller stride overlaps tiles. |
| `resolution` / `unit` | `TileSpec` | Physical resolution of tiles. Bioptimus feature models expect 224×224 at 0.5 µm/px. |
| `max_concurrency` | `TiledTissueMask.generate` | Concurrent requests to the tissue-seg endpoint (default 64). |
| `workers` | `TileExtractor.save` | Threads for writing tile images (default 4). |
For end-to-end inference you rarely call these directly — the [Inference pipeline](/guides/get-started/inference-pipeline) handles masking and tiling and streams results to disk. Use the manual API for QC, custom region extraction, or building your own pipeline. To read regions, thumbnails, and slide metadata, see [WSI processing](/guides/reference/wsi-processing).
# Spatial transcriptomics
Source: https://docs.bioptimus.com/guides/workflows/spatial-transcriptomics
Predict spatial gene expression from an H&E slide with M-Optimus, end to end.
M-Optimus predicts expression of thousands of genes directly from H\&E histology — no spatial transcriptomics assay required — enabling spatial analysis on any archived slide. This guide uses the core API (`Backbone` + `SlideInference`) for explicit control; the [Inference pipeline](/guides/get-started/inference-pipeline) wraps the same steps.
See `m-jumpstart` for a full, runnable M-Optimus example.
## 1. Configure the model and tissue masking
Pick your backend — the rest of the guide is identical for both.
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
API_URL = "http://localhost:8080"
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url=API_URL)
tissue_backbone = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
tissue_backbone = Backbone("tissue-seg", backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
```python theme={null}
mask_provider = TiledTissueMask(model=BioptimusTissueMaskModel(backbone=tissue_backbone))
print(f"{len(model.input_gene_names or [])} input genes, "
f"{len(model.output_gene_names or [])} output genes")
```
## 2. Run inference (image only)
`SlideInference` tiles the slide at 0.5 µm/px, dispatches tiles concurrently, and streams results to Zarr so memory stays constant regardless of slide size.
```python theme={null}
from bioptimus.inference.inference import SlideInference
from bioptimus.inference.writers import OutputFormat
inferrer = SlideInference(
wsi_path="/data/wsi/tcga_coad.svs",
model=model,
mask_provider=mask_provider,
mask_threshold=0.5,
)
result_path = inferrer.predict(
output_path="/data/output/tcga_coad.zarr",
output_format=OutputFormat.ZARR,
max_concurrency=256, # see note for single-GPU SageMaker endpoints
)
```
Tissue masking filters background tiles before prediction — on a typical TCGA slide this skips most of the \~16k tile grid. Set `mask_threshold` to the minimum tissue fraction to keep a tile (default 0.5).
On a single-GPU SageMaker endpoint (`ml.g5.xlarge`), lower `max_concurrency` (e.g. 32) to avoid out-of-memory from concurrent tile requests. On a multi-GPU on-premise server, 256 is a reasonable default.
## 3. Add bulk RNA (multimodal prediction)
M-Optimus prediction can be conditioned on **bulk RNA** for a multimodal readout. Bulk RNA is attached through a [`Cohort`](/guides/workflows/cohort) via late-binding: run image-only first, then link the RNA and re-predict. Modality-aware tracking processes only the missing outputs, and image-only results are preserved separately for comparison.
Send **TPM-normalized expression values** — do **not** `log1p` them yourself. The server transforms the input and the model output for you. Genes are auto-aligned to the model's input set; genes the model expects but your file lacks are encoded as `-1` (non-measured).
The server applies two transforms around the model:
| Stage | Transform | Applied to |
| -------------------- | --------- | ------------------------------------------------------- |
| Input (preprocess) | `log1p` | your TPM-normalized expression values, before the model |
| Output (postprocess) | `expm1` | the model output, to return expression values |
```python theme={null}
from bioptimus.data.cohort import Cohort
from bioptimus.inference import Inference
from bioptimus.models.types import Models
cohort = Cohort.from_directories(wsi_dir="/data/wsi/tcga_mini_coad")
infer_m = Inference(model_name=Models.M_OPTIMUS, cohort=cohort,
api_url="http://localhost:8080", tissue=True,
output_path="/data/output", experiment="tcga_coad", run=1)
infer_m.run(mode="predict") # image-only baseline
```
Link bulk RNA and re-predict:
```python theme={null}
# Plain CSV (Ensembl IDs as columns):
cohort.link_bulk_rna("/data/rna/tcga_mini_coad")
# GDC/TCGA gene-quantification TSVs — use the tpm_unstranded column (not
# fpkm_* or raw counts) and strip Ensembl versions:
cohort.link_bulk_rna(
"/data/rna/tcga_mini_coad",
gene_column="gene_id",
value_column="tpm_unstranded", # TPM-normalized expression values (the expected scale)
strip_version=True, # ENSG00000000003.15 -> ENSG00000000003
)
infer_m.run(mode="predict", force=True) # re-predict with bulk RNA context
```
Retrieve each stage's output by modality:
```python theme={null}
out = cohort[0].outputs.get(infer_m.model_name)
img_only = out.get_stage_path("predict", ["image"])
multimodal = out.get_stage_path("predict", ["image", "bulk_rna"])
```
## 4. Inspect the output
Load either output — image-only or multimodal — with the built-in helper:
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
outputs, coords = data["outputs"], data["coords"] # (n_tiles, n_genes), (n_tiles, 2)
gene_names = data["gene_names"]
```
## 5. Visualize
Genes are identified by **Ensembl gene ID** (e.g. `ENSG00000198851`). Useful markers include EPCAM (`ENSG00000119888`, epithelium), CD3E (`ENSG00000198851`, T-cells), CD8A (`ENSG00000153563`, cytotoxic T-cells), MKI67 (`ENSG00000148773`, proliferation), COL1A1 (`ENSG00000108821`, stroma), and SFTPC (`ENSG00000168484`, alveolar).
`bioptimus.utils.plot_gene_panel_overlay` renders a whole panel on the slide thumbnail in one call. Load the two prediction sets from step 3 and compare them — the multimodal output uses bulk RNA, the image-only output does not:
```python theme={null}
from bioptimus import utils
GENE_PANEL = {
"EPCAM": "ENSG00000119888", # epithelium / tumor
"CD3E": "ENSG00000198851", # T-cells
"CD8A": "ENSG00000153563", # cytotoxic T-cells
"MKI67": "ENSG00000148773", # proliferation
"COL1A1": "ENSG00000108821", # stroma
"SFTPC": "ENSG00000168484", # alveolar (lung)
}
```
```python theme={null}
bulk = utils.load_zarr_output(multimodal)
utils.plot_gene_panel_overlay(bulk["outputs"], bulk["coords"], bulk["gene_names"],
GENE_PANEL, wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Six-gene spatial expression panel, image plus bulk RNA]()
```python theme={null}
image = utils.load_zarr_output(img_only)
utils.plot_gene_panel_overlay(image["outputs"], image["coords"], image["gene_names"],
GENE_PANEL, wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Six-gene spatial expression panel, image only]()
A FP16 model variant is available for faster inference. Run the FP32 and FP16 outputs through the same analysis to compare the impact of quantization.
# License
Source: https://docs.bioptimus.com/license
Copyright (c) 2026 Bioptimus. All rights reserved.
## Commercial and Proprietary License
This software, including all source code, documentation, configuration files, model artifacts, and binary packages (including but not limited to `.pt2` files), is the exclusive property of Bioptimus.
**All rights are reserved.** No part of this software may be reproduced, distributed, transmitted, modified, or used in any form or by any means without the prior written permission of Bioptimus.
## Restrictions
1. **No unauthorized use.** Use of this software requires a separate license agreement or contract with Bioptimus.
2. **No reverse engineering.** Reverse engineering, decompilation, disassembly, or any other attempt to derive source code, algorithms, model weights, or training data from any binary packages — specifically including `.pt2` model files — is strictly prohibited.
3. **No redistribution.** This software may not be copied, shared, sublicensed, sold, or otherwise distributed to any third party without explicit written authorization from Bioptimus.
4. **No derivative works.** Creating derivative works based on this software or any of its components is prohibited without a license agreement.
## Disclaimer
THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL BIOPTIMUS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY ARISING FROM THE USE OF THIS SOFTWARE.
## Contact
To obtain a license or for any inquiries regarding the use of this software:
**Bioptimus** 86 90 Rue Notre Dame de Nazareth 75003 Paris, France
Email: [contact@bioptimus.com](mailto:contact@bioptimus.com) Website: [https://www.bioptimus.com/contact](https://www.bioptimus.com/contact) LinkedIn: [https://www.linkedin.com/company/bioptimus](https://www.linkedin.com/company/bioptimus)
# cohort
Source: https://docs.bioptimus.com/sdk-reference/data/cohort
Cohort manifest for pairing WSIs with bulk RNA and clinical metadata.
Provides `Cohort`, a typed registry that tracks the mapping between patients, wsis, bulk RNA samples, timepoints, labels, and arbitrary clinical metadata. `WSIRecord` is the single source of truth for each WSI — including per-model outputs and processing status.
Construction paths:
1. **From a user-provided CSV** via `from_csv`.
2. **Auto-matching from directories** via `from_directories`.
3. **From a YAML manifest** via `load` (for resume).
The cohort can be serialised to YAML via `save` and loaded back for full reproducibility and resume support.
**Example:**
```python theme={null}
from bioptimus.data.cohort import Cohort
# Auto-match from directories:
cohort = Cohort.from_directories(
wsi_dir="/data/wsis",
bulk_rna_dir="/data/rna",
)
# Or load a saved manifest (resume):
cohort = Cohort.load(".output/brca/run_1/manifest.yaml")
# Add labels post-hoc:
cohort.add_labels({"patient_001": {"mutation": "BRAF"}})
# Iterate:
for record in cohort:
print(record.patient_id, record.wsi_path)
```
## ModelOutput
```python theme={null}
@dataclass
class ModelOutput()
```
Per-model output record for a single WSI.
Name of the model that produced this output.
Path to the embedding output file.
Path to the prediction output file.
Path to the tile coordinates CSV.
Directory containing exported tile images.
Input modalities used for this output (e.g. `["image"]` or `["image", "bulk_rna"]`).
ISO 8601 string of when the output was produced.
Processing status (`"pending"`, `"done"`, `"error"`).
Error message if status is `"error"`.
#### is\_done
```python theme={null}
@property
def is_done() -> bool
```
Returns `True` if status is done.
#### has\_embedding
```python theme={null}
@property
def has_embedding() -> bool
```
Returns `True` if an embedding path is set and exists.
#### has\_prediction
```python theme={null}
@property
def has_prediction() -> bool
```
Returns `True` if a prediction path is set and exists.
#### set\_stage\_output
```python theme={null}
def set_stage_output(stage: str,
modalities: list[str],
path: Path,
timestamp: str | None = None) -> None
```
Records an output for a specific stage and modality combo.
Updates both the per-modality tracking dict and the top-level convenience fields (`embedding_path` / `prediction_path`).
`"embed"` or `"predict"`.
Modalities used for this output.
Output file path.
ISO 8601 timestamp.
#### get\_stage\_path
```python theme={null}
def get_stage_path(stage: str,
modalities: list[str] | None = None) -> Path | None
```
Returns the output path for a stage and modality combo.
`"embed"` or `"predict"`.
Modality combination to look up. Falls back to the top-level path when `None`.
Resolved path, or `None` if not recorded.
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Serialises to a plain dict for YAML output.
A dictionary representation of the model output.
#### from\_dict
```python theme={null}
@classmethod
def from_dict(cls, data: dict[str, Any]) -> ModelOutput
```
Constructs from a plain dict (YAML round-trip).
Dictionary with serialised model output fields.
A new `ModelOutput` instance.
## WSIRecord
```python theme={null}
@dataclass
class WSIRecord()
```
Single WSI entry — the single source of truth.
Tracks inputs (WSI, RNA, mask), per-model outputs, and arbitrary annotations for one WSI within a cohort.
Unique patient identifier.
Unique WSI identifier (typically the filename stem).
Path to the WSI file.
Identifier for the paired bulk RNA sample.
Path to the bulk RNA CSV/TSV file.
Path to a tissue mask (pre-computed or cached).
When the mask was computed/discovered.
Temporal ordering label (e.g. `"t0"`, `"t1"`).
Per-model output records keyed by model name.
Arbitrary categorical annotations.
Arbitrary clinical/treatment metadata.
#### get\_output
```python theme={null}
def get_output(model_name: str) -> ModelOutput
```
Returns the output for *model\_name*, creating if absent.
Model identifier string.
The `ModelOutput` for the given model.
#### is\_stage\_done
```python theme={null}
def is_stage_done(model_name: str, stage: str) -> bool
```
Checks whether a stage output exists on disk.
When `modality_outputs` are tracked, the check is scoped to the record's current `available_modalities` so that linking new data (e.g. bulk RNA) automatically surfaces pending work without requiring `force=True`.
Model identifier.
`"embed"` or `"predict"`.
`True` if the output path exists on disk.
#### has\_mask
```python theme={null}
@property
def has_mask() -> bool
```
Returns `True` if a mask path is set and exists on disk.
#### available\_modalities
```python theme={null}
@property
def available_modalities() -> list[str]
```
Returns the input modalities available for this record.
Always includes `"image"`. Includes `"bulk_rna"` when `bulk_rna_path` is set.
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Serialises to a plain dict for YAML output.
A dictionary representation of the WSI record.
#### from\_dict
```python theme={null}
@classmethod
def from_dict(cls, data: dict[str, Any]) -> WSIRecord
```
Constructs from a plain dict (YAML round-trip).
Dictionary with serialised WSI record fields.
A new `WSIRecord` instance.
## PatientRecord
```python theme={null}
@dataclass
class PatientRecord()
```
Groups all WSIs belonging to a single patient.
Unique patient identifier.
Ordered list of WSI records (by timepoint).
Patient-level categorical annotations.
Patient-level clinical metadata.
## Cohort
```python theme={null}
class Cohort()
```
Typed cohort manifest — single source of truth for an experiment.
A cohort is an ordered collection of `WSIRecord` entries grouped by patient. It stores all information needed to reproduce and resume an inference run: file locations, pairing logic, timepoints, per-model outputs, labels, and clinical metadata.
Construction: Use `from_csv`, `from_directories`, or `load` rather than calling the constructor directly.
Initial list of `WSIRecord` entries.
#### from\_csv
```python theme={null}
@classmethod
def from_csv(cls,
path: str | Path,
*,
wsi_dir: str | Path | None = None,
bulk_rna_dir: str | Path | None = None,
columns: dict[str, str] | None = None) -> Cohort
```
Creates a cohort from a user-provided CSV manifest.
Required columns:
* `wsi_id`: WSI identifier (filename stem or full name).
Optional columns:
* `patient_id`: if absent, derived from `wsi_id`.
* `bulk_rna_id`: paired RNA sample identifier.
* `timepoint`: temporal label.
* `wsi_path`: explicit path override.
* `bulk_rna_path`: explicit path override.
* Any other columns are treated as labels if prefixed with `label_` or as metadata otherwise.
When *wsi\_dir* or *bulk\_rna\_dir* are provided, paths are resolved by joining the directory with the corresponding ID (plus matching extension found on disk).
The *columns* dict maps canonical field names to the actual CSV column names. Only the keys that differ from the defaults need to be specified. For example:
```python theme={null}
Cohort.from_csv(
"cohort.csv",
columns={
"wsi_id": "imageid",
"patient_id": "subject_id",
},
)
```
Recognised canonical keys: `wsi_id`, `patient_id`, `bulk_rna_id`, `timepoint`, `wsi_path`, `bulk_rna_path`.
Path to the CSV manifest.
Directory to resolve WSI paths from.
Directory to resolve bulk RNA paths from.
Mapping of canonical field names to actual CSV column names. Unmapped fields fall back to their canonical name.
**Returns**:
A populated `Cohort`.
#### from\_directories
```python theme={null}
@classmethod
def from_directories(
cls,
wsi_dir: str | Path,
bulk_rna_dir: str | Path | None = None,
*,
mask_dir: str | Path | None = None,
patient_id_fn: Callable[[str], str] | None = None) -> Cohort
```
Creates a cohort by auto-matching files from directories.
WSI and bulk RNA files are paired by **exact filename stem** equality (case-sensitive). Patient IDs default to the shared stem. When multiple wsis share the same patient ID, they are assigned sequential timepoints (`t0`, `t1`, ...) in alphabetical order.
Directory containing WSI files.
Directory containing bulk RNA files. When `None`, wsis are registered without RNA.
Directory containing pre-computed masks (PNG files whose stem matches the WSI stem).
Optional callable that maps a filename stem to a patient ID.
A populated `Cohort`.
#### save
```python theme={null}
def save(path: str | Path) -> Path
```
Persists the cohort manifest to a YAML file.
The YAML includes all inputs, per-model outputs, status, labels, and metadata — everything needed to resume.
Destination file path.
The resolved output path.
#### load
```python theme={null}
@classmethod
def load(cls, path: str | Path) -> Cohort
```
Loads a cohort from a previously saved YAML manifest.
Path to the YAML manifest file.
A fully-populated `Cohort`.
#### upsert
```python theme={null}
def upsert(wsi_id: str,
*,
patient_id: str | None = None,
wsi_path: Path | None = None,
bulk_rna_path: Path | None = None,
mask_path: Path | None = None,
timepoint: str | None = None) -> WSIRecord
```
Inserts or updates a WSI record.
If a record with *wsi\_id* exists, updates the non-None fields. Otherwise creates a new record.
WSI identifier.
Patient identifier (defaults to wsi\_id).
Path to WSI file.
Path to bulk RNA file.
Path to tissue mask.
Timepoint label.
The inserted or updated `WSIRecord`.
#### get\_wsi
```python theme={null}
def get_wsi(wsi_id: str) -> WSIRecord | None
```
Returns the record for *wsi\_id*, or `None`.
Unique WSI identifier (typically the file stem).
The matching `WSIRecord`, or `None` if not found.
#### link\_bulk\_rna
```python theme={null}
def link_bulk_rna(bulk_rna_dir: str | Path,
*,
extensions: set[str] | None = None,
separator: str | None = None,
gene_column: str | None = None,
value_column: str | None = None,
strip_version: bool = False) -> int
```
Links bulk RNA files to existing WSI records by stem.
Scans *bulk\_rna\_dir* for recognised RNA files and pairs them with existing records by exact filename-stem equality. Records that already have a `bulk_rna_path` are skipped.
This enables late-binding of bulk RNA data: build a cohort from WSIs first, then call this method to attach RNA when it becomes available.
Directory containing bulk RNA files.
File extensions to match. Defaults to `{".csv", ".tsv"}`.
Column delimiter for parsing. When `None` the separator is inferred from the file extension.
Column name containing gene identifiers. Required (with `value_column`) for long-format files (e.g. GDC/TCGA gene quantification TSVs).
Column name containing expression values to read (e.g. `"tpm_unstranded"`).
When `True`, strips version suffixes from gene identifiers (e.g. `ENSG…00003.15` → `ENSG…00003`).
Number of records that were linked.
#### add\_labels
```python theme={null}
def add_labels(labels: dict[str, dict[str, Any]],
*,
by: str = "patient_id") -> None
```
Adds categorical labels to records.
Mapping of `{identifier: {label_name: value}}`.
Key to match on — `"patient_id"` or `"wsi_id"`.
#### add\_metadata
```python theme={null}
def add_metadata(metadata: dict[str, dict[str, Any]],
*,
by: str = "patient_id") -> None
```
Adds clinical/treatment metadata to records.
Mapping of `{identifier: {field: value}}`.
Key to match on — `"patient_id"` or `"wsi_id"`.
#### add\_labels\_from\_csv
```python theme={null}
def add_labels_from_csv(path: str | Path, *, by: str = "patient_id") -> None
```
Adds labels and metadata from a CSV file.
The CSV must have a column matching *by* (default `patient_id`). Columns prefixed with `label_` are treated as labels; remaining columns as metadata.
Path to the labels CSV.
Join key column name.
#### num\_patients
```python theme={null}
@property
def num_patients() -> int
```
Number of unique patients in the cohort.
#### patients
```python theme={null}
@property
def patients() -> dict[str, PatientRecord]
```
Patient-indexed view of the cohort.
#### get\_patient
```python theme={null}
def get_patient(patient_id: str) -> PatientRecord
```
Returns the `PatientRecord` for a given patient.
Patient identifier string.
The matching `PatientRecord`.
**Raises:**
* `KeyError` — If *patient\_id* is not found.
#### wsi\_ids
```python theme={null}
@property
def wsi_ids() -> list[str]
```
Ordered list of all WSI IDs.
#### patient\_ids
```python theme={null}
@property
def patient_ids() -> list[str]
```
Ordered list of unique patient IDs.
#### wsi\_paths
```python theme={null}
@property
def wsi_paths() -> list[Path | None]
```
Ordered list of WSI paths (may contain `None`).
#### pending
```python theme={null}
def pending(model_name: str, stage: str) -> list[WSIRecord]
```
Returns wsis that have not completed a stage for a model.
Checks whether the output file exists on disk. Records with missing WSI paths are excluded.
Model identifier.
`"embed"` or `"predict"`.
List of `WSIRecord` entries still needing processing.
#### to\_csv
```python theme={null}
def to_csv(path: str | Path) -> Path
```
Writes input columns to a CSV file (labels + metadata).
For full persistence including outputs, use `save`.
Destination file path.
The resolved output path.
#### summary
```python theme={null}
def summary() -> str
```
Returns a human-readable summary of the cohort.
Multi-line summary string.
# multi_model_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/multi-model/multi_model_dataset
Multi-model dataset for joint histology and omics inference.
Wraps a single WSI together with optional bulk RNA data, providing lazy patch access and a PyTorch-compatible `__getitem__` interface.
## MultiModelData
```python theme={null}
class MultiModelData()
```
Wraps a single Whole Slide Image with its extraction plan.
On construction the slide is opened via `WSI` and the extractor is fitted + executed to produce a list of `RegionSpec`. Each spec describes one tile/patch location. Patches are read lazily via `get_patch`.
Path to the WSI file (any format supported by `WSI`).
Optional path to bulk RNA CSV.
A *configured* `TileExtractor`. A fresh `fit_extract` is called for every slide so the same extractor object can be reused across slides.
Optional callable applied to the raw `np.ndarray` patch **before** it is returned. Receives `(H, W, C)` uint8 and should return a transformed array (or tensor).
Resolved slide path.
Resolved path to bulk RNA CSV.
Open reader for the slide.
Extraction plan (one entry per patch).
Stem of the slide filename.
**Example:**
```python theme={null}
wsi = MultiModelData("tissue.svs", extractor)
patch, meta = wsi.get_patch(0)
```
#### get\_patch
```python theme={null}
def get_patch(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Reads a single patch from the slide.
Index into `specs`.
A tuple `(patch, metadata)` where *patch* is an `np.ndarray` of shape `(H, W, C)` (or whatever the transform returns), and *metadata* is a dict with at minimum `source`, `x`, `y`, `width`, `height`, `slide_name`, and `tissue_ratio`.
**Raises:**
* `IndexError` — If *idx* is out of range.
#### close
```python theme={null}
def close() -> None
```
Closes the underlying WSI reader and releases resources.
**Example**:
```pycon theme={null}
>>> wsi.close()
```
# bioptimus.data.omics.bulkrna
Source: https://docs.bioptimus.com/sdk-reference/data/omics/bulkrna
Bulk RNA-seq data reader.
Provides `BulkRNAData` — a lightweight wrapper around a CSV/TSV of gene expression counts for a single patient sample.
**Wide format** (default) — one row per sample, genes as columns:
```text theme={null}
sample_id,gene_1,gene_2,...,gene_N
patient_001,12.3,4.5,...,0.1
```
The first column is treated as the sample identifier (index). All remaining columns are gene expression values. Only the first row is used; additional rows are ignored.
**Long format** — one row per gene, activated by setting `gene_column` and `value_column`:
```text theme={null}
gene_name,raw_count,normalized
gene_1,12.3,0.5
gene_2,4.5,0.2
```
The separator is inferred from the file extension (`.tsv` → tab, otherwise comma) but can be overridden via `separator`.
## BulkRNAData
```python theme={null}
class BulkRNAData()
```
Single-sample bulk RNA-seq counts for PyTorch pipelines.
Reads a CSV or TSV with gene expression counts and stores the values as a contiguous `np.float32` array for zero-copy tensor creation via `counts`.
Supports two layouts:
* **Wide** (default): one row per sample, genes as columns. The first column is the sample identifier (index).
* **Long**: one row per gene, activated by setting `gene_column` and `value_column`.
Path to the CSV or TSV file.
Optional gene list to filter/align columns to. Missing genes are zero-filled, extras are dropped. When `None` all columns are kept.
Optional preprocessing transform applied when accessing `counts`.
Column delimiter. When `None` the separator is inferred from the file extension (`.tsv` → tab, otherwise comma).
Column name that contains gene identifiers. Required for long-format files.
Column name that contains expression values to read. Required for long-format files.
Sample identifier (first row index in wide format, or the file stem in long format).
Ordered list of gene column names.
#### num\_genes
```python theme={null}
@property
def num_genes() -> int
```
Number of gene columns.
#### counts
```python theme={null}
@property
def counts() -> list[float]
```
Gene counts as a list of floats.
#### get\_counts
```python theme={null}
def get_counts(sample_name: str | None = None) -> list[float]
```
Return gene counts as a flat sequence.
When no `omic_transform` is configured, returns a plain `list[float]`. With a transform, returns the transform output (typically an `np.ndarray`).
Ignored (kept for backward compatibility).
Gene expression values in column order.
# tile_spec
Source: https://docs.bioptimus.com/sdk-reference/data/tile_spec
Tile specification dataclass.
Defines `TileSpec`, a pure data structure describing tile geometry (size, stride, resolution, measurement unit). This module lives in the data layer so it can be imported by both `bioptimus.extraction` and `bioptimus.models` without creating circular dependencies.
For backward compatibility, `TileSpec` is re-exported from `types`.
## TileSpec
```python theme={null}
@dataclass(frozen=True)
class TileSpec()
```
Defines the specifications of a tile within a WSI for extraction.
The size of the tile as `(width, height)` or a single integer for both dimensions.
The stride of the tile as `(stride_width, stride_height)` or a single integer for both dimensions.
The resolution at which the tile is defined.
The unit of measurement for the tile dimensions.
**Example**:
```pycon theme={null}
>>> spec = TileSpec(
... size=(256, 256), stride=(128, 128),
... resolution=Resolution(level=0),
... unit=MeasurementUnit.PIXELS,
... )
>>> print(spec.size)
(256, 256)
```
#### size
(width, height)
#### stride
(stride\_width, stride\_height)
#### width
```python theme={null}
@property
def width() -> int
```
Returns the tile width.
#### height
```python theme={null}
@property
def height() -> int
```
Returns the tile height.
#### stride\_width
```python theme={null}
@property
def stride_width() -> int
```
Returns the horizontal stride.
#### stride\_height
```python theme={null}
@property
def stride_height() -> int
```
Returns the vertical stride.
#### scale\_to\_reference
```python theme={null}
def scale_to_reference(downsample_ratio: float, mpp: MPP) -> TileSpec
```
Returns a new TileSpec scaled to reference-level pixels.
Ratio of the reference-level downsample to the target-resolution downsample.
Microns-per-pixel at the target resolution.
A new `TileSpec` in reference-level pixels.
**Raises:**
* `ValueError` — If `unit` is not `PIXELS` or `UM`.
# torch_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/wsi/torch_dataset
PyTorch adapter for WSI tile datasets.
Provides `PytorchTileDataset`, a `Dataset` adapter that wraps `WSIDataset` for use with `DataLoader`. Two sampling modes are supported:
* `"sequential"` — patches in slide-load order (deterministic).
* `"random"` — same patches, but indices are shuffled each epoch.
Both modes visit every patch exactly once per epoch.
The *data* argument accepts three forms:
* A ready-made `WSIDataset`.
* A directory path (`str` or `Path`) — all supported WSI files inside it are discovered and loaded.
* A list of file paths — each path is opened as a slide.
**Example**:
```pycon theme={null}
>>> from bioptimus.data.wsi.torch_dataset import PytorchTileDataset
>>> ds = PytorchTileDataset(wsi_dataset, sampling="sequential")
>>> ds = PytorchTileDataset("slides/", extractor=ext, sampling="random", seed=0)
>>> loader = DataLoader(ds, batch_size=64, num_workers=4)
```
## SamplingMode
```python theme={null}
class SamplingMode(str, Enum)
```
Patch sampling strategy.
Iterate over every patch in slide-load order.
Shuffle the global patch indices each epoch.
**Example**:
```pycon theme={null}
>>> mode = SamplingMode("random")
>>> mode == SamplingMode.RANDOM
True
```
## PytorchTileDataset
```python theme={null}
class PytorchTileDataset(Dataset)
```
PyTorch-compatible dataset over WSI tiles with configurable ordering.
Accepts a `WSIDataset`, a directory path, or a list of file paths. When a path or list is given, *extractor* is required so the slides can be opened and tiled automatically.
`"sequential"` (default) Patches are returned in the order slides were loaded — slide 0 patch 0, slide 0 patch 1, …, slide N patch M.
`"random"` The same set of patches, but the global indices are shuffled. Call `shuffle` between epochs (or at init) to re-randomise.
In both modes `len()` equals the total patch count, and every patch is visited exactly once per full iteration.
One of: - A `WSIDataset` instance (used directly). - A `str` or `Path` pointing to a **directory** — all supported WSI files are discovered and loaded. - A `list` of file paths — each is opened as a slide.
Required when *data* is a path or list of paths. A configured `TileExtractor` used to tile each slide.
`"sequential"` or `"random"` (default `"sequential"`).
Optional callable applied to the `np.ndarray` `(H, W, C)` patch.
Optional RNG seed for reproducible shuffling.
**Raises:**
* `ValueError` — If *data* is a path/list but *extractor* is not provided.
* `FileNotFoundError` — If a directory path contains no supported WSIs.
**Example:**
```python theme={null}
ds = PytorchTileDataset(wsi_dataset, sampling="sequential")
ds = PytorchTileDataset("slides/", extractor=ext, sampling="random", seed=0)
ds = PytorchTileDataset(["a.svs", "b.svs"], extractor=ext)
```
#### shuffle
```python theme={null}
def shuffle(seed: Optional[int] = None) -> None
```
Re-shuffles the global index mapping.
Call this between epochs to get a different ordering. Has no effect when `sampling` is `"sequential"`.
Optional RNG seed for reproducibility.
**Example**:
```pycon theme={null}
>>> ds = PytorchTileDataset(dataset, sampling="random", seed=42)
>>> ds.shuffle(seed=123) # new ordering for next epoch
```
# wsi_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/wsi/wsi_dataset
WSI Data & Dataset Module.
Provides `WSIData` (single-slide wrapper) and `WSIDataset` (multi-slide collection) that bridge the bioptimus I/O + extraction layers into structures ready for feature extraction or training.
**Typical usage:**
```python theme={null}
from bioptimus.data.wsi import WSIData, WSIDataset
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.io.wsi.types import Level, MeasurementUnit
spec = TileSpec(size=(256, 256), stride=(256, 256), resolution=Level(0), unit=MeasurementUnit.PIXELS)
extractor = TileExtractor(tile_spec=spec, mask_threshold=0.5)
dataset = WSIDataset.from_paths(
paths=["slide_1.svs", "slide_2.svs"],
extractor=extractor,
)
# Iterate over all patches (flat indexing across slides)
for i in range(len(dataset)):
patch, meta = dataset[i] # np.ndarray (H, W, C), dict
```
## WSIData
```python theme={null}
class WSIData()
```
Wraps a single Whole Slide Image with its extraction plan.
On construction the slide is opened via `WSI` and the extractor is fitted + executed to produce a list of `RegionSpec`. Each spec describes one tile/patch location. Patches are read lazily via `get_patch`.
Path to the WSI file (any format supported by `WSI`).
A *configured* `TileExtractor`. A fresh `fit_extract` is called for every slide so the same extractor object can be reused across slides.
Optional callable applied to the raw `np.ndarray` patch **before** it is returned. Receives `(H, W, C)` uint8 and should return a transformed array (or tensor).
Resolved slide path.
Open reader for the slide.
Extraction plan (one entry per patch).
Stem of the slide filename.
**Example:**
```python theme={null}
wsi = WSIData("tissue.svs", extractor)
patch, meta = wsi.get_patch(0)
```
#### get\_patch
```python theme={null}
def get_patch(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Reads a single patch from the slide.
Index into `specs`.
A tuple `(patch, metadata)` where *patch* is an `np.ndarray` of shape `(H, W, C)` (or whatever the transform returns), and *metadata* is a dict with at minimum `source`, `x`, `y`, `width`, `height`, `slide_name`, and `tissue_ratio`.
**Raises:**
* `IndexError` — If *idx* is out of range.
#### close
```python theme={null}
def close() -> None
```
Closes the underlying WSI reader and releases resources.
**Example**:
```pycon theme={null}
>>> wsi.close()
```
## WSIDataset
```python theme={null}
class WSIDataset()
```
A collection of `WSIData` wrappers with flat patch indexing.
Supports two access patterns:
* **Slide-level** — `dataset.slides[i]` returns a `WSIData`.
* **Patch-level** (flat) — `dataset[k]` maps a global patch index across all slides and returns `(patch, metadata)`.
The flat indexing uses a cumulative-sum lookup (`O(log N)` via `bisect`) so random access is fast regardless of how many slides are loaded.
Pre-built `WSIData` instances.
**Example:**
```python theme={null}
dataset = WSIDataset.from_paths(paths, extractor)
len(dataset) # total patch count across all slides
patch, meta = dataset[42]
slide = dataset.slides[0]
```
#### from\_paths
```python theme={null}
@classmethod
def from_paths(
cls,
paths: Sequence[Union[str, Path]],
extractor: TileExtractor,
transform: Optional[Callable[[np.ndarray], Any]] = None) -> WSIDataset
```
Creates a dataset by opening and extracting every slide.
Iterable of WSI file paths.
Shared `TileExtractor` (re-fitted per slide).
Optional callable applied to each extracted tile array.
A new `WSIDataset`.
**Example:**
```python theme={null}
dataset = WSIDataset.from_paths(glob.glob("slides/*.svs"), extractor)
```
#### num\_slides
```python theme={null}
@property
def num_slides() -> int
```
Returns the number of loaded slides.
Slide count.
**Example**:
```pycon theme={null}
>>> dataset.num_slides
3
```
#### slide\_patch\_counts
```python theme={null}
def slide_patch_counts() -> Dict[str, int]
```
Returns a mapping of slide names to their patch counts.
Dict\[str, int]: `{slide_name: patch_count}` for every slide.
**Example**:
```pycon theme={null}
>>> dataset.slide_patch_counts()
{'slide_001': 150, 'slide_002': 350}
```
#### close
```python theme={null}
def close() -> None
```
Closes all underlying WSI readers.
**Example**:
```pycon theme={null}
>>> dataset.close()
```
# megatile_extraction
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/megatile_extraction
WSI Mega-Tile Extraction Module.
This module provides the `MegaTileExtractor` class, which groups individual tile positions from a Whole Slide Image into *mega tiles* — fixed- size grids of `rows × cols` tiles. It builds on the tile-level tissue filtering from `TileExtractor` and adds a second filtering stage: only mega tiles where the fraction of tissue-positive tiles falls within a configurable `[min_valid_tiles_ratio, max_valid_tiles_ratio]` band are kept.
The class follows the same **scikit-learn–style** lifecycle as `TileExtractor`:
1. **Configure** — instantiate with a `MegaTileSpec`, optional tissue mask, and threshold.
2. **Fit** — call `fit` with an open `WSIReader`.
3. **Extract** — call `extract` (or the shortcut `fit_extract`). Results are stored in `megatile_regions_` and also returned.
4. **Export** — call `to_csv` or `to_json` to persist the results.
**Usage:**
```pycon theme={null}
>>> from bioptimus.extraction.wsi.megatile_extraction import MegaTileExtractor
>>> from bioptimus.extraction.wsi.types import MegaTileSpec, TileSpec, GridLayout
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> tile = TileSpec(
... size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS,
... )
>>> mega = MegaTileSpec(
... megatile_shape=(5, 5), tile_spec=tile,
... grid_layout=GridLayout.RECTANGULAR,
... min_valid_tiles_ratio=0.5,
... )
>>> extractor = MegaTileExtractor(
... megatile_spec=mega,
... mask=tissue_mask, mask_threshold=0.5,
... )
>>> with SomeWSIReader("path/to/slide.svs") as reader:
... megatiles = extractor.fit_extract(reader)
... extractor.to_csv("output/")
... extractor.to_json("output/")
```
## MegaTileExtractor
```python theme={null}
class MegaTileExtractor(WSIExtractor)
```
Extracts mega tiles (grids of tiles) from a WSI, filtered by tissue.
A *mega tile* is a rectangular or hexagonal grid of individual tiles. Extraction proceeds in two stages:
1. **Tile-level filtering** — a binary tissue mask and `mask_threshold` determine which individual tile positions contain sufficient tissue.
2. **Mega-tile-level filtering** — only mega tiles where the fraction of valid (tissue-positive) tiles falls within `[min_valid_tiles_ratio, max_valid_tiles_ratio]` are retained.
The extractor follows the same lifecycle as `TileExtractor`:
```python theme={null}
extractor = MegaTileExtractor(megatile_spec=spec, mask=mask)
extractor.fit(reader)
megatiles = extractor.extract()
extractor.to_csv("output/")
```
Or via method chaining:
```python theme={null}
megatiles = MegaTileExtractor(megatile_spec=spec).fit_extract(reader)
```
**Configuration attributes** (set at init):
Mega-tile grid shape, per-tile spec, layout, stride, and valid-ratio bounds.
Binary tissue mask.
Per-slide mask generator.
Per-tile tissue fraction threshold.
Whether to store per-tile masks.
Optional cap on returned mega tiles. **Fitted attributes** (populated by `fit` / `extract`):
Bound WSI reader.
Stem of the slide filename.
Extracted mega tiles.
#### fit
```python theme={null}
def fit(source: WSIReader) -> "MegaTileExtractor"
```
Binds the extractor to an open WSI reader.
Stores the reader and slide name. If a `mask_provider` is set, its `generate` method is called to create a per-slide tissue mask.
An open WSI reader.
`self` for method chaining.
**Example**:
```pycon theme={null}
>>> extractor.fit(reader)
>>> extractor.slide_name_
'slide_001'
```
#### extract
```python theme={null}
def extract(source: Optional[WSIReader] = None) -> List[MegaTileRegion]
```
Extracts mega-tile regions from a WSI.
The method operates in five stages:
1. **Reference-level setup** — as in `TileExtractor`.
2. **Mask scaling** — resize tissue mask to the reference level.
3. **Tile-spec scaling** — convert tile dimensions to reference-level pixels.
4. **Per-tile tissue evaluation** — build an integral image and compute the tissue fraction for every candidate tile position.
5. **Mega-tile grouping** — slide a `rows × cols` window (in tile units) across the tile grid, collect per-tile validity, and filter by `[min_valid_tiles_ratio, max_valid_tiles_ratio]`.
Results are stored in `megatile_regions_` **and** returned.
An open WSI reader. When given, the extractor is fitted automatically. Otherwise `fit` must have been called.
Mega-tile regions that pass filtering.
**Raises:**
* `RuntimeError` — If no source is provided and `fit` has not been called.
**Example**:
```pycon theme={null}
>>> megatiles = extractor.fit(reader).extract()
>>> len(megatiles)
12
```
#### fit\_extract
```python theme={null}
def fit_extract(source: WSIReader) -> List[MegaTileRegion]
```
Convenience method: `fit` + `extract` in one call.
An open WSI reader.
Extracted mega-tile regions.
**Example**:
```pycon theme={null}
>>> megatiles = extractor.fit_extract(reader)
>>> len(megatiles)
12
```
#### to\_csv
```python theme={null}
def to_csv(output_dir: Union[str, Path],
regions: Optional[List[MegaTileRegion]] = None) -> Path
```
Saves mega-tile metadata to a CSV file named after the slide.
Each row represents one *tile* within a mega tile. Mega-tile–level columns (`megatile_index`, `megatile_top`, etc.) allow grouping.
Destination directory (created if needed).
Override the stored regions. Defaults to `megatile_regions_`.
Path of the written CSV file.
**Raises:**
* `RuntimeError` — If no regions are available.
**Example**:
```pycon theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.to_csv("output/")
PosixPath('output/slide_001.csv')
```
#### to\_json
```python theme={null}
def to_json(output_dir: Union[str, Path],
regions: Optional[List[MegaTileRegion]] = None,
indent: int = 2) -> Path
```
Saves mega-tile metadata to a JSON file named after the slide.
The output is a JSON array of mega-tile objects, each containing the mega-tile location, grid info, valid ratio, and a nested `tiles` array with per-tile metadata.
Destination directory (created if needed).
Override the stored regions.
JSON pretty-printing indent.
Path of the written JSON file.
**Raises:**
* `RuntimeError` — If no regions are available.
**Example**:
```pycon theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.to_json("output/")
PosixPath('output/slide_001_megatiles.json')
```
# tile_extraction
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/tile_extraction
WSI Tile Extraction Module.
This module provides the `TileExtractor` class, which generates a grid of tile region specifications from a Whole Slide Image (WSI). It uses a binary tissue mask to filter out background tiles and supports resolution scaling via `Level`, `Magnification`, or `MPP`.
The class follows a **scikit-learn–style** API:
1. **Configure** — instantiate with a `TileSpec`, optional tissue mask, and threshold.
2. **Fit** — call `fit` with an open `WSIReader` to bind the extractor to a specific slide.
3. **Extract** — call `extract` (or the shortcut `fit_extract`) to compute tile positions. Results are stored in `region_specs_` and also returned.
4. **Export** — call `save`, `to_csv`, or `to_json` to persist the results. These methods use the fitted state so you do not need to pass the reader or regions again.
**Usage:**
```pycon theme={null}
>>> from bioptimus.extraction.wsi.tile_extraction import TileExtractor
>>> from bioptimus.extraction.wsi.types import TileSpec
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> spec = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> extractor = TileExtractor(tile_spec=spec, mask=tissue_mask, mask_threshold=0.5)
>>> with SomeWSIReader("path/to/slide.svs") as reader:
... extractor.fit(reader).extract()
... extractor.to_csv("output/") # writes output/slide.csv
... extractor.to_json("output/") # writes output/slide.json
... extractor.save("tiles/")
```
## TileExtractor
```python theme={null}
class TileExtractor(WSIExtractor)
```
Extracts a grid of tile regions from a WSI, filtered by a tissue mask.
The extractor follows a **scikit-learn–style** lifecycle:
```python theme={null}
extractor = TileExtractor(tile_spec=spec, mask=mask)
extractor.fit(reader) # bind to a slide
tiles = extractor.extract() # compute tile positions
extractor.save("out/") # write tile images
extractor.to_csv("tiles.csv") # export metadata
```
Or more concisely via method chaining:
```python theme={null}
tiles = TileExtractor(tile_spec=spec, mask=mask).fit_extract(reader)
```
**Configuration attributes** (set at init, do not change after fitting):
Tile size, stride, resolution, and measurement unit.
Binary tissue mask (non-zero = tissue).
Provider that generates a per-slide mask during `fit`. When set, takes precedence over the static *mask* array.
Minimum mean mask value for a tile to be kept.
Whether to include the per-tile mask array in each `RegionMaskSpec`.
**Fitted attributes** (populated by `fit` / `extract`):
The WSI reader bound by `fit`.
Stem of the slide filename (e.g. `"slide_001"`).
Tile regions produced by `extract`. Empty until extraction is run.
#### fit
```python theme={null}
def fit(source: WSIReader) -> "TileExtractor"
```
Binds the extractor to an open WSI reader.
Stores the reader and slide name so that subsequent calls to `extract`, `save`, `to_csv`, and `to_json` do not need the reader passed again.
If a `mask_provider` is set, its `generate` method is called here so that each slide receives its own tissue mask automatically.
Following scikit-learn convention, fitted attributes are suffixed with an underscore (`source_`, `slide_name_`, `region_specs_`).
An **open** WSI reader for the target slide.
`self`, to allow method chaining (e.g. `extractor.fit(reader).extract()`).
**Example:**
```pycon theme={null}
>>> extractor.fit(reader)
>>> extractor.slide_name_
'slide_001'
```
#### extract
```python theme={null}
def extract(source: Optional[WSIReader] = None) -> List[RegionSpec]
```
Extracts tile region specs from a WSI.
If *source* is provided it is used directly (and the extractor is automatically `fit` to it). If omitted, the previously fitted source is used — call `fit` first in that case.
The method operates in four stages:
1. **Reference-level setup** — identifies the lowest-resolution pyramid level and computes the relative downsample between it and the target resolution.
2. **Mask scaling** — resizes the tissue mask (or creates an all-tissue mask) to match the reference level's bounded dimensions.
3. **Tile-spec scaling** — converts tile size and stride from the target resolution (pixels or microns) into reference-level pixel coordinates.
4. **Grid walk** — iterates over the reference-level grid, filters by `mask_threshold`, and maps surviving tile coordinates back to the target resolution.
Results are stored in `region_specs_` **and** returned.
An open WSI reader. When given, the extractor is fitted to it automatically. When `None` (default), the extractor must already be fitted.
Region specifications for every tile that passes the tissue-mask threshold.
**Raises:**
* `RuntimeError` — If no *source* is provided and `fit` has not been called.
* `ValueError` — If `tile_spec.unit` is not `MeasurementUnit.PIXELS` or `MeasurementUnit.UM`.
**Example:**
```pycon theme={null}
>>> extractor.fit(reader)
>>> tiles = extractor.extract()
>>> tiles[0].location
RegionLocation(top=1024, left=512)
```
#### fit\_extract
```python theme={null}
def fit_extract(source: WSIReader) -> List[RegionSpec]
```
Convenience method: `fit` + `extract` in one call.
An open WSI reader.
Extracted tile region specifications.
**Example:**
```pycon theme={null}
>>> tiles = extractor.fit_extract(reader)
>>> len(tiles)
42
```
#### to\_csv
```python theme={null}
def to_csv(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None) -> Path
```
Saves region metadata to a CSV file named after the slide.
The file is written as `/.csv`, reusing the `slide_name_` captured during `fit`.
Uses the internally stored `region_specs_` by default. Pass *regions* explicitly to override.
Each row contains the **slide name**, location, shape, resolution, and tissue ratio. Mask arrays are **not** included — use `save` to persist tile images instead.
Destination directory. Created (including parents) if it does not exist.
Region specifications to serialise. Defaults to `region_specs_`.
The resolved path of the written CSV file (e.g. `output/slide_001.csv`).
**Raises:**
* `RuntimeError` — If the extractor has not been fitted or no regions are available.
**Example:**
```pycon theme={null}
>>> extractor.fit_extract(reader) # slide file is 'abc.svs'
>>> extractor.to_csv("output/")
PosixPath('output/abc.csv')
```
#### to\_json
```python theme={null}
def to_json(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None,
indent: int = 2) -> Path
```
Saves region metadata to a JSON file named after the slide.
The file is written as `/.json`, reusing the `slide_name_` captured during `fit`.
Uses the internally stored `region_specs_` by default. Pass *regions* explicitly to override.
The output is a JSON array of objects, one per `RegionSpec`, containing the slide name, location, shape, resolution, and tissue ratio. Mask arrays are **not** included.
Destination directory. Created (including parents) if it does not exist.
Region specifications to serialise. Defaults to `region_specs_`.
Number of spaces for pretty-printing. Defaults to `2`. Set to `0` or `None` for compact output.
The resolved path of the written JSON file (e.g. `output/slide_001.json`).
**Raises:**
* `RuntimeError` — If the extractor has not been fitted or no regions are available.
**Example:**
```pycon theme={null}
>>> extractor.fit_extract(reader) # slide file is 'abc.svs'
>>> extractor.to_json("output/")
PosixPath('output/abc.json')
```
#### save
```python theme={null}
def save(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None,
workers: int = 4,
image_format: str = "png") -> None
```
Saves extracted tile images to disk using multi-threaded I/O.
Uses the fitted `source_` and internally stored `region_specs_` by default. Pass *regions* explicitly to override the region list.
Each tile is read from the WSI via `read_region` and written to *output\_dir* with a metadata-rich filename:
```text theme={null}
{slide_stem}_x{left}_y{top}_w{width}_h{height}_{res}_mask_r{ratio}.{fmt}
```
Where `{res}` encodes the resolution (e.g. `l_0`, `mpp_0.25`, `mag_40.0x`) and `{ratio}` is the tissue fraction rounded to two decimal places.
The tile coordinates produced by `extract` are **absolute** slide coordinates, so this method reads regions with `bounded=False` to avoid double-offsetting.
Directory where tile images will be written. Created (including parents) if it does not exist.
Region specifications to save. Defaults to `region_specs_`.
Maximum number of threads for parallel I/O. Defaults to `4`.
Image file extension / format accepted by `PIL.Image.save` (e.g. `"png"`, `"jpeg"`, `"tiff"`). Defaults to `"png"`.
**Returns**:
None
**Raises:**
* `RuntimeError` — If the extractor has not been fitted, no regions are available, or one or more tiles fail to save.
**Example:**
```pycon theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.save("output/tiles")
```
# types
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/types
Tile and mega-tile specification types for WSI extraction.
Defines the `TileSpec` and `MegaTileSpec` dataclasses as well as the `GridLayout` enum and the `MegaTileRegion` container.
`TileSpec` Encapsulates tile size, stride, target resolution, and measurement unit. Used by `TileExtractor`.
`GridLayout` Enum for rectangular vs. hexagonal tiling grids.
`MegaTileSpec` Describes a *mega tile* — a grid of smaller tiles — including grid dimensions, per-tile specification, layout, stride, and valid-tile-ratio bounds. Used by `MegaTileExtractor`.
`MegaTileRegion` Immutable container returned by the mega-tile extractor. Holds the top-left location, grid shape, per-tile `RegionSpec` list, tile-availability mask, and aggregate tissue statistics.
**Example**:
```pycon theme={null}
>>> from bioptimus.extraction.wsi.types import TileSpec, MegaTileSpec, GridLayout
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> tile = TileSpec(
... size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS,
... )
>>> mega = MegaTileSpec(
... megatile_shape=(5, 5), tile_spec=tile,
... grid_layout=GridLayout.RECTANGULAR,
... )
>>> mega.num_tiles
25
```
## GridLayout
```python theme={null}
class GridLayout(str, Enum)
```
Layout strategy for arranging tiles within a mega tile.
Standard row-major rectangular grid.
Offset hex grid — odd rows are shifted right by half a tile width, producing a honeycomb pattern.
**Example**:
```pycon theme={null}
>>> GridLayout.RECTANGULAR.value
'RECTANGULAR'
```
## MegaTileSpec
```python theme={null}
@dataclass(frozen=True)
class MegaTileSpec()
```
Defines the specification of a mega tile — a grid of smaller tiles.
A mega tile groups `rows × cols` individual tiles into a single logical region that can be extracted from a WSI. The per-tile geometry is delegated to `TileSpec`.
Number of tiles as `(rows, cols)`.
Size, stride, resolution, and unit for each individual tile.
Rectangular or hexagonal arrangement. Defaults to `RECTANGULAR`.
Stride expressed in **number of tiles** as `(stride_rows, stride_cols)`. `None` defaults to `megatile_shape` (no overlap).
Minimum fraction of tiles that must pass the tissue-mask threshold for the mega tile to be kept. Defaults to `0.5`.
Optional upper bound on the valid-tile fraction. `None` means no upper limit.
**Example**:
```pycon theme={null}
>>> tile = TileSpec(
... size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS,
... )
>>> mega = MegaTileSpec(megatile_shape=(5, 5), tile_spec=tile)
>>> mega.rows, mega.cols
(5, 5)
>>> mega.num_tiles
25
```
#### megatile\_shape
(rows, cols) in number of tiles
#### megatile\_stride
(stride\_rows, stride\_cols)
#### rows
```python theme={null}
@property
def rows() -> int
```
Number of tile rows in the mega tile.
#### cols
```python theme={null}
@property
def cols() -> int
```
Number of tile columns in the mega tile.
#### num\_tiles
```python theme={null}
@property
def num_tiles() -> int
```
Total number of tiles in the mega tile grid.
#### effective\_stride
```python theme={null}
@property
def effective_stride() -> tuple[int, int]
```
Returns the stride, defaulting to `megatile_shape` when not set.
tuple\[int, int]: `(stride_rows, stride_cols)` in tiles.
## MegaTileRegion
```python theme={null}
@dataclass(frozen=True)
class MegaTileRegion()
```
A group of tiles forming one extracted mega tile.
Returned by `MegaTileExtractor`.
Absolute slide coordinates of the top-left corner of the mega tile.
The specification that produced this mega tile (grid shape, tile spec, layout, stride, and valid-ratio bounds).
Per-tile region specifications, ordered row-major (`tile_specs[row * cols + col]`).
Boolean array of shape `(rows, cols)` indicating which tiles passed the tissue threshold.
Fraction of tiles that are valid (tissue).
**Example**:
```pycon theme={null}
>>> mega.grid_shape
(5, 5)
>>> mega.valid_ratio
0.72
>>> mega.tile_availability.sum()
18
```
#### grid\_shape
```python theme={null}
@property
def grid_shape() -> tuple[int, int]
```
`(rows, cols)` — number of tiles in each direction.
#### grid\_layout
```python theme={null}
@property
def grid_layout() -> GridLayout
```
Layout used during extraction.
#### rows
```python theme={null}
@property
def rows() -> int
```
Number of tile rows.
#### cols
```python theme={null}
@property
def cols() -> int
```
Number of tile columns.
#### num\_valid\_tiles
```python theme={null}
@property
def num_valid_tiles() -> int
```
Count of tiles that passed the tissue threshold.
#### num\_tiles
```python theme={null}
@property
def num_tiles() -> int
```
Total tile slots in the grid.
# schemas
Source: https://docs.bioptimus.com/sdk-reference/inference/schemas
Request and response schemas for model inference.
These Pydantic models define the JSON payloads exchanged between clients and a deployed model endpoint (e.g. SageMaker or a local server). Each request/response pair corresponds to a single tile.
## ModelRequest
```python theme={null}
class ModelRequest(BaseModel)
```
Payload sent to the model endpoint for a single tile.
The tile image is base64-encoded for JSON transport. Bulk RNA counts can optionally be included for multimodal models.
Base64-encoded tile image (PNG).
Optional bulk RNA counts vector.
Stem of the source slide filename.
Tile x-coordinate in the slide.
Tile y-coordinate in the slide.
Tile width in pixels.
Tile height in pixels.
Fraction of tissue in the tile.
Index of this tile in the extraction plan.
#### image
```python theme={null}
@property
def image() -> Image.Image
```
Decodes and returns the tile image as PIL.
## ModelResponse
```python theme={null}
class ModelResponse(BaseModel)
```
Payload returned by the model endpoint for a single tile.
Carries the model output together with the full tile metadata from the originating request, so that downstream writers can persist everything without needing a separate schema.
Model output vector for this tile.
Stem of the source slide filename.
Tile x-coordinate in the slide.
Tile y-coordinate in the slide.
Tile width in pixels.
Tile height in pixels.
Fraction of tissue in the tile.
Index of this tile in the extraction plan.
Extraction resolution (MPP) if available.
## SageMakerRequest
```python theme={null}
class SageMakerRequest(ModelRequest)
```
Extended request for the SageMaker `/invocations` endpoint.
Adds a `model_name` field so the server can dispatch to the correct model and a `mode` field to select the scheduler (`"prediction"` or `"embedding"`).
# writers
Source: https://docs.bioptimus.com/sdk-reference/inference/writers
Streaming writers for persisting tile-level predictions.
Provides a `PredictionWriter` protocol and three concrete implementations (`ZarrWriter`, `HDF5Writer`, `NPZWriter`) that save per-tile outputs and metadata to disk in a memory-efficient way.
Zarr and HDF5 writers stream results row-by-row so that the full output tensor never needs to reside in memory. The NPZ writer accumulates results in Python lists and flushes on close.
## OutputFormat
```python theme={null}
class OutputFormat(str, enum.Enum)
```
Supported on-disk formats for prediction output.
## PredictionWriter
```python theme={null}
class PredictionWriter(ABC)
```
Abstract base for streaming prediction writers.
#### open
```python theme={null}
@abstractmethod
def open(num_tiles: int, embedding_dim: int) -> None
```
Pre-allocates storage once the embedding dim is known.
Total number of tiles to be written.
Dimensionality of each prediction vector.
#### write
```python theme={null}
@abstractmethod
def write(response: ModelResponse) -> None
```
Writes a single tile's model response.
The model response to persist.
#### set\_thumbnail
```python theme={null}
@abstractmethod
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail image.
RGB image array of shape `(H, W, 3)`.
#### set\_metadata
```python theme={null}
@abstractmethod
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores slide-level metadata alongside the arrays.
Key-value pairs (slide\_name, tile\_size, etc.).
#### set\_tissue\_mask
```python theme={null}
@abstractmethod
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask used during extraction.
Binary mask array of shape `(H, W)`, dtype `uint8`.
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores input and output gene name lists.
Default implementation is a no-op. Subclasses that support gene annotation should override this.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
#### close
```python theme={null}
@abstractmethod
def close() -> None
```
Flushes and finalises the output file.
## ZarrWriter
```python theme={null}
class ZarrWriter(PredictionWriter)
```
Streams tile predictions into a Zarr directory store.
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Pre-allocates Zarr datasets for outputs, coordinates, and tissue ratios.
Total number of tiles to be written.
Dimensionality of each prediction vector.
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the Zarr datasets.
The model response to persist.
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail as a Zarr dataset.
RGB image array of shape `(H, W, 3)`.
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata as Zarr root attributes.
Key-value pairs (slide\_name, tile\_size, etc.).
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask as a Zarr dataset.
Binary mask array of shape `(H, W)`, dtype `uint8`.
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name arrays as Zarr string datasets.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
#### close
```python theme={null}
def close() -> None
```
Logs output path (Zarr stores are flushed on write).
## HDF5Writer
```python theme={null}
class HDF5Writer(PredictionWriter)
```
Streams tile predictions into an HDF5 file.
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Opens the HDF5 file and pre-allocates datasets.
Total number of tiles to be written.
Dimensionality of each prediction vector.
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the HDF5 datasets.
The model response to persist.
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail as an HDF5 dataset.
RGB image array of shape `(H, W, 3)`.
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata as HDF5 file-level attributes.
Key-value pairs (slide\_name, tile\_size, etc.).
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask as an HDF5 dataset.
Binary mask array of shape `(H, W)`, dtype `uint8`.
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name arrays as HDF5 string datasets.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
#### close
```python theme={null}
def close() -> None
```
Closes the HDF5 file handle and flushes to disk.
## NPZWriter
```python theme={null}
class NPZWriter(PredictionWriter)
```
Accumulates tile predictions in memory, saves as compressed npz.
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Allocates in-memory arrays for accumulating results.
Total number of tiles to be written.
Dimensionality of each prediction vector.
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the in-memory arrays.
The model response to persist.
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores the slide thumbnail for later NPZ serialization.
RGB image array of shape `(H, W, 3)`.
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata for later NPZ serialization.
Key-value pairs (slide\_name, tile\_size, etc.).
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask for later NPZ serialization.
Binary mask array of shape `(H, W)`, dtype `uint8`.
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name lists for later NPZ serialization.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
#### close
```python theme={null}
def close() -> None
```
Flushes all accumulated arrays to a compressed `.npz` file.
#### create\_writer
```python theme={null}
def create_writer(fmt: OutputFormat, path: Path) -> PredictionWriter
```
Creates a writer for the requested format.
Output format.
Destination file or directory path.
An initialised (but not yet opened) `PredictionWriter`.
# bioptimus.io.wsi.factory
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/factory
Factory for WSI reader instantiation with automatic backend selection.
Provides the `WSI` convenience class that selects the best available backend (CuCIM, OpenSlide, TiffSlide) based on the file extension and installed libraries.
#### get\_extension
```python theme={null}
def get_extension(path: str) -> str
```
Extract and normalize file extension from a path.
File path or filename string.
Lowercase file extension including the leading dot.
## WSI
```python theme={null}
class WSI()
```
Factory class for creating WSI readers with automatic backend selection.
Usage:
```pycon theme={null}
>>> reader = WSI("slide.svs", backend=WSI.Backend.AUTO)
>>> reader = WSI("slide.svs", backend=WSI.Backend.OPENSLIDE)
```
## Backend
```python theme={null}
class Backend(str, Enum)
```
Supported backend identifiers and the AUTO selection flag.
#### list
```python theme={null}
@classmethod
def list(cls) -> List[str]
```
Return list of valid backend string values (excluding AUTO).
#### supported\_extensions
```python theme={null}
@classmethod
def supported_extensions(cls) -> List[str]
```
Get list of all supported file extensions.
List of extension strings (e.g. `[".svs", ".tiff", ...]`).
#### available\_backends
```python theme={null}
@classmethod
def available_backends(cls) -> Dict[str, bool]
```
Get availability status of all physical backends.
Mapping of backend name to availability boolean.
# bioptimus.io.wsi
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/index
WSI (Whole Slide Image) module for reading digital pathology slides.
WSI (Whole Slide Image) module for reading digital pathology slides.
This module provides a unified interface for reading WSI files across different backends (OpenSlide, cuCIM, TiffFile) with automatic backend selection.
Backend implementations live in the private `_backends` package and should not be imported directly. Use `WSI` to obtain a reader.
# bioptimus.io.wsi.interface
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/interface
WSI Reader Base Module.
This module provides the `WSIReader` abstract base class, which standardizes access to Whole Slide Images (WSI). It handles coordinate translations, resolution scaling (Level, Magnification, MPP), and tissue-aware cropping.
Typical usage example: with OpenSlideReader("path/to/slide.svs") as reader:
# Get tissue-only dimensions
dims = reader.dimensions(bounded=True)
# Read a region relative to the slide origin
region = reader.read\_region((0, 0), (512, 512), Level(0), bounded=False)
## WSIReader
```python theme={null}
class WSIReader(ABC)
```
Abstract Base Class for all WSI Backends.
Standardizes the interface for reading digital pathology slides across different vendors. It manages metadata, pyramid levels, and tissue bounding box transformations.
Path object pointing to the slide file.
Metadata properties extracted from the slide.
#### open\_slide
```python theme={null}
@abstractmethod
def open_slide() -> None
```
Opens the file handle for the slide.
**Raises:**
* `NotImplementedError` — If not implemented by subclass.
#### level\_count
```python theme={null}
@property
@abstractmethod
def level_count() -> int
```
Number of pyramid levels available in the slide.
Number of pyramid levels.
**Raises:**
* `NotImplementedError` — If not implemented by subclass.
#### mpp
```python theme={null}
@property
def mpp() -> Union[MPP, None]
```
Native microns per pixel (MPP) value.
The MPP value if found in metadata, else None.
**Example**:
```pycon theme={null}
>>> reader.mpp
0.25
```
#### downsample\_dimensions
```python theme={null}
def downsample_dimensions(downsample_factor: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Calculates slide dimensions at a specific downsample factor.
The ratio to scale down by.
Whether to return tissue bounds or full slide size.
Union\[WSIDims, WSIBounds]: Scaled slide dimensions or tissue bounds.
**Example**:
```pycon theme={null}
>>> reader.downsample_dimensions(4.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.downsample_dimensions(4.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
#### level\_dimensions
```python theme={null}
def level_dimensions(level: Level,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns the dimensions of a specific pyramid level.
The target pyramid level (0 is highest resolution).
If True, returns tissue area at that level.
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified level.
**Raises:**
* `ValueError` — If level is out of range.
**Example**:
```pycon theme={null}
>>> reader.level_dimensions(Level(2), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.level_dimensions(Level(2), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
#### magnification\_dimensions
```python theme={null}
def magnification_dimensions(
magnification: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific optical magnification.
Target magnification (e.g., 40x, 20x, 10x).
If True, returns tissue area at that magnification.
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified magnification.
**Example**:
```pycon theme={null}
>>> reader.magnification_dimensions(10.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.magnification_dimensions(10.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
#### mpp\_dimensions
```python theme={null}
def mpp_dimensions(mpp: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific MPP (microns per pixel).
Target microns per pixel.
If True, returns tissue area at that MPP.
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified MPP.
**Example**:
```pycon theme={null}
>>> reader.mpp_dimensions(1.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.mpp_dimensions(1.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: bool = True,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns full or bounded slide dimensions at Level 0.
Acts as the `bounded` toggle when a bool.
If True, returns tissue bounds.
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: Level,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a pyramid level.
Target pyramid level.
If True, returns tissue bounds.
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: Magnification,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a magnification.
Target optical magnification.
If True, returns tissue bounds.
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: MPP,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a target MPP.
Target microns per pixel.
If True, returns tissue bounds.
#### dimensions
```python theme={null}
def dimensions(resolution: Union[Resolution, bool] = True,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific resolution.
Desired resolution (Level, Magnification, or MPP).
If True, returns tissue area at that resolution.
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified resolution.
**Example**:
```pycon theme={null}
>>> reader.dimensions(Resolution(level=2), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(level=2), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
```pycon theme={null}
>>> reader.dimensions(Resolution(magnification=10.0), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(magnification=10.0), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
```pycon theme={null}
>>> reader.dimensions(Resolution(mpp=1.0), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(mpp=1.0), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
#### get\_downsample\_factor
```python theme={null}
def get_downsample_factor(resolution: Resolution) -> float
```
Calculates the downsample factor for a given resolution.
The target resolution (Level, Magnification, or MPP).
Downsample factor for the given resolution.
**Example**:
```pycon theme={null}
>>> reader.get_downsample_factor(Resolution(level=2))
4.0
>>> reader.get_downsample_factor(Resolution(magnification=10.0))
4.0
>>> reader.get_downsample_factor(Resolution(mpp=1.0))
4.0
```
#### level\_downsample
```python theme={null}
@abstractmethod
def level_downsample(level: int) -> float
```
Returns downsample factor for a pyramid level.
Pyramid level index.
Downsample factor for the given level.
**Raises:**
* `NotImplementedError` — If not implemented by subclass.
#### mpp\_downsample
```python theme={null}
def mpp_downsample(mpp: float) -> float
```
Calculates downsample needed for target MPP.
Target microns per pixel.
Downsample factor for the target MPP.
**Raises:**
* `ValueError` — If metadata is missing or target is too high-res.
**Example**:
```pycon theme={null}
>>> reader.mpp_downsample(1.0)
4.0
```
#### magnification\_downsample
```python theme={null}
def magnification_downsample(magnification: float) -> float
```
Calculates downsample needed for target magnification.
Target magnification power.
Downsample factor for the target magnification.
**Raises:**
* `ValueError` — If metadata is missing or target is out of range.
**Example**:
```pycon theme={null}
>>> reader.magnification_downsample(10.0)
4.0
```
#### props
```python theme={null}
@property
def props() -> WSIProps
```
Slide metadata properties.
Metadata extracted from the slide.
**Example**:
```pycon theme={null}
>>> reader.props.OBJECTIVE_POWER
40
```
#### get\_best\_level\_for\_downsample
```python theme={null}
def get_best_level_for_downsample(downsample: float) -> int
```
Finds highest pyramid level smaller than target downsample factor.
Target scaling factor.
Best pyramid level index for the downsample.
**Raises:**
* `ValueError` — If downsample is too small.
**Example**:
```pycon theme={null}
>>> reader.get_best_level_for_downsample(4.0)
2
```
#### get\_downsample\_scaling\_and\_level
```python theme={null}
def get_downsample_scaling_and_level(
resolution: Resolution) -> tuple[float, float, int]
```
Determine the downsample factor, scaling factor, and best pyramid level for a given resolution.
Desired resolution of the slide. Can be one of: - Level: integer pyramid level - Magnification: desired optical magnification - MPP: microns per pixel
(downsample\_factor (float), scaling\_factor (float), best\_level (int)) - downsample\_factor: Factor by which the base level is downsampled. - scaling\_factor: Additional scaling needed at the chosen level. - best\_level: Pyramid level that best matches the resolution.
**Raises:**
* `ValueError` — If required slide properties (objective power or MPP) are unavailable.
* `TypeError` — If the resolution type is unsupported.
**Example**:
```pycon theme={null}
>>> reader.get_downsample_scaling_and_level(Level(2))
(4.0, 1.0, 2)
```
#### read\_region
```python theme={null}
def read_region(location: tuple[int, int],
size: tuple[int, int],
resolution: Resolution,
measurement_unit: MeasurementUnit = MeasurementUnit.PIXELS,
bounded: bool = True) -> Region
```
Reads a region from the slide.
(x, y) at Level 0.
(width, height) at target resolution.
Resolution level/mpp/magnification.
Unit for the region shape (default pixels).
If True, (0,0) is tissue top-left. Else slide top-left.
Object containing the image and metadata for the requested area.
**Example**:
```pycon theme={null}
>>> region = reader.read_region((0, 0), (512, 512), Level(0), bounded=False)
>>> region = reader.read_region((0, 0), (512, 512), Level(0), bounded=True)
```
#### is\_supported\_file
```python theme={null}
@abstractmethod
def is_supported_file() -> bool
```
Checks if backend supports this file extension.
True if supported, False otherwise.
**Raises:**
* `NotImplementedError` — If not implemented by subclass.
#### close
```python theme={null}
@abstractmethod
def close() -> None
```
Closes slide file handle.
**Returns**:
None
**Raises:**
* `NotImplementedError` — If not implemented by subclass.
#### get\_thumbnail
```python theme={null}
def get_thumbnail(size: tuple[int, int], bounded: bool = True) -> Image.Image
```
Generates a slide thumbnail.
Max (width, height) for result.
If True, crop to tissue. Else show full slide (incl. white space).
RGB thumbnail image.
**Example**:
```pycon theme={null}
>>> reader.get_thumbnail((1024, 1024), bounded=False)
>>> reader.get_thumbnail((1024, 1024), bounded=True)
```
# bioptimus.io.wsi.metadata
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/metadata
WSI Metadata Properties Module.
WSI Metadata Properties Module.
This module defines the schema and container for metadata extracted from Whole Slide Images (WSI). It provides a unified structure for handling physical, optical, and structural properties across different scanner vendors and file formats.
## WSIProps
```python theme={null}
@dataclass(frozen=True)
class WSIProps()
```
Whole Slide Image (WSI) properties container.
This class stores essential metadata required for WSI processing pipelines. It supports attribute access and dictionary-style key-based lookup with an automated fallback to raw vendor metadata.
Full slide width and height.
The tissue-containing region of the slide.
The number of resolution levels in the pyramid.
Microns per pixel in the X dimension.
Microns per pixel in the Y dimension.
The magnification power of the objective lens.
The name of the scanner vendor or backend.
The filesystem path to the slide file.
A dictionary containing all unprocessed metadata from the slide.
**Example**:
```pycon theme={null}
>>> dims = WSIDims(50000, 30000)
>>> props = WSIProps(DIMENSIONS=dims, VENDOR="Hamamatsu")
>>> print(props.DIMENSIONS.width)
50000
>>> print(props["VENDOR"])
'Hamamatsu'
```
# types
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/types
Core types for representing resolutions and regions in Whole Slide Images (WSIs).
This module defines lightweight, immutable classes for commonly used WSI attributes such as pyramid levels, physical pixel size (MPP), and objective magnification.
**Classes:**
* `Level` — Pyramid level index (0 = highest resolution).
* `MPP` — Microns per pixel (µm/pixel).
* `Magnification` — Objective power (e.g., 20x).
* `WSIDims` — Width and height dimensions.
* `WSIBounds` — A bounding box for tissue regions.
* `MeasurementUnit` — Enum for pixels or microns.
* `RegionLocation` — X/Y coordinates of a region.
* `RegionShape` — Dimensions and units of a region.
* `RegionSpec` — Full specification (Location + Shape + Resolution).
* `Region` — Container for PIL image data and its RegionSpec.
## Level
```python theme={null}
class Level(int)
```
Represents a pyramid level index in a WSI.
Level 0 is the highest resolution. Higher indices are lower resolutions.
The pyramid level index (0-based).
**Raises:**
* `ValueError` — If `index` is negative.
**Example**:
```pycon theme={null}
>>> level = Level(0) # highest resolution
>>> int(level)
0
>>> level
Level 0
>>> Level(-1)
Traceback (most recent call last):
...
ValueError: Level index must be >= 0
```
**Notes**:
* Level 0 is the highest resolution.
* Higher indices correspond to lower resolution levels in the pyramid.
* This class stores the level index as a primitive integer and is immutable.
## MPP
```python theme={null}
class MPP(float)
```
A positive float representing the physical size of a pixel in microns per pixel (µm/pixel) in a whole slide image (WSI).
MPP values indicate the real-world size of one image pixel and must be strictly greater than 0.
The pixel size in microns to represent as MPP.
**Raises:**
* `ValueError` — If `value` is not positive.
**Example**:
```pycon theme={null}
>>> mpp = MPP(0.25) # 0.25 µm/pixel
>>> float(mpp)
0.25
>>> MPP(-1)
Traceback (most recent call last):
...
ValueError: MPP must be positive, got -1
```
## Magnification
```python theme={null}
class Magnification(float)
```
Objective power of a microscope used to acquire a whole slide image (WSI).
Magnification indicates the target resolution (e.g., 20x, 40x). It is always stored as a positive float.
Typical magnifications:
* 10x: low-resolution overview
* 20x: standard diagnostic resolution
* 40x: high-resolution for detailed analysis
The objective power.
**Raises:**
* `ValueError` — If `value` is not positive.
**Example**:
```pycon theme={null}
>>> mag = Magnification(40) # 40x objective
>>> float(mag)
40.0
>>> mag
40.00x
>>> Magnification(20.0)
20.00x
>>> Magnification(-10)
Traceback (most recent call last):
...
ValueError: Magnification must be positive
```
## WSIDims
```python theme={null}
class WSIDims(NamedTuple)
```
Dimensions of a slide or level at full resolution.
The width of the image in pixels.
The height of the image in pixels.
**Example**:
```pycon theme={null}
>>> dims = WSIDims(width=10000, height=20000)
>>> print(dims.width)
10000
```
## WSIBounds
```python theme={null}
class WSIBounds(NamedTuple)
```
The non-empty region (Bounding Box) of the slide in pixels.
This represents the "tissue-containing" area of a slide, allowing users to ignore large background areas.
The X coordinate of the top-left corner.
The Y coordinate of the top-left corner.
The width of the bounding box.
The height of the bounding box.
**Example**:
```pycon theme={null}
>>> bounds = WSIBounds(x=100, y=100, width=500, height=500)
>>> print(bounds.dims)
WSIDims(width=500, height=500)
```
#### dims
```python theme={null}
@property
def dims() -> WSIDims
```
Extracts width and height as a WSIDims object.
The dimensions of the bounding box.
**Example**:
```pycon theme={null}
>>> bounds = WSIBounds(x=10, y=20, width=100, height=200)
>>> bounds.dims
WSIDims(width=100, height=200)
```
## MeasurementUnit
```python theme={null}
class MeasurementUnit(str, Enum)
```
Enumeration of measurement units for WSI tiles and regions.
#### has\_value
```python theme={null}
@classmethod
def has_value(cls, value: str) -> bool
```
Checks if a string is a valid MeasurementUnit.
The string to check.
True if valid, False otherwise.
**Example**:
```pycon theme={null}
>>> MeasurementUnit.has_value("PIXELS")
True
>>> MeasurementUnit.has_value("INCHES")
False
```
## RegionLocation
```python theme={null}
@dataclass(frozen=True)
class RegionLocation()
```
Defines the location of a region within a WSI.
The top coordinate of the top-left corner of the region.
The left coordinate of the top-left corner of the region.
The width of the region in pixels.
The height of the region in pixels.
**Example**:
```pycon theme={null}
>>> location = RegionLocation(top=100, left=200)
>>> print(location.top)
100
```
#### x
```python theme={null}
@property
def x() -> int
```
Alias for left coordinate.
#### y
```python theme={null}
@property
def y() -> int
```
Alias for top coordinate.
## RegionShape
```python theme={null}
@dataclass(frozen=True)
class RegionShape()
```
Defines the shape of a region within a WSI.
The width of the region in pixels.
The height of the region in pixels.
The unit of measurement for the shape dimensions (e.g., PIXELS, UM).
**Example**:
```pycon theme={null}
>>> shape = RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS)
>>> print(shape.unit)
PIXELS
>>> print(shape.width)
500
```
#### shape
```python theme={null}
@property
def shape() -> tuple[int, int]
```
Returns the width and height as a tuple.
tuple\[int, int]: A tuple containing (width, height).
#### to\_um
```python theme={null}
def to_um(mpp: MPP) -> "RegionShape"
```
Converts pixels to microns.
The pixel count to convert.
The microns-per-pixel scale.
The length in microns.
**Example**:
```pycon theme={null}
>>> unit = MeasurementUnit.PIXELS
>>> unit.pixels_to_um(100, MPP(0.5))
50.0
```
#### to\_pixels
```python theme={null}
def to_pixels(mpp: MPP) -> "RegionShape"
```
Converts microns to pixels.
The length in microns to convert.
The microns-per-pixel scale.
The length in pixels.
**Example**:
```pycon theme={null}
>>> unit = MeasurementUnit.UM
>>> unit.um_to_pixels(50, MPP(0.5))
100.0
```
## RegionMaskSpec
```python theme={null}
@dataclass(frozen=True)
class RegionMaskSpec()
```
Defines the mask applied/corresponding to the region e.g., during extraction can be binary or multiclass mask.
Pixels belonging to the mask relative to the total region area.
The mask applied to the region, or `None` if mask storage was not requested.
## RegionSpec
```python theme={null}
@dataclass(frozen=True)
class RegionSpec()
```
Defines the specifications of a region within a WSI.
The location of the region within the slide.
The shape of the region.
The resolution at which the region is defined (Level, MPP, or Magnification).
**Example**:
```pycon theme={null}
>>> location = RegionLocation(top=100, left=200)
>>> spec = RegionSpec(
... location=location,
... shape=RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS),
... resolution=Level(0),
... )
>>> print(spec.location.left)
200
```
## Region
```python theme={null}
@dataclass(frozen=True)
class Region()
```
Defines a region of interest within a WSI, including its image data and specifications. This class encapsulates both the pixel data of a region and its metadata, such as location, shape, and resolution.
The pixel data of the region as a PIL Image.
The specifications of the region, including location, shape, resolution, and mask.
**Example**:
```pycon theme={null}
>>> from PIL import Image
>>> img = Image.new('RGB', (500, 500)) # blank image for demonstration
>>> location = RegionLocation(top=100, left=200)
>>> shape = RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS)
>>> mask_spec = RegionMaskSpec(mask=np.zeros((500, 500)), ratio=0.0)
>>> spec = RegionSpec(location=location, shape=shape, resolution=Level(0), mask_spec=mask_spec)
>>> region = Region(image=img, spec=spec)
>>> print(region.spec.location.left)
200
```
#### location
```python theme={null}
@property
def location() -> RegionLocation
```
Convenience property to access the region's location directly.
#### shape
```python theme={null}
@property
def shape() -> tuple[int, int]
```
Convenience property to access the region's shape directly.
#### resolution
```python theme={null}
@property
def resolution() -> Resolution
```
Convenience property to access the region's resolution directly.
#### measurement\_unit
```python theme={null}
@property
def measurement_unit() -> MeasurementUnit
```
Convenience property to access the region's measurement unit directly.
#### mask
```python theme={null}
@property
def mask() -> Union[np.ndarray, None]
```
Convenience property to access the region's mask directly.
np.ndarray | None: The mask array, or `None` if mask storage was not requested during extraction.
#### mask\_ratio
```python theme={null}
@property
def mask_ratio() -> float
```
Convenience property to access the region's mask ratio directly.
# backbones
Source: https://docs.bioptimus.com/sdk-reference/models/backbones
Backbone feature-extractor models (H1, M-Optimus, tissue-seg).
Model specifications are loaded from YAML config files in the `configs/` directory. The `Backbone` factory wires them together with a pluggable client so that a single `Backbone(...)` call works for any backend (HTTP, AWS, GCP, …).
**Example:**
```python theme={null}
backbone = Backbone("h1", base_url="http://localhost:8080")
backbone = Backbone("h1", backend="aws", endpoint_name="prod")
```
## Backbone
```python theme={null}
class Backbone()
```
Factory for creating backbone endpoint models.
Model identity (spec, endpoint path) is read from YAML config files. The factory pairs each config with a pluggable client backend to produce a ready-to-use `EndpointModel`.
**Usage:**
```pycon theme={null}
>>> Backbone("h1", base_url="http://localhost:8080")
>>> Backbone("h1", backend="aws", endpoint_name="ep")
>>> Backbone.available_backbones()
['h1', 'm-optimus', 'tissue-seg']
```
## Backend
```python theme={null}
class Backend(str, Enum)
```
Supported inference backends.
#### available\_backbones
```python theme={null}
@classmethod
def available_backbones(cls) -> List[str]
```
Return list of all registered backbone names.
# aws
Source: https://docs.bioptimus.com/sdk-reference/models/clients/aws
AWS SageMaker client.
Sends JSON payloads to a SageMaker endpoint via the `sagemaker-runtime` `invoke_endpoint` API. The `model_name` field is injected into every request body for multi-model dispatch.
## AWSClient
```python theme={null}
class AWSClient()
```
Sends JSON payloads to an AWS SageMaker endpoint.
Name of the SageMaker endpoint.
Model identifier injected into the request body for SageMaker multi-model dispatch.
AWS region. When *None*, uses the default from the environment or *boto\_session*.
Optional pre-configured `Session` (e.g. with explicit credentials or a custom profile). When *None*, a default session is created using the standard credential chain.
Read timeout in seconds.
**Example:**
```python theme={null}
client = AWSClient(
endpoint_name="h1-prod",
model_name="h1",
)
resp_json = client.predict(request_json)
```
#### predict
```python theme={null}
def predict(body: str) -> str
```
Invoke the SageMaker endpoint for prediction.
Serialized JSON request payload.
Response body as a JSON string.
#### embed
```python theme={null}
def embed(body: str) -> str
```
Invoke the SageMaker endpoint for embedding.
Serialized JSON request payload.
Response body as a JSON string.
#### metadata
```python theme={null}
def metadata() -> str
```
Fetch model metadata via `/invocations`.
Sends a minimal request with `mode` set to `"metadata"` so the server returns model metadata instead of running inference.
Response body as a JSON string.
#### predict\_async
```python theme={null}
async def predict_async(body: str, session: Any = None) -> str
```
Predict asynchronously via `run_in_executor`.
`boto3` is not natively async, so the synchronous `predict` call is delegated to a thread pool.
Serialized JSON payload.
Unused. Accepted for interface compatibility with the `Client` protocol.
Response body as a string.
#### embed\_async
```python theme={null}
async def embed_async(body: str, session: Any = None) -> str
```
Embed asynchronously via `run_in_executor`.
Serialized JSON payload.
Unused. Accepted for interface compatibility.
Response body as a string.
# http
Source: https://docs.bioptimus.com/sdk-reference/models/clients/http
HTTP client for direct API endpoints.
Sends JSON payloads to a model server via plain HTTP POST using `requests` (synchronous) and `aiohttp` (asynchronous).
## HTTPClient
```python theme={null}
class HTTPClient()
```
Sends JSON payloads over HTTP.
Fully configured at construction — `predict` / `embed` only need the serialized body.
Root URL of the model server, e.g. `"http://localhost:8080"`.
Mapping of endpoint type to path, e.g. `{"prediction": "/api/predict/m-optimus", "embedding": "/api/embed/m-optimus"}`. At least one key must be present.
HTTP request timeout in seconds.
**Example:**
```python theme={null}
client = HTTPClient(
"http://localhost:8080",
endpoints={
"prediction": "/api/predict/m-optimus",
"embedding": "/api/embed/m-optimus",
},
)
pred_json = client.predict(request_json)
emb_json = client.embed(request_json)
```
#### close
```python theme={null}
def close() -> None
```
Closes the underlying HTTP session and releases connections.
#### predict
```python theme={null}
def predict(body: str) -> str
```
POST to the prediction endpoint.
Serialized JSON request payload.
Response body as a JSON string.
#### embed
```python theme={null}
def embed(body: str) -> str
```
POST to the embedding endpoint.
Serialized JSON request payload.
Response body as a JSON string.
#### metadata
```python theme={null}
def metadata() -> str
```
GET the metadata endpoint.
Response body as a JSON string.
#### predict\_async
```python theme={null}
async def predict_async(body: str, session: Any = None) -> str
```
POST to the prediction endpoint asynchronously.
Serialized JSON request payload.
An `ClientSession` for connection pooling.
Response body as a JSON string.
#### embed\_async
```python theme={null}
async def embed_async(body: str, session: Any = None) -> str
```
POST to the embedding endpoint asynchronously.
Serialized JSON request payload.
An `ClientSession` for connection pooling.
Response body as a JSON string.
# config_loader
Source: https://docs.bioptimus.com/sdk-reference/models/config_loader
Model configuration loader.
Reads model definition YAML files from the `configs/` directory and builds `ModelSpec` instances. This is the single source of truth for all model specifications.
## EndpointPaths
```python theme={null}
@dataclass(frozen=True)
class EndpointPaths()
```
Parsed endpoint routes for a model.
Each field corresponds to an optional endpoint type declared in the model's YAML config under `endpoints:`.
## ModelConfig
```python theme={null}
@dataclass(frozen=True)
class ModelConfig()
```
A fully parsed model configuration.
Combines the `ModelSpec` with routing metadata that is not part of the spec itself.
#### endpoint\_path
```python theme={null}
@property
def endpoint_path() -> str
```
Primary endpoint path (backward compatibility).
#### genes\_endpoint\_path
```python theme={null}
@property
def genes_endpoint_path() -> str | None
```
Genes/metadata endpoint path (backward compatibility).
#### load\_model\_config
```python theme={null}
def load_model_config(path: Path) -> ModelConfig
```
Loads a single model config YAML file.
Path to the YAML file.
A `ModelConfig` with the parsed spec and routing.
#### load\_all\_model\_configs
```python theme={null}
def load_all_model_configs(
configs_dir: Path | None = None) -> Dict[str, ModelConfig]
```
Discovers and loads all model configs from a directory.
Scans for `*.yaml` files and returns a dict keyed by `model_name`.
Directory to scan. Defaults to the package-bundled `configs/` directory.
Mapping of model name to `ModelConfig`.
# endpoint_model
Source: https://docs.bioptimus.com/sdk-reference/models/endpoint_model
Client-agnostic endpoint model.
Provides a single `EndpointModel` that pairs a `ModelSpec` with any `Client` implementation (HTTP, AWS SageMaker, GCP, Azure, …). This eliminates the need for per-backend model subclasses.
## EndpointModel
```python theme={null}
class EndpointModel()
```
Model endpoint backed by a pluggable client.
Combines a `ModelSpec` (what the model expects and produces) with a `Client` (how to reach it).
Supports both synchronous (`predict` / `embed`) and asynchronous (`predict_async` / `embed_async`) dispatch.
Model specification describing input requirements and output shape.
A configured `Client` instance (e.g. `HTTPClient`, `AWSClient`).
Optional ordered Ensembl IDs expected as bulk RNA input (M-Optimus ).
Optional ordered Ensembl IDs of predicted output genes (M-Optimus).
**Example:**
```python theme={null}
from bioptimus.models.clients.http import HTTPClient
client = HTTPClient(
"http://localhost:8080",
endpoints={"embedding": "/api/embed/h1"},
)
model = EndpointModel(H1_SPEC, client)
response = model(request)
```
#### model\_spec
```python theme={null}
@property
def model_spec() -> ModelSpec
```
Model specification including tile and output config.
#### input\_gene\_names
```python theme={null}
@property
def input_gene_names() -> list[str] | None
```
Ordered Ensembl IDs expected as bulk RNA input.
#### output\_gene\_names
```python theme={null}
@property
def output_gene_names() -> list[str] | None
```
Ordered Ensembl IDs of predicted output genes.
#### predict
```python theme={null}
def predict(request: ModelRequest) -> ModelResponse
```
Send a prediction request to the endpoint.
Tile request payload to send.
Parsed model response with predictions.
#### embed
```python theme={null}
def embed(request: ModelRequest) -> ModelResponse
```
Send an embedding request to the endpoint.
Tile request payload to send.
Parsed model response with embeddings.
#### predict\_async
```python theme={null}
async def predict_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Send a prediction request asynchronously.
Tile request payload to send.
An `ClientSession` for connection pooling.
Parsed model response with predictions.
#### embed\_async
```python theme={null}
async def embed_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Send an embedding request asynchronously.
Tile request payload to send.
An `ClientSession` for connection pooling.
Parsed model response with embeddings.
# types
Source: https://docs.bioptimus.com/sdk-reference/models/types
Model specification types.
Defines `ModelSpec` which encapsulates a model's identity, input tile requirements, normalisation parameters, output contract, and compute hints.
Reuses `TileSpec` for tile geometry so the same spec drives both the extraction pipeline and the model's expectations.
**Example:**
```python theme={null}
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.io.wsi.types import MPP, MeasurementUnit
from bioptimus.models.types import ModelSpec
spec = ModelSpec(
model_name="h1",
version="1.0.0",
tile_spec=TileSpec(
size=(224, 224),
stride=(224, 224),
resolution=MPP(0.5),
unit=MeasurementUnit.PIXELS,
),
embedding_dim=1536,
output_type="cls+patch_mean",
num_prefix_tokens=1,
patch_size=14,
mean=(0.707, 0.579, 0.704),
std=(0.212, 0.230, 0.178),
weights_source="bioptimus/H1",
)
```
## Models
```python theme={null}
class Models(str, Enum)
```
Known model identifiers.
Values correspond to the `model_name` field in each YAML config under `bioptimus/models/configs/`.
## ModelSpec
```python theme={null}
@dataclass(frozen=True)
class ModelSpec()
```
Describes a backbone model's input and output contract.
Combines tile extraction requirements (via `TileSpec`) with preprocessing, architecture, and output metadata so that data pipelines can prepare inputs and interpret outputs correctly without manual configuration.
This is **framework-agnostic** — it contains only plain Python types and can be consumed by PyTorch, TensorFlow, JAX, or any other framework.
Groups: **Identity & provenance** — who is this model? **Input / preprocessing** — what does the model expect? **Architecture** — structural hints for generic code. **Output** — how to interpret the raw model output. **Compute** — precision & hardware hints.
Unique identifier / registry key (e.g. `"h1"`).
Model version string (e.g. `"1.0.0"`). Ensures embeddings extracted with v1 are not mixed with v2.
HuggingFace repo, URL, or local path to weights (e.g. `"bioptimus/H1"`). `None` = no auto-download.
SPDX identifier or short description (e.g. `"proprietary"`, `"apache-2.0"`).
Tile geometry the model expects (size, stride, resolution, measurement unit). Directly reusable by `TileExtractor`.
Number of input image channels (default 3 for RGB).
Per-channel normalisation mean (channel order matches input).
Per-channel normalisation std.
Resize interpolation mode string (e.g. `"bicubic"`, `"bilinear"`).
Whether the resize operation should use antialiasing.
Expected colour space of the input image (`"RGB"`, `"BGR"`, `"HED"`).
Stain normalisation method applied *before* the model, or `None` for no stain normalisation. (e.g. `"macenko"`, `"reinhard"`, `"vahadane"`).
Architecture family (`"vit"`, `"swin"`, `"resnet"`, `"convnext"`, …).
ViT patch size (e.g. 14, 16). Determines the number of output tokens = `(tile_size / patch_size)²`. `None` for non-ViT architectures.
Number of non-spatial prefix tokens before patch tokens in the output sequence (CLS, register tokens, …). E.g. DINOv2-reg = 5, most ViTs = 1.
Dimensionality of the *final* output feature vector (after any post-processing like CLS + mean pooling).
How the raw model output is consumed: `"cls"` | `"patch_mean"` | `"cls+patch_mean"` | `"dense"` | `"token_sequence"`.
For dense / segmentation models that output a spatial feature map, e.g. `(16, 16)`. `None` for pooled-output models.
Recommended inference precision (`"fp32"`, `"fp16"`, `"bf16"`).
# base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/base
# factory
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/factory
# transforms
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/transforms
# bioptimus.preprocess.wsi.models.base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/models/base
Abstract interface for tissue segmentation models.
Defines the `TissueMaskModel` abstract base class used by tissue mask providers to process individual tile requests into binary segmentation masks.
## TissueMaskModel
```python theme={null}
class TissueMaskModel(ABC)
```
Abstract interface for tissue segmentation models.
Accepts a **batch** of fixed-size RGB tiles and returns a batch of binary masks. GPU-backed models can process the whole batch at once; CPU-only models (like Otsu) can iterate internally.
The batch dimension is always the first axis:
* Input: `(B, H, W, 3)` — `uint8` RGB.
* Output: `(B, H, W)` — `uint8`, values in `{0, 1}`.
**Example** class MyGPUModel(TissueMaskModel): def **init**(self, net): self.net = net
def process(self, batch): tensor = torch.from\_numpy(batch).permute(0, 3, 1, 2).float() / 255 with torch.no\_grad(): logits = self.net(tensor.cuda()) return (logits.squeeze(1).cpu().numpy() > 0.5).astype(np.uint8)
#### tile\_spec
```python theme={null}
@property
@abstractmethod
def tile_spec() -> TileSpec | None
```
Returns the tile specification for this model, or `None`.
Subclasses must implement this property. Return `None` for models that operate on arbitrary tile sizes.
#### process
```python theme={null}
@abstractmethod
def process(request: ModelRequest) -> ModelResponse
```
Processes a single RGB tile into a binary mask.
Model request containing the RGB tile.
Model response containing the binary mask.
# bioptimus_mask
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/models/bioptimus_mask
Remote tissue segmentation mask model.
Wraps a tissue segmentation `EndpointModel` as a `TissueMaskModel`, forwarding each `ModelRequest` to the remote tissue segmentation endpoint.
## BioptimusTissueMaskModel
```python theme={null}
class BioptimusTissueMaskModel(TissueMaskModel)
```
Tissue mask model backed by a segmentation endpoint.
Forwards each `ModelRequest` to the `EndpointModel` and returns the `ModelResponse`.
An `EndpointModel` configured for tissue segmentation.
**Example:**
```python theme={null}
from bioptimus.models.backbones import Backbone
backbone = Backbone("tissue-seg", base_url="http://localhost:8080")
model = BioptimusTissueMaskModel(backbone)
response = model.process(request)
```
#### tile\_spec
```python theme={null}
@property
def tile_spec() -> TileSpec | None
```
Returns the tile spec from the backbone's model spec.
#### process
```python theme={null}
def process(request: ModelRequest) -> ModelResponse
```
Sends a request to the remote tissue segmentation endpoint.
A `ModelRequest` containing the tile image and metadata.
A `ModelResponse` with the segmentation output.
#### process\_async
```python theme={null}
async def process_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Sends a request asynchronously to the segmentation endpoint.
A `ModelRequest` containing the tile image and metadata.
An `ClientSession` for connection pooling.
A `ModelResponse` with the segmentation output.
# bioptimus.preprocess.wsi.provider.base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/base
Abstract interface for tissue mask providers.
Abstract interface for tissue mask providers.
Defines the `TissueMaskProvider` abstract base class that `TileExtractor` depends on for obtaining binary tissue masks.
## TissueMaskProvider
```python theme={null}
class TissueMaskProvider(ABC)
```
Abstract base class for tissue mask providers.
Every provider must implement `generate`, which receives an **open** `WSIReader` and returns a 2-D `uint8` binary mask where `1` = tissue.
`TileExtractor` depends only on this interface, so any concrete provider — algorithmic, model-based, or pre-computed — can be plugged in.
**Example** class MyCustomMask(TissueMaskProvider): def generate(self, reader): return load\_from\_database(reader.path)
#### generate
```python theme={null}
@abstractmethod
def generate(reader: WSIReader, *, show_progress: bool = True) -> np.ndarray
```
Generates a binary tissue mask for the given slide.
An **open** WSI reader.
Whether to display a progress bar. Defaults to `True`.
A `uint8` mask of shape `(H, W)` with values in `{0, 1}` where `1` = tissue.
# bioptimus.preprocess.wsi.provider.precomputed
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/precomputed
Pre-computed tissue mask provider.
Pre-computed tissue mask provider.
Loads a pre-existing mask from a directory on disk, matched by slide name. Useful for clinician annotations or masks produced by an external pipeline.
## PrecomputedTissueMask
```python theme={null}
class PrecomputedTissueMask(TissueMaskProvider)
```
Loads a pre-existing mask from a directory, matched by slide name.
Clinicians or annotation pipelines often produce per-slide masks (e.g. tissue regions, tumour annotations) and store them alongside the slides. This provider looks up the mask file whose **stem** matches the slide name and loads it.
Supported formats: PNG, JPEG, TIFF, BMP (loaded via OpenCV) and `.npy` (loaded via `load`).
The loaded mask is binarised (non-zero → 1) and returned as `uint8 (H, W)`.
Directory containing mask files.
Expected file extension **including the dot** (e.g. `".png"`). When `None` (default), the provider searches for the first file in *mask\_dir* whose stem matches the slide name, trying all supported extensions.
**Raises:**
* `FileNotFoundError` — At `generate` time if no matching mask file is found for a slide. **Example**
```python theme={null}
from bioptimus.preprocess.wsi.provider.precomputed import (
PrecomputedTissueMask,
)
```
# Directory layout:
# /data/masks/slide\_001.png
# /data/masks/slide\_002.png
provider = PrecomputedTissueMask(mask\_dir="/data/masks/") mask = provider.generate(reader) # reader.path → …/slide\_001.svs
# With explicit suffix:
provider = PrecomputedTissueMask( mask\_dir="/data/masks/", suffix=".tiff" )
#### generate
```python theme={null}
def generate(reader: WSIReader, *, show_progress: bool = True) -> np.ndarray
```
Loads the pre-computed mask that matches the slide name.
Extracts the slide stem from `reader.path`, finds the corresponding file in `mask_dir`, loads and binarises it.
An **open** WSI reader.
Ignored for pre-computed masks (kept for interface compatibility).
A `uint8` mask of shape `(H, W)` with values in `{0, 1}` where `1` = tissue / region of interest.
**Raises:**
* `FileNotFoundError` — If no mask file is found for the slide.
# bioptimus.preprocess.wsi.provider.tiled
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/tiled
Tiled tissue mask generation using a batched segmentation model.
Implements `TiledTissueMask`, which walks a WSI in fixed-size patches, processes them through a `TissueMaskModel`, and stitches the per-tile predictions into a full-slide binary mask.
## TiledTissueMask
```python theme={null}
class TiledTissueMask(TissueMaskProvider)
```
Tile-based, batch-aware tissue mask generation.
Combines a `TissueMaskModel` with a tiling strategy to produce a full-slide binary mask. The workflow is:
1. Compute the tissue bounding box at *resolution*.
2. Walk the bounding box in non-overlapping *tile\_size* patches.
3. Collect patches into batches of *batch\_size*.
4. Pass each batch to `model.process()` — this is where GPU parallelism happens for DL models.
5. Stitch the per-tile masks back into the output array.
Boundary tiles that are smaller than *tile\_size* are zero-padded before model processing and the padding is stripped from the output.
The processing model.
`(width, height)` of each patch. When `None`, derived from `model.tile_spec`.
Resolution at which to read tiles. When `None`, derived from `model.tile_spec`.
Number of tiles per model call. Set higher for GPU models (e.g. 32–64). Defaults to `1`.
**Example:**
```python theme={null}
provider = TiledTissueMask(
model=MyGPUModel(net),
tile_size=(512, 512),
resolution=Level(0),
batch_size=32,
)
mask = provider.generate(reader)
# Or let the model's tile_spec drive the defaults:
provider = TiledTissueMaskProvider(model=my_model)
```
#### generate
```python theme={null}
def generate(reader: WSIReader,
*,
show_progress: bool = True,
max_concurrency: int = _DEFAULT_MAX_CONCURRENCY) -> np.ndarray
```
Generates a tissue mask by tiling over the WSI.
Tile images are read sequentially from the WSI (readers are not thread-safe), then dispatched concurrently to the remote model endpoint for GPU-efficient batched inference. Results are stitched back into a single binary mask.
An **open** WSI reader.
Whether to display a per-tile progress bar. Defaults to `True`.
Maximum concurrent HTTP requests. Defaults to 64.
A `uint8` mask of shape `(H, W)` covering the tissue bounding box at `resolution`, with values in `{0, 1}` where `1` = tissue.