# Anonymized diagnostics or usage records
Source: https://docs.bioptimus.com/api-reference/diagnostics/anonymized-diagnostics-or-usage-records
/api-reference/openapi.json get /diagnostics
Returns selected anonymized diagnostics categories.
Reuses :func:`bioptimus.observability.probes.snapshot` to capture
versions, host, GPU, dependencies, disk, and loaded-model state for the
models currently held by this worker, and :func:`read_recent_logs` to
read the tail of the server's log file. Both are anonymized so the
response is safe to attach to a support request. The snapshot is
fail-safe: a failing probe degrades to an error section rather than
failing the request. Logs are empty when no log file is configured.
When the optional ``categories`` query parameter is omitted, the
response keeps the historical full-diagnostics shape. When provided,
``usage`` returns only model-request usage log entries and
``diagnostics`` returns the full diagnostics payload under a
``diagnostics`` key.
Security: this endpoint is unauthenticated and returns the server's
environment and recent activity, so it must sit behind the same VPN or
network protection as the rest of the deployment rather than being
exposed publicly.
# List available endpoints
Source: https://docs.bioptimus.com/api-reference/discovery/list-available-endpoints
/api-reference/openapi.json get /bioptimus/
# Bioptimus Inference Server
Source: https://docs.bioptimus.com/api-reference/introduction
The REST API served by the Bioptimus Model Server, and the SageMaker dispatch endpoint.
The Bioptimus Model Server exposes a JSON REST API. The same `ModelRequest` / `ModelResponse` schema is used everywhere; what differs is how you reach it:
* **On-premise** — path-based endpoints (`/api/embed/h1`, `/api/predict/m-optimus`, …) on the container (default port 8080). Interactive Swagger UI is served at `/docs`, and the live schema at `/openapi.json`.
* **AWS SageMaker** — a single `/invocations` endpoint that accepts a `SageMakerRequest` (a `ModelRequest` plus `model_name` and `mode` for dispatch).
* **SDK** — the [Bioptimus SDK](/guides/get-started/sdk) wraps both and is the recommended interface for whole-slide inference.
The model endpoints (embedding, prediction, metadata) are listed in the **Endpoints** section, generated from the server's OpenAPI schema. The endpoints below are not part of that schema:
| Endpoint | Method | Purpose |
| - | - | - |
| `/invocations` | POST | SageMaker dispatch (adds `model_name`, `mode`) |
| `/ping` | GET | Health check |
| `/bioptimus/` | GET | Service discovery |
## Authentication
The Model Server REST API requires no authentication. Deploy it on a private network and restrict access at the network layer. On AWS, the SageMaker endpoint is protected by IAM (requests are SigV4-signed); on Hugging Face, model access is gated.
## Errors
All endpoints return JSON. Common statuses:
| Status | Meaning |
| - | - |
| `200` | Success |
| `422` | Validation error — malformed or missing fields (FastAPI `HTTPValidationError`) |
| `500` | Inference error |
| `503` | Model still loading, or GPU unavailable (see `/ping` states) |
A `422` body follows the FastAPI shape: `{"detail": [{"loc": [...], "msg": "...", "type": "..."}]}`.
# h1 embedding
Source: https://docs.bioptimus.com/api-reference/models/h1-embedding
/api-reference/openapi.json post /api/embed/h1
# h1 metadata
Source: https://docs.bioptimus.com/api-reference/models/h1-metadata
/api-reference/openapi.json get /api/metadata/h1
# m-optimus embedding
Source: https://docs.bioptimus.com/api-reference/models/m-optimus-embedding
/api-reference/openapi.json post /api/embed/m-optimus
# m-optimus metadata
Source: https://docs.bioptimus.com/api-reference/models/m-optimus-metadata
/api-reference/openapi.json get /api/metadata/m-optimus
# m-optimus prediction
Source: https://docs.bioptimus.com/api-reference/models/m-optimus-prediction
/api-reference/openapi.json post /api/predict/m-optimus
# m-optimus prediction_with_embedding
Source: https://docs.bioptimus.com/api-reference/models/m-optimus-prediction_with_embedding
/api-reference/openapi.json post /api/predict_with_embedding/m-optimus
# tissue-seg metadata
Source: https://docs.bioptimus.com/api-reference/models/tissue-seg-metadata
/api-reference/openapi.json get /api/metadata/tissue-seg
# tissue-seg prediction
Source: https://docs.bioptimus.com/api-reference/models/tissue-seg-prediction
/api-reference/openapi.json post /api/predict/tissue-seg
# Performance
Source: https://docs.bioptimus.com/api-reference/performance
Model-server performance characteristics: what's fixed by the deployment, and what to expect.
For Bioptimus embedding and prediction models, **the GPU forward pass is usually the bottleneck** of a request — a couple of server-level choices dominate throughput. This page covers those. If you're driving inference through the SDK, the client-side knobs (concurrency, tissue masking, cohort workers) are covered in [Performance tuning](/guides/get-started/performance-tuning).
## Precision (fp16 vs fp32)
The single biggest lever is which model variant is deployed: **fp16 is roughly 3× faster than fp32** on the same GPU, for the same workload (benchmarked on an A10G-class GPU). This is a **deployment-time choice**, not a per-request option — it depends on which image/artifact was built and deployed. See [On-premise deployment](/deployment/platforms/on-premise) and [AWS SageMaker](/deployment/platforms/aws-sagemaker) for choosing the fp16 build.
## Batching (on-premise / self-hosted)
The server auto-batches concurrent requests before running them through the GPU. These env vars control that behavior:
| Variable | Default | Effect |
| - | - | - |
| `BATCH_MAX_SIZE` | `48` | Maximum samples per GPU batch. |
| `BATCH_MAX_WAIT_MS` | `30.0` | Maximum time to wait before dispatching a partial batch. |
| `PREPROCESS_WORKERS` | `8` | Thread-pool size for request decoding/preprocessing before batching. |
In practice, once the GPU is saturated, tuning these away from their defaults has little effect — they matter mainly if requests are arriving faster than the GPU can drain them. Set them with `-e` when starting the container; see [Environment variables](/deployment/platforms/on-premise#environment-variables).
## GPU-bound behavior
Because the forward pass dominates, throughput scales with how much work is sent to the server — reducing the number of tiles per slide (tissue masking, done client-side) reduces total time proportionally. Once precision and tile count are fixed, the remaining lever is the GPU itself: a bigger/newer GPU raises the ceiling directly, and additional GPUs scale a multi-GPU deployment (1:1 worker-to-GPU mapping).
Seeing lower throughput than this would suggest? See [Troubleshooting](/api-reference/troubleshooting).
# Troubleshooting
Source: https://docs.bioptimus.com/api-reference/troubleshooting
Diagnosing slow throughput or errors from the Model Server.
Not everything slow is a tuning problem. Start here before changing any settings.
## Is the GPU actually the bottleneck?
Watch utilization while a run is in progress:
```bash theme={null}
watch nvidia-smi
```
* **GPU near 100%** → you're GPU-bound. Only precision, tile count, or a bigger/newer GPU will help further — see [Performance](/api-reference/performance).
* **GPU well below 100%** → the GPU is starved. Check the client's concurrency setting first, then disk/network I/O and CPU load.
At `INFO` level, the server also logs per-batch timing (`forward=... cpu_transfer=... total=...`). If `forward` ≈ `total` and batches fire back-to-back, you're GPU-bound (expected); a gap between consecutive "batch done" lines means the GPU is waiting on tile production instead.
## Failure modes that look like slowness
Cold start — model loading and first-batch kernel warmup. Don't benchmark the first request; measure steady-state throughput instead.
Storage can't keep up with the GPU. Stage slides on fast local disk where possible.
Another process may be sharing the GPU. Check `nvidia-smi` for other processes and ensure exclusive access.
Oversubscribing CPU cores causes thread thrashing. Don't set it above the host's core count; it rarely helps once the GPU is the bottleneck anyway.
`.pt2` artifacts extract to a temp directory (`TMPDIR`, default `/tmp`) on load; a failed load can leave stale extraction directories behind and fill the disk. Check `df -h /tmp`, clear stale extraction directories, or point `TMPDIR` at a larger volume.
## Errors
`503` and out-of-memory responses usually mean too many requests are arriving concurrently for the deployed batch size — see [Errors](/api-reference/introduction#errors) for the full status code reference.
Calling the server through the SDK? For connection issues (e.g. `/ping` not responding), see the SDK's [Common issues](/guides/get-started/troubleshooting) guide.
# Deployment overview
Source: https://docs.bioptimus.com/deployment/overview
Choose the right way to access Bioptimus models.
Bioptimus models are available several ways. The on-premise and SageMaker options share the same JSON API and are both supported by the [Bioptimus SDK](/guides/get-started/sdk), so your inference code is portable between them. Proscia is a partner-operated path for slides already in a Concentriq® LS workspace.
Managed cloud endpoints from AWS Marketplace. Best for production and scale.
Self-hosted container. Best for data residency and air-gapped environments.
Direct H-Optimus weights for non-commercial academic research.
H-Optimus embeddings for slides in a Concentriq® LS workspace. Operated by Proscia.
## Which should I choose?
| Consideration | AWS SageMaker | On-premise | Hugging Face | Proscia |
| - | - | - | - | - |
| Time to first inference | Minutes | Moderate (infra setup) | Minutes | Minutes |
| Data leaves your environment | Within your AWS account/region | No | No | Within Concentriq® LS |
| Commercial use | Yes | Yes | **No** — academic only | Yes |
| Models | H-Optimus, M-Optimus | H-Optimus, M-Optimus | H-Optimus only | H-Optimus |
| Best for | Cloud-native production | Regulated / air-gapped | Academic research | Slides in Concentriq® LS |
For data residency and PHI considerations, see [Security & compliance](/documentation/resources/security-compliance).
# AWS SageMaker
Source: https://docs.bioptimus.com/deployment/platforms/aws-sagemaker
Deploy a Bioptimus model from AWS Marketplace to a SageMaker endpoint.
Subscribe to a model on [AWS Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-buy.html), deploy it as a [SageMaker endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html), and run inference either with the [Bioptimus SDK](/guides/get-started/sdk) (recommended) or the raw SageMaker runtime API.
## 1. Subscribe and configure
If your Bioptimus contact sent you a [**private offer**](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-private-offers.html), accept it via the link they provided before continuing. After acceptance the model appears under **Manage your subscriptions**, and the remaining steps are identical.
Subscribe to the model. In **Manage your subscriptions**, open your model subscription and click **Configure**.
You will be asked to select the model access interface — **SageMaker AI** (Python SDK) or **CLI** — based on your preference and use case.
Select your **region** and **inference mode** (e.g. a [real-time inference endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) or [batch transform jobs](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html)). This guide spawns a simple real-time inference endpoint.
Copy the [**Model Package ARN**](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-buy.html) shown in the right-hand pane of the configuration page — make sure you copy the one for your chosen region.
The region must be identical across every command and in the Model Package ARN. Mixing regions is the most common cause of deployment failures.
## 2. Deploy an endpoint
Deploy with the Python SDK (recommended) or the AWS CLI.
```python theme={null}
import sagemaker
from sagemaker import ModelPackage
session = sagemaker.Session()
role = ""
model = ModelPackage(role=role,
model_package_arn="",
sagemaker_session=session)
predictor = model.deploy(initial_instance_count=1,
instance_type="ml.g5.xlarge",
endpoint_name="bioptimus-prod",
inference_ami_version="al2-ami-sagemaker-inference-gpu-2")
```
AWS walks you through these steps from the configuration page. The commands below are provided in case you prefer to run them yourself. For full option details, see the [`aws sagemaker` command reference](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/index.html).
First, [install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).
Find your **SSO start URL** and **SSO region** by following the *Access Keys* link on the AWS landing page for your role, then [configure an IAM Identity Center (SSO) profile](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html):
```bash theme={null}
aws configure sso
```
Answer the prompts:
1. **Session name** — anything you like (e.g. your username).
2. **SSO start URL** — the URL you obtained above.
3. **SSO region** — the region you obtained above.
4. **SSO registration scopes** — leave blank; it defaults to `sso:account:access`.
5. Authorize access in the browser page that opens.
6. Select the account to use.
7. Select the **`AWSAdministratorAccess`** role (required for the next steps).
8. **CLI default client Region** — leave blank.
9. **CLI default output format** — leave blank.
10. **CLI profile name** — a name to reuse this profile later.
If subsequent steps fail, activate the profile explicitly:
```bash theme={null}
export AWS_PROFILE=
```
Create (or reuse) a [SageMaker execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html):
```bash theme={null}
role_name="AmazonSageMaker-ExecutionRole-AWSMarketplace"
aws iam create-role \
--role-name "${role_name}" \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "sagemaker.amazonaws.com" },
"Action": "sts:AssumeRole"
}
]
}'
aws iam attach-role-policy \
--role-name "${role_name}" \
--policy-arn "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
aws iam put-role-policy \
--role-name "${role_name}" \
--policy-name "AmazonSageMaker-ExecutionPolicy" \
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::sagemaker"]
},
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
"Resource": ["arn:aws:s3:::sagemaker/*"]
}
]
}'
```
Store the role ARN in a variable for the next steps:
```bash theme={null}
execution_role_arn="arn:aws:iam::$(aws sts get-caller-identity --query 'Account' --output text):role/${role_name}"
```
Use the **Model Package ARN** for your region. [Network isolation](https://docs.aws.amazon.com/sagemaker/latest/dg/mkt-algo-model-internet-free.html) is required for Marketplace model packages. The names below are arbitrary and work for either package (H-Optimus or M-Optimus) — choose your own.
```bash theme={null}
model_name="bioptimus-model"
endpoint_name="bioptimus-endpoint"
aws sagemaker create-model \
--model-name "${model_name}" \
--execution-role-arn "${execution_role_arn}" \
--primary-container ModelPackageName="" \
--enable-network-isolation \
--region
```
Create the endpoint configuration (instance type and count), then the endpoint itself. The generous timeouts give the large model weights time to download and load before SageMaker's startup health check — without them, endpoint creation can fail.
Set `` to a [supported SageMaker inference AMI](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ProductionVariant.html) for your region (e.g. `al2-ami-sagemaker-inference-gpu-2`), or omit the field to let SageMaker pick the default.
```bash theme={null}
aws sagemaker create-endpoint-config \
--endpoint-config-name "${endpoint_name}-config" \
--production-variants '[{
"VariantName": "AllTraffic",
"ModelName": "'"${model_name}"'",
"InitialInstanceCount": 1,
"InstanceType": "ml.g5.xlarge",
"InitialVariantWeight": 1.0,
"InferenceAmiVersion": "al2-ami-sagemaker-inference-gpu-2",
"ModelDataDownloadTimeoutInSeconds": 1800,
"ContainerStartupHealthCheckTimeoutInSeconds": 1800
}]' \
--region
aws sagemaker create-endpoint \
--endpoint-name "${endpoint_name}" \
--endpoint-config-name "${endpoint_name}-config" \
--region
```
Endpoint creation takes about 5–10 minutes. Check the status from the CLI:
```bash theme={null}
aws sagemaker describe-endpoint \
--endpoint-name "${endpoint_name}" \
--region \
--query 'EndpointStatus'
```
Or open **SageMaker AI** in the AWS console and go to *Deployment & Inference > Endpoints*. The endpoint is ready once its status is `InService`.
## 3. Run inference with the Bioptimus SDK (recommended)
The Bioptimus SDK's AWS backend routes through the SageMaker `/invocations` endpoint and adds the `model_name` dispatch field for you.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.H1,
backend="aws",
endpoint_name="bioptimus-prod",
region_name="",
)
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.M_OPTIMUS,
backend="aws",
endpoint_name="bioptimus-prod",
region_name="",
)
```
For whole-slide inference (tiling, tissue masking, bulk RNA, output formats), see the [Bioptimus SDK](/guides/get-started/sdk).
## 4. Or call the runtime API directly
A SageMaker request is a `ModelRequest` plus `model_name` and `mode` fields. See the [API reference](/api-reference/introduction) for the full schema.
## 5. Clean up
Endpoints incur charges while running. Delete the endpoint when finished.
```python theme={null}
predictor.delete_endpoint()
```
Delete the endpoint, its configuration, and the model:
```bash theme={null}
aws sagemaker delete-endpoint --endpoint-name "${endpoint_name}" --region
aws sagemaker delete-endpoint-config --endpoint-config-name "${endpoint_name}-config" --region
aws sagemaker delete-model --model-name "${model_name}" --region
```
If you created the IAM execution role just for this deployment, delete it as well:
```bash theme={null}
role_name="AmazonSageMaker-ExecutionRole-AWSMarketplace"
aws iam delete-role-policy \
--role-name "${role_name}" \
--policy-name "AmazonSageMaker-ExecutionPolicy"
aws iam detach-role-policy \
--role-name "${role_name}" \
--policy-arn "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
aws iam delete-role --role-name "${role_name}"
```
## Reference notebooks
Bioptimus maintains runnable end-to-end examples for deploying and running inference on SageMaker. Use them as the source of truth:
End-to-end SageMaker examples for H-Optimus.
End-to-end SageMaker examples for M-Optimus.
# Hugging Face
Source: https://docs.bioptimus.com/deployment/platforms/hugging-face
Load H-Optimus-1 weights directly for non-commercial academic research.
For academic users, H-Optimus-1 weights are available directly on Hugging Face and can be loaded with `timm` — no server or AWS account required.
**Non-commercial academic use only.** H-Optimus-1 on Hugging Face is released under **CC-BY-NC-ND 4.0**. Commercial use, sale, or monetization (including models trained on its outputs) is prohibited without prior approval. For commercial use, deploy via [AWS](/deployment/platforms/aws-sagemaker) or [on-premise](/deployment/platforms/on-premise), or contact Bioptimus.
## 1. Request access
Access is gated. On [huggingface.co/bioptimus/H-optimus-1](https://huggingface.co/bioptimus/H-optimus-1), accept the terms using your **institutional email** (it must match your Hugging Face account email to be approved).
## 2. Load the model and extract features
H-Optimus-1 expects 224×224 images extracted at 0.5 µm/pixel.
```python theme={null}
from huggingface_hub import login
import torch
import timm
from torchvision import transforms
login() # token from https://huggingface.co/settings/tokens
model = timm.create_model(
"hf-hub:bioptimus/H-optimus-1",
pretrained=True, init_values=1e-5, dynamic_img_size=False,
)
model.to("cuda").eval()
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(
mean=(0.707223, 0.578729, 0.703617),
std=(0.211883, 0.230117, 0.177517),
),
])
tile = transforms.ToPILImage()(torch.rand(3, 224, 224))
with torch.autocast(device_type="cuda", dtype=torch.float16):
with torch.inference_mode():
features = model(transform(tile).unsqueeze(0).to("cuda"))
assert features.shape == (1, 1536)
```
Mixed precision (`autocast`) is recommended for faster inference. The normalization constants above are specific to H-Optimus-1 — use them as shown.
## When to use a different option
M-Optimus and commercial licensing are available via AWS Marketplace.
The Bioptimus SDK handles tiling and tissue masking against a deployed server.
# On-premise
Source: https://docs.bioptimus.com/deployment/platforms/on-premise
Load, run, and verify the Bioptimus Model Server container in your own environment.
The Bioptimus Model Server runs H-Optimus and M-Optimus entirely within your infrastructure — keeping data on your hardware for residency, compliance, or air-gapped requirements. It serves the same JSON API as the [SageMaker deployment](/deployment/platforms/aws-sagemaker), and the [Bioptimus SDK](/guides/get-started/sdk) works against both.
Two package variants are available. Each image contains the relevant model **plus** the tissue segmentation model:
| Package | Models | Key endpoints |
| - | - | - |
| **H1** | H-Optimus (`h1`), tissue-seg | `POST /api/embed/h1`, `POST /api/predict/tissue-seg` |
| **M** | M-Optimus (`m-optimus`), tissue-seg | `POST /api/embed/m-optimus`, `POST /api/predict/m-optimus`, `POST /api/predict/tissue-seg` |
## Prerequisites
| Requirement | Detail |
| - | - |
| Docker | Version 20.10 or later |
| NVIDIA Container Toolkit | Required for GPU inference ([install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)) |
| GPU | NVIDIA GPU with **CUDA Compute Capability 8.6** (e.g. A10G-class, equivalent to SageMaker `ml.g5.xlarge`). A CUDA GPU is **required** — the container exits if none is detected. |
| CPU architecture | **x86-64** — CC 8.6 + x86-64 runs out of the box; other architectures require recompilation. |
| GPU access | Direct passthrough (`--gpus all`) |
| OS | Linux with an NVIDIA GPU |
| Disk | Enough to hold the image (weights are baked in) |
The REST API currently requires **no authentication** — run the server on a private network and restrict access at the network layer. See the [API reference](/api-reference/introduction#authentication).
## 1. Load the container image
You receive the container as a compressed archive plus a `sha256` checksum. Verify integrity, then load it into Docker.
```bash theme={null}
sha256sum -c bioptimus-h1-onpremise-v1.0.1.sha256
docker load < bioptimus-h1-onpremise-v1.0.1.tar.gz
docker images bioptimus-h1-onpremise
```
```bash theme={null}
sha256sum -c bioptimus-m-onpremise-v1.0.1.sha256
docker load < bioptimus-m-onpremise-v1.0.1.tar.gz
docker images bioptimus-m-onpremise
```
## 2. Start the container
The server is self-contained — all weights and assets are baked into the image, so no volume mounts are required.
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
bioptimus-h1-onpremise:v1.0.1 serve
```
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
bioptimus-m-onpremise:v1.0.1 serve
```
Models load at startup. Follow the logs until the server is ready:
```bash theme={null}
docker logs -f bioptimus-server 2>&1 | grep -m1 "models ready"
```
## 3. Air-gapped install
For environments with no outbound network, transfer the `.tar.gz` archive via your approved process, then run `sha256sum -c`, `docker load`, and `docker run` exactly as above. No registry access is needed — the image is self-contained.
## 4. Verify the deployment
```bash H-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
```bash M-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["m-optimus", "tissue-seg"]}
```
A 503 `{"status": "loading"}` means models are still initialising — retry. See the [API reference](/api-reference/introduction) for all health states.
Open `http://localhost:8080/docs` for a Swagger UI to explore endpoints and try calls. Service discovery is at `http://localhost:8080/bioptimus/`.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
model = Backbone(Models.H1, base_url="http://localhost:8080")
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
model = Backbone(Models.M_OPTIMUS, base_url="http://localhost:8080")
```
For whole-slide inference, see the [Bioptimus SDK](/guides/get-started/sdk).
## Managing the container
```bash theme={null}
docker stop bioptimus-server # stop (preserves state)
docker start bioptimus-server # restart
docker logs bioptimus-server # view logs
docker rm -f bioptimus-server # remove
```
## Environment variables
Set with `-e` at start:
| Variable | Default | Description |
| - | - | - |
| `PORT` | `8080` | Server port |
| `LOG_LEVEL` | `INFO` | Log level (`DEBUG`, `INFO`, `WARNING`, `ERROR`) |
| `SM_MODEL_DIR` | `/opt/ml/model` | Model artefact directory (used by SageMaker) |
## Troubleshooting
Container not running or still starting. Check `docker ps` and `docker logs bioptimus-server`.
Models are still loading. Wait for the `models ready` log line.
The GPU is no longer available, or the CUDA context was corrupted. Check `nvidia-smi` on the host and restart the container.
Common causes: no CUDA GPU detected (a GPU is required), port conflict, or insufficient memory. Run `docker logs bioptimus-server` for the error.
The server auto-batches concurrent requests (max batch size 32). Reduce the number of concurrent SDK requests to lower peak memory use.
# Proscia
Source: https://docs.bioptimus.com/deployment/platforms/proscia
Generate H-Optimus embeddings from slides in your Concentriq® LS workspace with Proscia's Concentriq® Embeddings service.
[Proscia Concentriq® Embeddings](https://github.com/Proscia/proscia-ai-tools) extracts [H-Optimus](/documentation/models/h-optimus) embeddings from slides in a [Concentriq® LS](https://www.proscia.com/) workspace — no downloading, tiling, or self-hosted server required. It's the best fit when your slides already live in Concentriq® LS: Proscia tiles each slide, runs the model, and returns one safetensor file of embeddings per slide.
This integration is operated by Proscia. Bioptimus provides the H-Optimus foundation model; Proscia provides the Concentriq® platform, the embeddings service, and the [`proscia-ai-tools`](https://github.com/Proscia/proscia-ai-tools) client. For platform access, workspace provisioning, and support, contact Proscia.
## How it works
Your whole-slide images are stored in your Concentriq® LS workspace. Nothing is downloaded locally.
Using the `proscia-ai-tools` Python client, you point at one or more image repositories, pick the H-Optimus model tag, and submit an embedding job. The call returns a ticket ID.
Concentriq® Embeddings tiles each slide at your chosen magnification and runs H-Optimus on Proscia's infrastructure.
Fetch the results by ticket ID. Each slide comes back as a compact safetensor file of feature vectors that you load into memory for downstream tasks.
## 1. Install the client
Install the client directly from GitHub:
```bash theme={null}
pip install git+https://github.com/Proscia/proscia-ai-tools.git
```
## 2. Authenticate
The client connects to your Concentriq® workspace endpoint. It supports three authentication methods — provide exactly one:
```python Email + password theme={null}
from proscia_ai_tools.client import ClientWrapper as Client
client = Client(
url="https://.concentriq.proscia.com",
email="",
password="",
)
```
```python API key theme={null}
from proscia_ai_tools.client import ClientWrapper as Client
client = Client(
url="https://.concentriq.proscia.com",
api_key="",
)
```
```python JWT token theme={null}
from proscia_ai_tools.client import ClientWrapper as Client
client = Client(
url="https://.concentriq.proscia.com",
token="",
)
```
Your workspace URL and credentials are issued by Proscia. Keep them out of source control — load them from environment variables or a secrets manager rather than hard-coding them.
## 3. Generate embeddings with H-Optimus
Submit an embedding job against one or more Concentriq® image repositories, selecting the H-Optimus model tag and the microns-per-pixel (`mpp`) resolution. The call returns a ticket ID for retrieving the results.
```python theme={null}
# Submit an embedding job for one or more repositories.
ticket_id = client.embed_repos(
ids=[1234], # Concentriq® repository IDs
model="bioptimus/H-optimus-1", # H-Optimus foundation model tag
mpp=0.5, # target resolution (microns per pixel)
)
# Retrieve the embeddings once the job completes.
embeddings = client.get_embeddings(ticket_id)
```
The client also exposes `embed_images` (for specific image IDs) and `embed_roi` (for annotation- or region-based embeddings). See the [`proscia-ai-tools`](https://github.com/Proscia/proscia-ai-tools) client for the full API.
The model tag selects the foundation model Concentriq® Embeddings runs. Use the H-Optimus tag published by Proscia for your workspace; the exact set of available models is controlled by Proscia and may expand over time.
## 4. Explore end-to-end examples
Proscia maintains runnable notebooks for complete workflows — embedding generation, regions of interest, annotations, clustering, segmentation, and zero-shot classification:
Browse the `notebooks/` folder in `proscia-ai-tools` for end-to-end Concentriq® Embeddings workflows using H-Optimus.
## When to use a different option
If your slides live outside Concentriq® LS, deploy H-Optimus or M-Optimus via AWS, on-premise, or Hugging Face.
The Bioptimus SDK handles tiling, tissue masking, and output formats against a Bioptimus-deployed server.
# Requirements
Source: https://docs.bioptimus.com/deployment/requirements
Prerequisites for each deployment option.
Prerequisites depend on how you deploy:
An IAM role with `AmazonSageMakerFullAccess`, plus permission to subscribe to AWS Marketplace products.
An active subscription to the relevant model package ([H-Optimus](https://aws.amazon.com/marketplace/pp/prodview-cuad7l27fobx4) / M-Optimus) to obtain the Model Package ARN.
A SageMaker execution environment and sufficient quota for `ml.g5.xlarge` instances in your region. Outside SageMaker, set the execution role ARN explicitly (`get_execution_role()` is unavailable).
| Requirement | Detail |
| - | - |
| Docker | Version 20.10 or later |
| NVIDIA Container Toolkit | Required for GPU inference ([install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)) |
| GPU | NVIDIA GPU with **CUDA Compute Capability 8.6** (e.g. A10G-class). A 24 GB GPU — equivalent to SageMaker `ml.g5.xlarge` — is a good baseline. A CUDA GPU is **required** (container exits without one). |
| CPU architecture | **x86-64** (CC 8.6 + x86-64 runs out of the box; other architectures require recompilation) |
| GPU access | Direct passthrough via the NVIDIA Container Toolkit (`--gpus all`) |
| OS | Linux with an NVIDIA GPU |
| Disk | Enough to hold the image archive (weights are baked in) |
See the [on-premise guide](/deployment/platforms/on-premise) for the full install flow.
| Requirement | Detail |
| - | - |
| License | CC-BY-NC-ND 4.0 — **non-commercial academic research only** |
| Access | Gated; accept the terms on Hugging Face using an institutional email |
| Python | `timm`, `torch`, `torchvision`, `huggingface_hub`; a CUDA GPU for inference |
See the [Hugging Face guide](/deployment/platforms/hugging-face).
## Python SDK & dependencies
The [Bioptimus SDK](/guides/get-started/sdk) handles WSI reading, tiling, tissue masking, and concurrent dispatch for both the on-premise and SageMaker backends. The SageMaker reference notebooks pin:
```bash theme={null}
pip install sagemaker==2.254.1 boto3==1.42.2
```
# Validation & acceptance testing
Source: https://docs.bioptimus.com/deployment/validation
Verify a deployment produces correct, expected outputs.
Use these checks after deploying ([on-premise](/deployment/platforms/on-premise) or [SageMaker](/deployment/platforms/aws-sagemaker)) to confirm the service is healthy and producing expected results.
## 1. Service health
```bash H-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
```bash M-Optimus package theme={null}
curl -s http://localhost:8080/ping | python3 -m json.tool
# {"status": "ok", "models": ["m-optimus", "tissue-seg"]}
```
## 2. Model availability (SDK)
`available_backbones()` lists the models the Bioptimus SDK can construct (read from its bundled configs) — it does not query the server. The `/ping` response in step 1 is the source of truth for what the deployed endpoint actually serves.
```python theme={null}
from bioptimus.models.backbones import Backbone
print(Backbone.available_backbones())
# ['h1', 'm-optimus', 'tissue-seg']
```
## 3. Synthetic-tile verification (recommended)
One command runs steps 1–4 together: it sends a synthetic tile through the endpoint and checks that the request succeeds (auth and routing), that the output matches the model's declared contract (length, no NaN/infinity, correct tile echoed back), and reports an error code plus a fix for anything that fails.
```bash On-premise / remote theme={null}
bioptimus verify --model h1 --backend remote --base-url http://localhost:8080
# h1 on remote: ok (4 checks)
```
```bash SageMaker theme={null}
bioptimus verify --model m-optimus --backend aws \
--endpoint-name my-endpoint --region-name eu-north-1
```
```python SDK theme={null}
from bioptimus.observability.healthcheck import verify_endpoint
report = verify_endpoint("h1", "remote", base_url="http://localhost:8080")
print(report.summary())
for check in report.failures:
print(check.name, check.error_code, check.remediation)
```
The command exits non-zero when any check fails; add `--json` for machine-readable output in a deployment pipeline. It issues one real inference request.
## 4. Output shape checks
| Model | Expected output length |
| - | - |
| H-Optimus (`/api/embed/h1`) | 1536 |
| M-Optimus embedding | 1536 |
| M-Optimus prediction | number of output genes (see `/api/metadata/m-optimus`) |
| Tissue segmentation | H×W (262,144 for a 512×512 tile) |
# Welcome
Source: https://docs.bioptimus.com/documentation/introduction
Deploy and build with foundation models for biology — across histology, transcriptomics, and genomics.
Bioptimus builds foundation models that learn the dynamics of human biology across scales — from cell to tissue to organ. This documentation covers how to access, deploy, and build with our models on AWS and on-premise.
## How it works
Bioptimus models are pretrained on large, diverse histology data, so their representations transfer to many tasks without training from scratch. Point the SDK at a slide and a model, and it tiles the slide, masks out background, runs the model, and writes results to disk. For precise definitions, see the [Glossary](/documentation/resources/glossary).
A scanned H\&E slide can be **billions of pixels** — too large to process at once — so the pipeline starts from the slide and splits it into small tiles, each processed independently.
![H&E whole-slide image thumbnail]()
Slides are mostly background. The pipeline tiles coarsely (512×512 at 8 µm/pixel) and runs [tissue segmentation](/documentation/models/tissue-segmentation) to produce a tissue map, keeping only tissue-bearing tiles — cutting cost before the expensive feature step.
![Slide thumbnail alongside its binary tissue mask]()
Tissue tiles (224×224 at 0.5 µm/pixel) are embedded by [H-Optimus](/documentation/models/h-optimus) into a **1536-d feature vector** each. Principal components of these embeddings reveal the dominant axes of morphology — tumor, stroma, and immune compartments.
![Spatial heatmaps of the first three principal components of tile embeddings]()
[M-Optimus](/documentation/models/m-optimus) produces the same 1536-d embeddings **and** predicts spatial gene expression directly from each tile (optionally informed by bulk RNA) — a molecular readout without a spatial assay.
![Spatial gene-expression panel predicted from H&E by M-Optimus]()
## The models
A vision foundation model for histology. Extracts tile-level features from H\&E whole slide images.
A multimodal, multi-scale model (M-Optimus-1) that predicts spatial gene expression from routine H\&E, refined with bulk RNA.
Both models are trained on data from STELA, our data engine:
A multi-institutional data engine generating the deeply profiled, clinically linked patient data our models train on.
## Where to get each model
A quick map of which model is available on which channel, and where to start.
| Model | AWS SageMaker | On-premise | Hugging Face | Proscia |
| - | - | - | - | - |
| **H-Optimus** | ✅ [Deploy](/deployment/platforms/aws-sagemaker) | ✅ [Deploy](/deployment/platforms/on-premise) | ✅ [Academic](/deployment/platforms/hugging-face) | ✅ [Concentriq® LS](/deployment/platforms/proscia) |
| **M-Optimus** | ✅ [Deploy](/deployment/platforms/aws-sagemaker) | ✅ [Deploy](/deployment/platforms/on-premise) | — | — |
| **Tissue segmentation** | ✅ (bundled) | ✅ (bundled) | — | — |
Managed endpoints from AWS Marketplace.
Self-hosted container for full data control.
H-Optimus weights for non-commercial academic use.
H-Optimus embeddings for slides in Concentriq® LS.
## Where to start
Deploy a model and get embeddings back in minutes. For ML engineers and data scientists.
See how teams use our models for biomarker discovery, indication expansion, and trial design.
Data handling, residency, and deployment options for regulated environments.
Bioptimus models are for **research use** and are not approved medical devices. See [Responsible use](/documentation/resources/responsible-use) for intended use and your responsibilities.
# Benchmarks
Source: https://docs.bioptimus.com/documentation/models/benchmarks
Independent benchmark results for Bioptimus pathology models.
Bioptimus models are evaluated on independent, peer-reviewed benchmarks. The two most relevant for pathology foundation models are **PathBench** (HKUST) and **HEST** (Harvard; Jaume et al. 2025).
## PathBench
PathBench is a multi-task, multi-organ benchmark for pathology foundation models. Bioptimus reports H-Optimus-1 as the top-ranked model on the overall rank score (lower is better).
| Model | Overall rank score |
| - | - |
| **H-Optimus-1** | **6.06** |
| Virchow2 | 6.34 |
| H-Optimus-0 | 6.86 |
| UNI2 | 7.10 |
| mSTAR | 7.65 |
**Rankings are task- and organ-dependent.** H-Optimus-1 ranks first overall and leads on several organs (e.g. lung and colorectal); other models lead on specific tasks. Always link to the source so readers can see the full picture.
Sources: [PathBench paper (arXiv:2505.20202)](https://arxiv.org/abs/2505.20202) · [bioptimus.com](https://www.bioptimus.com)
## HEST
HEST (Harvard; Jaume et al. 2025) measures how well a model predicts **gene expression from histology** across nine organs. The metric is Pearson's correlation coefficient (higher is better). As of May 2026, H-Optimus-1 is the top-ranked image-only model on HEST, and M-Optimus-1 surpasses it by adding multimodal training.
| Model | HEST (avg. Pearson r) |
| - | - |
| **M-Optimus-1** | **0.440** |
| **H-Optimus-1** | **0.423** |
Figures are reported in the M-Optimus-1 report (M-Optimus-1 vs. H-Optimus-1 on HEST). The full per-model HEST leaderboard is maintained by the HEST authors — see the source for the complete table and methodology.
Bioptimus's write-up of the HEST evaluation and methodology.
## At scale
* 100+ papers published using Bioptimus models
* Used by 16 of the top 20 pharma companies
* 1,000+ clinical practices · 1M+ total model downloads
Peer-reviewed work and case studies in the Bioptimus Knowledge Base.
# H-Optimus
Source: https://docs.bioptimus.com/documentation/models/h-optimus
A vision foundation model for histology feature extraction.
H-Optimus is a vision transformer foundation model for histology. It extracts tile-level **feature vectors (embeddings)** from H\&E whole slide images — the input to downstream tasks such as mutation prediction, survival analysis, and tissue classification. H-Optimus produces features; it does not predict molecular signal directly (for that, see [M-Optimus](/documentation/models/m-optimus)).
**H-Optimus-1** is current — a 1.1B-parameter vision transformer (ViT-g/14) trained with self-supervised learning on billions of tiles from over 1 million slides of more than 800,000 patients, spanning 50+ organs, 3 scanner types, and 4,000+ clinical centers. As of May 2026 it ranks **#1** on the public PathBench leaderboard and is the top image-only model on HEST. **H-Optimus-0** (2024) is the previous generation, open-source under Apache 2.0. See [Benchmarks](/documentation/models/benchmarks) and the [Hugging Face model details](/deployment/platforms/hugging-face).
![Spatial heatmaps of the first three principal components of tile embeddings]()
## Specifications
| Property | Value |
| - | - |
| Task | Tile embedding |
| Input | 224×224 RGB tile at 0.5 µm/pixel |
| Output | 1536-dimensional feature vector |
| On-prem endpoint | `POST /api/embed/h1` |
| SageMaker dispatch | `model_name: "h1"`, `mode: "embedding"` |
| Recommended instance | `ml.g5.xlarge` |
## How it's used
Run [tissue segmentation](/documentation/models/tissue-segmentation) to keep tissue-bearing tiles.
Embed 224×224 tiles to get a 1536-d vector each (the [SDK](/guides/get-started/sdk) tiles and dispatches for you).
Aggregate embeddings for slide-level prediction, retrieval, or classification.
## Deploy
H-Optimus runs on [AWS SageMaker](/deployment/platforms/aws-sagemaker), [on-premise](/deployment/platforms/on-premise), [Hugging Face](/deployment/platforms/hugging-face) (academic), or via [Proscia](/deployment/platforms/proscia) for slides in a Concentriq® LS workspace. See the [Deployment overview](/deployment/overview) for the full comparison and setup steps.
## Guides
Extract embeddings and visualize morphology.
Run whole-slide inference with the Bioptimus SDK.
# M-Optimus
Source: https://docs.bioptimus.com/documentation/models/m-optimus
A multimodal, multi-scale foundation model that predicts spatial gene expression from histology.
M-Optimus is a **multimodal, multi-scale** foundation model. The current generation, **M-Optimus-1 (M1)**, learns across three biological layers at once — H\&E pathology, bulk RNA-seq, and spatial transcriptomics — to build a unified representation of a patient across tissue and molecular scales.
Its headline capability is **predicting spatial gene expression directly from a routine H\&E tile**, across up to 6,002 genes, optionally refined with bulk RNA-seq — recovering an expensive molecular readout from a low-cost slide. M-Optimus also produces the same 1536-d tile embeddings as [H-Optimus](/documentation/models/h-optimus). It is trained on proprietary multimodal cohorts, powered by the [STELA](https://www.bioptimus.com/stela) data engine.
## What makes it different
Where H-Optimus stops at features, M-Optimus adds a **prediction** head over a defined set of output genes. Provide a tile and (optionally) a bulk RNA vector, and it returns predicted expression per output gene — which you can render as spatial heatmaps. Because it is multimodal at both training and inference, it can ingest H\&E alone or H\&E + bulk RNA, with no retraining needed to benefit from the extra modality.
![EPCAM overlay predicted from H&E: with bulk RNA vs image only]()
## Performance
Results below are from the [M-Optimus-1 report](https://www.bioptimus.com); rankings and metrics are task- and dataset-dependent.
| Result | Finding |
| - | - |
| Spatial gene expression from H\&E | **+60%** vs. DeepSpot, a leading H\&E→ST model (\~30% from proprietary pretraining data, \~30% from the multimodal method) |
| Adding bulk RNA at inference | **+4%** over H\&E-only, with no retraining (late-binding) |
| HEST (gene expression from histology) | **0.440** avg. Pearson vs. **0.423** for H-Optimus-1 |
| Classification (mean AUC, 9 tasks) | **0.664** vs. **0.661** for H-Optimus-1 — matches the image-only SOTA while adding molecular prediction |
| Generalist vs. specialists | **+8%** on colon and head & neck vs. indication-specific models; strong zero-shot generalization to unseen tissues (e.g. kidney, skin) |
## Modes
* **Endpoint:** `POST /api/predict/m-optimus`
* **Input:** 224×224 tile at 0.5 µm/pixel, plus optional `bulk_rna` aligned to the model's input gene set. Omitting bulk RNA uses a zero vector (H\&E-only mode).
* **Output:** predicted expression for each output gene.
* Gene sets are available at `GET /api/metadata/m-optimus`.
* **Endpoint:** `POST /api/embed/m-optimus`
* **Input:** 224×224 tile at 0.5 µm/pixel.
* **Output:** 1536-d tile feature vector.
The [SDK](/guides/get-started/sdk) aligns and reorders your bulk RNA to the model's input gene set automatically. See [Add bulk RNA](/guides/workflows/spatial-transcriptomics#3-add-bulk-rna-multimodal-prediction) for the full input contract and transform pipeline.
## Specifications
| Property | Value |
| - | - |
| Tasks | Spatial gene-expression prediction (spot-level); tile embedding |
| Genes | Up to 6,002 (model's defined output set; see `GET /api/metadata/m-optimus`) |
| Input | 224×224 RGB tile at 0.5 µm/pixel (+ optional bulk RNA) |
| Bulk RNA input | TPM-normalized expression values, Ensembl gene IDs |
| Bulk RNA transforms | Server-side: `log1p` on input, `expm1` on output |
| Embedding | 1536-d |
| SageMaker dispatch | `model_name: "m-optimus"` |
| Recommended instance | `ml.g5.xlarge` |
## What you can build
M-Optimus turns routine slides into a molecular map for translational research:
Surface candidate biomarkers and molecular signatures from H\&E at scale.
Enrich and stratify cohorts; analyze legacy trial slides retrospectively.
## Access & deploy
M-Optimus is available now by request. [Contact Bioptimus](https://www.bioptimus.com/contact) to discuss access, then deploy on AWS SageMaker or on-premise — see the [Deployment overview](/deployment/overview) for setup steps.
M-Optimus is not available on Hugging Face. For academic-only H\&E feature extraction, use [H-Optimus on Hugging Face](/deployment/platforms/hugging-face).
## Guides
Predict spatial gene expression end to end.
Run a model over many slides.
# Tissue segmentation
Source: https://docs.bioptimus.com/documentation/models/tissue-segmentation
Separate tissue from background before feature extraction.
The tissue segmentation model identifies tissue regions in a slide, letting you discard background tiles before running [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus). It is bundled with both model packages.
## Specifications
| Property | Value |
| - | - |
| Task | Binary tissue mask |
| Input | 512×512 RGB tile at 8.0 µm/pixel |
| Output | Flattened binary mask of length 262,144 (512×512), values 0.0 or 1.0 |
| SageMaker dispatch | `model_name: "tissue-seg"`, `mode: "prediction"` |
| Recommended instance | `ml.g5.xlarge` |
Run segmentation at the coarse 8.0 µm/pixel resolution first; it is cheap relative to per-tile feature extraction at 0.5 µm/pixel and can dramatically reduce the number of tiles you embed.
## Reshaping the output
The endpoint returns a flat array. Reshape it to a 512×512 mask:
```python theme={null}
import numpy as np
mask = np.array(result["output"]).reshape(512, 512)
```
## Deploy
Tissue segmentation has no separate deployment — it is **bundled with both the H-Optimus and M-Optimus packages** and served from the same endpoint. Deploy either model (see the [Deployment overview](/deployment/overview)) and `tissue-seg` is available alongside it.
## Using it in a pipeline
In practice you rarely call this endpoint directly — the Bioptimus SDK's tissue mask provider generates and caches masks for you, then filters tiles before feature extraction or prediction. See [Spatial transcriptomics](/guides/workflows/spatial-transcriptomics) and the [Bioptimus SDK](/guides/get-started/sdk).
Once masks are generated, `bioptimus.utils.plot_slide_and_mask` renders the mask next to the slide thumbnail for a quick sanity check:
```python theme={null}
from bioptimus import utils
record = cohort[0]
utils.plot_slide_and_mask(record.wsi_path, record.mask_path, threshold=0.5)
```
![Slide thumbnail alongside its binary tissue mask]()
# Quickstart
Source: https://docs.bioptimus.com/documentation/quickstart
Pick your model and platform, then run your first inference.
Three steps: deploy a model, connect the Bioptimus SDK, and run your first inference.
Not sure which model? **M-Optimus-1** predicts spatial gene expression and also produces embeddings; **H-Optimus-1** is the lighter, feature-only model. See [M-Optimus](/documentation/models/m-optimus) and [H-Optimus](/documentation/models/h-optimus).
## 1. Deploy your model
Deploy on the platform that fits your use case — each guide has the full steps:
| Platform | Best for | Guide |
| - | - | - |
| **AWS SageMaker** | Production pipelines and scale | [Deploy →](/deployment/platforms/aws-sagemaker) |
| **On-premise** | Data residency, air-gapped sites | [Deploy →](/deployment/platforms/on-premise) |
| **Hugging Face** | Non-commercial academic, H-Optimus only | [Load weights →](/deployment/platforms/hugging-face) |
| **Proscia** | Slides already in a Concentriq® LS workspace | [Generate embeddings →](/deployment/platforms/proscia) |
We recommend a real-time endpoint on `ml.g5.xlarge` (or the on-premise container) — the models are compiled for that GPU architecture (NVIDIA A10G, CUDA Compute Capability 8.6).
## 2. Connect the Bioptimus SDK
Point the [Bioptimus SDK](/guides/get-started/sdk) at your deployment (use `Models.H1` or `Models.M_OPTIMUS`):
```python On-premise theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url="http://localhost:8080")
```
```python AWS SageMaker theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
**Academic (Hugging Face) users** load H-Optimus-1 directly with `timm` instead of the Bioptimus SDK — see [Hugging Face](/deployment/platforms/hugging-face).
## 3. Get your first embedding
Send a single 224×224 tile through the connected model to confirm the full path works:
```python H-Optimus theme={null}
import numpy as np
from PIL import Image
from bioptimus.inference.schemas import ModelRequest
# Replace the blank tile with a real 224×224 H&E tile at 0.5 µm/pixel.
tile = Image.fromarray(np.zeros((224, 224, 3), dtype=np.uint8))
request = ModelRequest(image_data=tile, slide_name="demo",
x=0, y=0, width=224, height=224, patch_idx=0)
response = model.embed(request)
print(len(response.output)) # 1536
```
```python M-Optimus theme={null}
import numpy as np
from PIL import Image
from bioptimus.inference.schemas import ModelRequest
# Replace the blank tile with a real 224×224 H&E tile at 0.5 µm/pixel.
tile = Image.fromarray(np.zeros((224, 224, 3), dtype=np.uint8))
request = ModelRequest(image_data=tile, slide_name="demo",
x=0, y=0, width=224, height=224, patch_idx=0)
response = model.predict(request) # spatial gene expression for the tile
print(len(response.output)) # one value per output gene
```
For whole-slide inference — tiling, tissue masking, and writing results — see the [Bioptimus SDK](/guides/get-started/sdk).
## Next steps
Whole-slide inference: tiling, tissue masking, bulk RNA, and output formats.
End-to-end examples: [`h1-jumpstart`](https://github.com/bioptimus/h1-jumpstart) (H-Optimus) and [`m-jumpstart`](https://github.com/bioptimus/m-jumpstart) (M-Optimus).
# FAQ
Source: https://docs.bioptimus.com/documentation/resources/faq
Common questions about deploying and using Bioptimus models.
## Getting started
**M-Optimus** predicts spatial gene expression and also produces embeddings; **H-Optimus** is the lighter, feature-only model; run **tissue segmentation** first to filter background. See [M-Optimus](/documentation/models/m-optimus) and [H-Optimus](/documentation/models/h-optimus).
224×224 tiles at 0.5 µm/pixel for embeddings and M-Optimus prediction; 512×512 at 8.0 µm/pixel for tissue segmentation.
## Deployment & infrastructure
Two ways: an **AWS SageMaker endpoint**, or a **containerized Docker image for on-premise**. Both serve the same API. See [Deployment](/deployment/overview).
The on-premise server ships as a Docker image loaded via `docker load` and run with the NVIDIA Container Toolkit.
*TBD — the documented deployment is a single Docker container; confirm orchestration support.*
**x86-64** with an NVIDIA GPU of **CUDA Compute Capability 8.6** (e.g. A10G-class) runs the container out of the box. Other architectures require recompilation. A 24 GB GPU (equivalent to SageMaker `ml.g5.xlarge`) is a good baseline.
GPU access is direct passthrough via the NVIDIA Container Toolkit (`--gpus all`).
## Data security & privacy
Data stays entirely in your own environment (your AWS account, or your on-prem hardware). **Bioptimus does not see your inputs, outputs, or usage.** See [Security & compliance](/documentation/resources/security-compliance).
Indicative (to be confirmed): disease area / task, slide volumes, functions called, and crash/bug signals — no slide images or patient data.
## Model & SDK usage
No. H-Optimus returns the **CLS token** of size **1536**; M-Optimus returns its MLP output. The embedding type is fixed.
A 1536-dimensional float vector per tile (H-Optimus and M-Optimus embedding mode). See the [API reference](/api-reference/introduction).
Customer-managed **SageMaker** endpoints and **on-premise** containers; H-Optimus is also on **Hugging Face** for academic use. For slides already in a Concentriq® LS workspace, H-Optimus embeddings are available via **[Proscia](/deployment/platforms/proscia)** (operated by Proscia).
Install the provided Python wheel (or via a package index once confirmed). See the [Bioptimus SDK](/guides/get-started/sdk).
## Commercial & support
Three tiers: **Tier 1 — Co-Designed**, **Tier 2 — Supported**, and **Tier 3 — Self-Service**.
*TBD — contact Bioptimus to discuss evaluation options.*
Delete SageMaker endpoints and batch models when finished — endpoints bill while running.
# Glossary
Source: https://docs.bioptimus.com/documentation/resources/glossary
Key terms used across Bioptimus documentation.
A quick reference for technical and non-technical readers.
## Products & interfaces
| Term | Meaning |
| - | - |
| **Bioptimus SDK** | The Python client package. Handles WSI reading, tiling, tissue masking, bulk RNA alignment, and concurrent dispatch, against either the on-premise server or a SageMaker endpoint. See the [Bioptimus SDK](/guides/get-started/sdk). |
| **`Inference` (pipeline)** | The high-level, one-object pipeline (config + `tissue()`/`embed()`/`predict()`/`run()`/`save_config()`), with cached masks, a structured workspace, and reproducible config. Its `api_url` argument is the on-premise server URL — the same value as `Backbone`'s `base_url`. See the [Inference pipeline guide](/guides/get-started/inference-pipeline). |
| **`SlideInference`** | The lower-level, per-slide inference class (`Backbone` + mask provider + writer) for explicit control. |
| **`Cohort`** | A batch manifest of slides (and optional bulk RNA) that is the single source of truth for an experiment. Supports late-binding bulk RNA. |
| **`AWSClient`** | The low-level per-tile client for a SageMaker endpoint; injects `model_name`/`mode` into each request. |
| **`OutputFormat`** | The output writer format — Zarr (default), HDF5, or NPZ. |
| **Model Server / API** | The FastAPI inference server (shipped in the on-premise container) that exposes the REST endpoints (`/api/embed/h1`, `/api/predict/m-optimus`, …) and a Swagger UI at `/docs`. See the [API reference](/api-reference/introduction). |
| **Backbone** | The Bioptimus SDK factory class used to obtain a model client, e.g. `Backbone(Models.H1, backend="remote", base_url=...)` or `backend="aws"`. Takes a `Models` enum member (`Models.H1`, `Models.M_OPTIMUS`); the companion tissue model uses the string `"tissue-seg"`. |
| **Inference endpoint** | A running model service that accepts tile requests and returns outputs — an on-premise container or a SageMaker endpoint. |
| **SageMaker** | AWS's managed ML hosting service used for Bioptimus cloud deployment; the Bioptimus SDK reaches it via the `/invocations` dispatch endpoint. |
| **Model package** | The on-premise container variant — **H1** (H-Optimus + tissue-seg) or **M** (M-Optimus + tissue-seg). |
## Model & backend identifiers
The same model is referred to by a product name, a version, an SDK enum, and a server/endpoint id depending on context. This table is the canonical mapping.
| Model | Current version | SDK identifier | Server id | Endpoints | Hugging Face |
| - | - | - | - | - | - |
| **H-Optimus** | H-Optimus-1 | `Models.H1` (`"h1"`) | `h1` | `/api/embed/h1` | `bioptimus/H-optimus-1` |
| **M-Optimus** | M-Optimus-1 | `Models.M_OPTIMUS` (`"m-optimus"`) | `m-optimus` | `/api/embed/m-optimus`, `/api/predict/m-optimus` | — |
| **Tissue segmentation** | — | `"tissue-seg"` (string) | `tissue-seg` | `/api/predict/tissue-seg` | — |
The Bioptimus SDK reaches a deployment through a `backend` argument. The product term and the code value differ:
| Platform (product term) | SDK `backend` | Key connection args |
| - | - | - |
| On-premise container | `"remote"` | `base_url` (`api_url` on the `Inference` pipeline) |
| AWS SageMaker | `"aws"` | `endpoint_name`, `region_name` |
| In-process (local GPU, no server) | `"local"` | `model_dir` / `checkpoint` |
## Models & outputs
| Term | Meaning |
| - | - |
| **Foundation model (FM)** | A large model pre-trained on vast data that produces general-purpose representations reused across many downstream tasks. |
| **H-Optimus** | Bioptimus's histology foundation model; outputs tile embeddings. Current version: H-Optimus-1. |
| **M-Optimus** | Bioptimus's multimodal model; predicts spatial gene expression from histology (and optional bulk RNA), and also outputs embeddings. |
| **Embedding** | A numeric feature vector summarizing a tile. H-Optimus returns the **CLS token** of size **1536**. |
| **1536-d** | The dimensionality of H-Optimus and M-Optimus tile embeddings — a 1536-number feature vector per tile. |
| **CLS token** | The transformer's summary output used as the tile embedding. H-Optimus returns the 1536-d CLS token; M-Optimus returns its MLP output. The embedding type is fixed (not user-selectable). |
| **Spatial gene expression** | Gene expression mapped to locations across a slide. M-Optimus predicts this from H\&E tiles. |
| **Tissue segmentation (`tissue-seg`)** | A companion model (bundled in both packages) that produces a binary tissue/background mask. |
## Imaging & data
| Term | Meaning |
| - | - |
| **WSI (whole slide image)** | A digitized microscope slide — often gigapixel-scale — split into tiles for processing. |
| **H\&E** | Hematoxylin and eosin, the standard tissue stain in routine pathology. |
| **Tile (patch)** | A fixed-size crop of a WSI. Embeddings use 224×224; tissue segmentation uses 512×512. |
| **MPP (microns per pixel)** | Physical image resolution. Embeddings use 0.5 µm/pixel; tissue segmentation uses 8.0 µm/pixel. |
| **Bulk RNA-seq** | Aggregate (non-spatial) gene expression for a sample. Optional input to M-Optimus prediction. |
| **Ensembl gene ID** | Standardized gene identifier (e.g. `ENSG00000000003`) used in bulk RNA inputs and gene-set metadata. |
| **Zarr / HDF5 / NPZ** | Output file formats the Bioptimus SDK writes per-slide results to. |
## Infrastructure
| Term | Meaning |
| - | - |
| **CUDA Compute Capability** | An NVIDIA GPU architecture version. Compute Capability **8.6** on **x86-64** runs the container out of the box; other architectures require recompilation. |
| **NVIDIA Container Toolkit** | Enables GPU access inside Docker containers (`--gpus all`). |
| **`ml.g5.xlarge`** | The recommended SageMaker instance (a single 24 GB A10G-class GPU). |
## Benchmarks
| Term | Meaning |
| - | - |
| **PathBench** | A multi-task, multi-organ pathology FM benchmark (HKUST). H-Optimus-1 ranks first overall. |
| **HEST** | A benchmark for predicting gene expression from histology (Harvard; Jaume et al. 2025), scored by Pearson correlation. |
| **MIL (multiple instance learning)** | A method that aggregates many tile-level features into a slide-level prediction. |
# Responsible use
Source: https://docs.bioptimus.com/documentation/resources/responsible-use
Intended use, regulatory status, license terms, responsibilities, and disclaimer.
## Intended use
Bioptimus models are intended solely for non-clinical research use, including research workflows in biomarker discovery, spatial biology, and trial design.
They are not intended for, and must not be used for:
* clinical diagnosis;
* medical decision-making or patient management;
* patient treatment;
* development or use as a medical device, companion diagnostic, or other regulated product;
* any clinical, diagnostic, therapeutic, or other regulated use.
## Regulatory status
Bioptimus models have not been reviewed, approved, cleared, certified, or authorized by any regulatory authority for clinical, diagnostic, therapeutic, or other regulated use.
Bioptimus makes no representation or warranty that the models comply with requirements applicable to regulated products or uses in any jurisdiction.
## Your responsibilities
By accessing or using Bioptimus models, you are responsible for:
* complying with all applicable laws, regulations, and industry standards, including those relating to privacy, data protection, patient data, and healthcare or life sciences regulation;
* ensuring that your use of the models is permitted under your applicable license or commercial agreement with Bioptimus;
* independently evaluating and validating model outputs for your specific use case before relying on them.
## License terms
The terms governing your use of Bioptimus models depend on how you access them:
* **Via Hugging Face** — H-Optimus-1 is distributed under [CC-BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial academic use only. Commercial use under this license is not permitted.
* **Via Bioptimus API or platform** — use is governed by your commercial license agreement with Bioptimus.
## Disclaimer
Bioptimus models are provided "as is" and "as available," without warranties of any kind, express or implied, including any implied warranties of merchantability, fitness for a particular purpose, non-infringement, accuracy, or reliability.
Bioptimus does not warrant that the models will be uninterrupted, error-free, secure, or suitable for any particular use, nor that any outputs will be accurate, complete, or fit for your purposes.
To the maximum extent permitted by applicable law, Bioptimus disclaims liability for any losses, claims, damages, or decisions — including indirect, consequential, or incidental damages — arising out of or related to use of the models or their outputs.
## Data attribution
The example figures throughout this documentation were generated from **open-access [TCGA](https://www.cancer.gov/tcga) data** (whole-slide images and RNA-seq) obtained from the [NCI Genomic Data Commons](https://portal.gdc.cancer.gov/). TCGA data is open access, individually non-identifiable, and carries no restrictions on use in publications or presentations. In line with TCGA's request, we acknowledge:
> The results shown here are in whole or part based upon data generated by the TCGA Research Network: [https://www.cancer.gov/tcga](https://www.cancer.gov/tcga).
The figures are **real model outputs** on individual public slides (e.g. TCGA-LUAD case `TCGA-75-7027`), shown as representative examples of model behavior. They are not performance benchmarks, and outputs vary by slide, tissue type, and model variant.
# Drug target & biomarker discovery
Source: https://docs.bioptimus.com/documentation/use-cases/biomarker-discovery
Predict drug target distribution, immune microenvironment, and stromal architecture from routine histology slides.
Routine H\&E slides carry far more signal than the standard read extracts. Bioptimus models predict the spatial distribution of drug targets, the immune microenvironment, and stromal architecture directly from slides you already have — no new assays required to begin.
## What it enables
* Turn existing H\&E archives into a discovery substrate for targets and biomarkers
* Characterize the tumor microenvironment spatially, not just globally
* Generate hypotheses before committing to expensive spatial or molecular assays
## How it works
Run [H-Optimus](/documentation/models/h-optimus) over slide tiles to produce tile-level embeddings, or [M-Optimus](/documentation/models/m-optimus) when molecular data is available.
Train lightweight downstream models on the embeddings to predict target expression, microenvironment features, or architecture.
Use tile coordinates to render spatial maps of the predicted signal.
Deploy a model and extract your first features.
# Diagnostics & spatial biology
Source: https://docs.bioptimus.com/documentation/use-cases/diagnostics-spatial-biology
Integrate histology, spatial transcriptomics, and genomics in a single pipeline.
The standard workup leaves signal on the table. Bioptimus models integrate histology with spatial transcriptomics and genomics in one pipeline, reaching resolution the routine read does not.
## What it enables
* Combine modalities that are usually analyzed in isolation
* Surface clinically actionable biology at higher resolution
* Streamline multimodal workflows into a single feature pipeline
## How it works
Use [M-Optimus](/documentation/models/m-optimus) to jointly represent histology and molecular inputs.
Feed slides and molecular vectors through one inference path (see the [request schema](/api-reference/introduction)).
Map predictions back to slide coordinates for spatial interpretation.
# Indication expansion
Source: https://docs.bioptimus.com/documentation/use-cases/indication-expansion
Identify the patient subsets and indications where a therapeutic is most likely to succeed.
Your drug works — but for which patients, and in which other diseases? Bioptimus models score archival cohorts across disease, stage, and mechanism of action to surface the patient subsets whose spatial biology matches what your drug needs.
## What it enables
* Screen thousands of archived slides from completed trials as a biological engine
* Prioritize new indications grounded in spatial and molecular biology
* Focus enrollment on subsets most likely to respond
## How it works
Run [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus) across archived slides at scale with [cohorts](/guides/workflows/cohort).
Build downstream classifiers that score cohorts by disease, stage, and mechanism of action.
Identify patient subsets and indications with the strongest signal.
# Treatment response & trial design
Source: https://docs.bioptimus.com/documentation/use-cases/treatment-response
Discover multimodal biomarker signatures that distinguish responders from non-responders.
Two patients, same diagnosis, same standard biomarkers — different spatial biology and different outcomes. Bioptimus models discover multimodal signatures that separate responders from non-responders, even from the small cohorts typical of early-phase trials.
## What it enables
* Build response-predictive signatures from limited early-phase data
* Support patient enrichment strategies before late-stage commitment
* Inform trial design with biology the standard biomarker panel misses
## How it works
Extract features with [H-Optimus](/documentation/models/h-optimus) or [M-Optimus](/documentation/models/m-optimus).
Train downstream models that distinguish responders from non-responders.
Use the signature to inform enrollment and trial design decisions.
# Inference pipeline
Source: https://docs.bioptimus.com/guides/get-started/inference-pipeline
Configure the whole pipeline with one object — tissue masking, embeddings, predictions, and reproducible workspaces.
The `Inference` pipeline replaces the multi-step manual pipeline (`Backbone` + mask provider + `SlideInference` + writer) with a single, reusable object. The constructor is the pipeline config; the methods do the work. It caches tissue masks, organizes outputs into a structured workspace, and can serialize its config for reproducibility.
## Configure the pipeline
Pick your backend and model — the rest of the guide follows the same steps.
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.H1,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="h1-embedding-demo",
run=1,
description="H1 embeddings with tissue masking",
)
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.M_OPTIMUS,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="m-optimus-demo",
run=1,
description="M-Optimus gene expression with tissue masking",
)
```
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.H1,
backend="aws",
endpoint_name="h-optimus",
region_name="us-east-1",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="h1-embedding-demo",
run=1,
workers=1, # ml.g5.xlarge has a single GPU
description="H1 embeddings with tissue masking",
)
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.M_OPTIMUS,
backend="aws",
endpoint_name="m-optimus",
region_name="us-east-1",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="m-optimus-demo",
run=1,
workers=1, # ml.g5.xlarge has a single GPU
description="M-Optimus gene expression with tissue masking",
)
```
## Run the pipeline
```python H-Optimus theme={null}
infer.tissue(wsi_path) # generate (or load cached) tissue mask
result_path = infer.embed(wsi_path) # -> /h1/embeddings/.zarr
```
```python M-Optimus theme={null}
infer.tissue(wsi_path) # generate (or load cached) tissue mask
result_path = infer.predict(wsi_path) # -> /m-optimus/predictions/.zarr
```
The tissue mask is cached to `/tissue/.png` and reused automatically on later calls.
If the first call raises a connection error, the on-premise server behind `api_url` isn't reachable yet. Verify it with `requests.get(f"{api_url}/ping", timeout=5)` (a healthy server returns `{"status": "ok", ...}`) and see [Common issues](/guides/get-started/troubleshooting) for how to start it.
## Reproducibility
The config is auto-saved to `/config.yaml` on first use. Reconstruct the exact pipeline later:
```python theme={null}
infer.save_config()
infer2 = Inference.from_workspace(workspace_dir) # same workspace; cached masks reused
```
The same object works across multiple slides — call `infer.embed([s1, s2])` (or `infer.predict([s1, s2])` for M-Optimus) rather than creating a new `Inference` per slide. For full cohorts with bulk RNA, see [Cohorts](/guides/workflows/cohort).
## Workspaces & variants
Your workspace is the directory where all outputs are written. It's resolved from the config you pass:
```text theme={null}
//run_//
```
Only `output_path` is required; `experiment`, `run`, and `variant` are appended when set:
| Segment | Config field | Appended when |
| - | - | - |
| `` | `output_path` | always (required) |
| `` | `experiment` | set |
| `run_` | `run` | set |
| `` | `variant` | set |
`variant` (a string) adds a final sub-folder so you can keep multiple runs side by side — useful for experiments (e.g. comparing mask thresholds or model versions) without overwriting earlier outputs:
```python theme={null}
infer = Inference(model_name=Models.H1, api_url="http://localhost:8080",
output_path="/data/output", experiment="tcga_coad", run=1,
variant="mask_0.5")
# -> /data/output/tcga_coad/run_1/mask_0.5/
```
A populated workspace looks like:
```text H-Optimus theme={null}
//run_//
config.yaml
manifest.yaml
tissue/.png
h1/embeddings/.zarr
```
```text M-Optimus theme={null}
//run_//
config.yaml
manifest.yaml
tissue/.png
m-optimus/predictions/.zarr
```
# Logging
Source: https://docs.bioptimus.com/guides/get-started/logging
Structured logging with zero setup: where logs go, the default format, and how to customize them.
The SDK logs every failure once, with a machine-readable error code and full context, to a local JSON file — no setup required. This page covers where logs go, what a line looks like, and how to customize them. To package logs into something you can send to Bioptimus support, see [Support bundles](/guides/get-started/support-bundles).
## What the logging layer gives you
Logging is a small, dependency-light layer with a handful of distinct jobs. Each has its own section below — start with whichever capability you need.
| You want to… | Use | Section |
| - | - | - |
| Keep a durable JSON record of every failure, with zero setup | *automatic on first log* | [Zero setup](#zero-setup) |
| Send logs to a file, the console, a custom stream, or your app's handlers | `configure_logging`, `BIOPTIMUS_LOG_FILE` | [Log destinations](#log-destinations) |
| Know exactly what a log line contains | `JsonFormatter` output | [Default log format](#default-log-format) |
| Emit your own structured events, or record every failure of your functions | `log_event`, `log_failures` | [Emit your own log entries](#emit-your-own-log-entries) |
| Tag every line of a run / slide / request with shared correlation IDs | `bind_context`, `new_trace_id` | [Group related log lines](#group-related-log-lines) |
| Read logs back as plain dicts, without parsing JSON | `read_recent_logs`, `read_usage_records` | [Read your logs back](#read-your-logs-back-in-python) |
| Change level, rotation interval, retention, or destination | `configure_logging` | [Customize logging](#customize-logging) |
| Emit SDK logs in your own format, to your own sink | your own `logging` handler (records propagate) | [Can I change the log format?](#can-i-change-the-log-format) |
| Redact PII in the log file itself, not just on export | `configure_logging(anonymize=True)` | [Redact sensitive fields at rest](#redact-sensitive-fields-at-rest) |
| Catch typed, coded errors in your own code | `BioptimusError`, `ErrorCode` | [Handle errors in your code](#handle-errors-in-your-code) |
Every helper on this page imports from the package root:
```python theme={null}
from bioptimus.observability import (
configure_logging, default_log_file, get_log_file,
log_event, log_failures,
bind_context, get_context, new_trace_id,
read_recent_logs, read_usage_records,
BioptimusError, ErrorCode,
)
```
## Zero setup
Logs rotate daily and are kept for **14 days** by default. Change this with `configure_logging(backup_count=...)`.
The first emitted event auto-installs a daily-rotating JSON log file under the system temp directory — nothing to configure.
```python theme={null}
from bioptimus.observability import default_log_file, get_log_file
print(default_log_file()) # the zero-setup fallback, e.g. /tmp/bioptimus.log
print(get_log_file()) # the path actually in use now — honors BIOPTIMUS_LOG_FILE and
# configure_logging(); None when file logging is disabled
```
On first use, if logging falls back to the temp directory, the SDK emits a single `log_file_in_temp_dir` warning — that location can be wiped on reboot or by a temp-file cleaner. Point logging at a durable path (see [Customize logging](#customize-logging)) to silence it and keep a lasting record.
The log file is **not anonymized at rest** — it keeps full detail so you can debug locally, and it never contains slide images, tile pixels, or omics values, only pipeline metadata (paths, IDs, error codes, tracebacks). Anonymization of that metadata is applied only when you **export** a support bundle or query `/diagnostics` (see [Support bundles](/guides/get-started/support-bundles)).
## Log destinations
| Destination | How | Notes |
| - | - | - |
| File (default) | Nothing — zero setup | Writes to `default_log_file()`, under the system temp directory |
| A different file | `BIOPTIMUS_LOG_FILE=/path/to/file.log` or `configure_logging(log_file="/path/to/file.log")` | The env var also applies to the `bioptimus` CLI |
| Console (stderr) | `configure_logging(console=True)` | Human-readable, one line per event |
| A custom stream | `configure_logging(stream=my_stream)` | JSON lines; implies `console=True` |
| Disabled | `BIOPTIMUS_LOG_FILE=` (empty) | No file sink; records still propagate to your own handlers |
```bash theme={null}
export BIOPTIMUS_LOG_FILE=/var/log/bioptimus.log # custom path
export BIOPTIMUS_LOG_FILE= # disable file logging
```
## Default log format
Every line is a single JSON object:
```json theme={null}
{
"timestamp": "2026-09-08T12:00:00.123456+00:00",
"level": "ERROR",
"event": "wsi_open_failed",
"logger": "bioptimus.io.wsi",
"error_code": "BIO-WSI-002",
"remediation": "Verify the slide path points to a valid, non-truncated, readable WSI file.",
"trace_id": "01J9Z8Q3F9K2M4N6P8Q0R2S4T6",
"run_id": "01J9Z8Q3F9K2M4N6P8Q0R2S4T7",
"exception": "Traceback (most recent call last): ..."
}
```
| Field | Always present | Meaning |
| - | - | - |
| `timestamp` | Yes | ISO 8601, UTC |
| `level` | Yes | `INFO`, `WARNING`, `ERROR`, ... |
| `event` | Yes | Stable event slug, e.g. `wsi_open_failed` |
| `logger` | Yes | Python logger name that emitted the record |
| `error_code` | On a `BioptimusError` | One of the codes in [Error codes](#error-codes) below |
| `remediation` | When the code has one | The fix for that code |
| `trace_id` / `run_id` / `slide_id` / `stage` | When bound via `bind_context` | Correlation fields propagated down the pipeline |
| `exception` / `stack` | On a failure | Full traceback |
## Emit your own log entries
Wrap your own pipeline functions to get the same one-record-per-failure behavior with `log_failures`, or emit a one-off structured event with `log_event` — both write to the log file that's already active, no [`configure_logging`](#customize-logging) call required (see [Zero setup](#zero-setup)). In every example below, `run_slide` is *your* function and `inference` is a client you built earlier (for example `inference = Inference(...)` or `model = Backbone(...)`) — swap in your own call:
```python Decorator theme={null}
from bioptimus.observability import log_failures
# Records any exception once (with its error code and traceback), then re-raises it.
@log_failures
def run_slide(path: str):
return inference.predict(path)
```
```python Async theme={null}
from bioptimus.observability import log_failures
# log_failures auto-detects async functions — no separate decorator needed.
@log_failures
async def run_slide(path: str):
return await inference.predict_async(path)
```
```python Custom logger/level theme={null}
import logging
from bioptimus.observability import log_failures
logger = logging.getLogger(__name__)
# Log on your own logger, and as a WARNING instead of the default ERROR.
@log_failures(logger=logger, level=logging.WARNING)
def run_slide(path: str):
return inference.predict(path)
```
```python Manual event theme={null}
import logging
from bioptimus.observability import log_event
logger = logging.getLogger(__name__)
# Emit a single structured line yourself. Extra keyword args become JSON fields.
log_event(
logger,
logging.WARNING,
"slide_skipped", # the event slug (becomes the "event" field)
reason="tissue_mask_empty", # any extra fields you want on the line
slide_id="s1234",
)
```
Each path writes one JSON line to the log file. A failure caught by `log_failures` looks like:
```json theme={null}
{
"timestamp": "2026-09-08T12:00:00.123456+00:00",
"level": "ERROR",
"event": "wsi_open_failed",
"logger": "__main__",
"error_code": "BIO-WSI-002",
"remediation": "Verify the slide path points to a valid, non-truncated, readable WSI file.",
"operation": "run_slide",
"exception": "Traceback (most recent call last): ..."
}
```
and the manual `log_event` call above writes:
```json theme={null}
{
"timestamp": "2026-09-08T12:00:00.123456+00:00",
"level": "WARNING",
"event": "slide_skipped",
"logger": "__main__",
"reason": "tissue_mask_empty",
"slide_id": "s1234"
}
```
`log_event` also accepts `error_code=` — which auto-attaches the matching `remediation` — and `exc_info=exc` to record a caught exception's traceback on the line.
## Group related log lines
Bind correlation fields once and every log line emitted inside the block is tagged with them automatically — so all lines for a run, a slide, or a request share a `trace_id`/`slide_id` you can filter on. Fields whose value is `None` are ignored, and the previous context is restored on exit (even on error):
```python theme={null}
from bioptimus.observability import bind_context, get_context, new_trace_id
with bind_context(trace_id=new_trace_id(), slide_id="TCGA-AA-1234"):
run_slide("slide.svs") # every log line inside carries trace_id + slide_id
print(get_context()) # {'trace_id': '01J9Z8...', 'slide_id': 'TCGA-AA-1234'}
```
`new_trace_id()` returns a 26-character, time-sortable ID. `bind_context` accepts any field name — `trace_id`, `run_id`, `slide_id`, and `stage` are the ones the SDK's own pipeline sets and surface in the format table above, but you can bind your own (for example `batch_id`) the same way.
## Read your logs back in Python
Read the log file without parsing JSON yourself. Both readers anonymize by default — pass `anonymize=False` for full local detail:
```python theme={null}
from bioptimus.observability import read_recent_logs, read_usage_records
recent = read_recent_logs(max_entries=50) # last N entries of the active file, oldest first
usage = read_usage_records() # every model-request record, across rotations
```
Each entry is a plain `dict` with the fields shown in [Default log format](#default-log-format). `read_recent_logs` tails the active file only; `read_usage_records` spans the active file and all its rotations, so it is the full request history rather than a recent tail.
## Customize logging
Call `configure_logging` to override any default — for a custom path, console output, level, or retention:
```python theme={null}
from pathlib import Path
from bioptimus.observability import configure_logging
configure_logging(console=True, level="DEBUG", log_file=Path("~/.bioptimus/logs/bioptimus.log").expanduser())
```
With `console=True`, matching events also print a compact, human-readable line to `stderr` instead of the JSON payload above — for example the `slide_skipped` event from the manual `log_event` call:
```text theme={null}
12:00:00 WARNING slide_skipped reason=tissue_mask_empty slide_id=s1234
```
| Parameter | Purpose | Default |
| - | - | - |
| `level` | Minimum severity to emit | `logging.INFO` |
| `log_file` | Explicit file path, overrides `BIOPTIMUS_LOG_FILE` | The env var, else `default_log_file()` |
| `console` | Also print human-readable lines to `stderr` | `False` |
| `stream` | Custom stream for console output (implies `console=True`) | `None` |
| `anonymize` | Redact PII at rest, in the file itself | `False` |
| `when` | Rotation interval (`"midnight"`, `"H"`, `"D"`, ...) | `"midnight"` |
| `backup_count` | Number of rotated files kept | `14` |
| `file_handler_factory` | Swap in a different handler, e.g. size-based rotation | `None` |
| `logger_name` | Logger to configure (`None` targets the root logger) | `"bioptimus"` |
**Set the rotation period and retention** with `when` and `backup_count`. Rotated files are suffixed with their UTC date (`bioptimus.log.2026-06-24`), so each maps cleanly to a support window:
```python theme={null}
configure_logging(when="H", backup_count=48) # rotate hourly, keep two days
configure_logging(backup_count=0) # keep every rotated file
```
For a policy those two parameters can't express — size-based rotation, say — pass your own handler through `file_handler_factory`. The SDK still owns the path, the JSON format, and correlation-context injection:
```python theme={null}
from logging.handlers import RotatingFileHandler
configure_logging(
file_handler_factory=lambda path: RotatingFileHandler(
path, maxBytes=50_000_000, backupCount=5, encoding="utf-8"
),
)
```
### Can I change the log format?
The on-disk format is **structured JSON by design** — it is the contract the [readers](#read-your-logs-back-in-python), [support bundles](/guides/get-started/support-bundles), and a server's `/diagnostics` all parse — so there is no `format` parameter. For human-readable output, pass `console=True` (shown above).
To emit SDK logs in your own format to your own destination, attach your own handler. SDK records propagate up Python's logging hierarchy, so a handler on the `bioptimus` logger (or the root logger) receives every event:
```python theme={null}
import logging
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
logging.getLogger("bioptimus").addHandler(handler)
```
## Redact sensitive fields at rest
By default the log file keeps **full detail** so you can debug locally, and PII is redacted only when you **export** — when you build a support bundle or query a server's `/diagnostics`. To redact the on-disk file itself, turn on redaction at rest:
```python theme={null}
from bioptimus.observability import configure_logging
configure_logging(anonymize=True) # redact PII in the file, not only on export
```
With redaction on, every emitted record is scrubbed before it is written:
* **Identifier and path fields** (`slide_id`, `patient_id`, `wsi_id`, `slide_path`, `output_dir`, ...) are replaced by a **stable hash** — the same value keeps the same pseudonym, so *"which slide failed"* still correlates across every line.
* **Free text** — any path, `s3://`/`file://` URI, or slide file name embedded in a message or traceback under a field the policy doesn't recognize — is redacted outright instead; the user name in a home-directory path is masked.
The recognized fields and the per-field strategy (hash, redact, or keep verbatim) are configurable — see [Support bundles → Customize what gets redacted](/guides/get-started/support-bundles#customize-what-gets-redacted) for the full policy and the [`anonymize` reference](/sdk-reference/observability/anonymize).
Redaction is stable pseudonymization for support triage, not encryption. Review [what a bundle contains](/guides/get-started/support-bundles#what-gets-shared) before sharing logs outside your environment.
## Handle errors in your code
Every failure the SDK raises is a `BioptimusError` carrying a machine-readable `code`, structured `context`, and an actionable `remediation`. Catch the base class to handle any SDK failure and read those fields directly:
```python theme={null}
from bioptimus.observability import BioptimusError, ErrorCode
try:
inference.predict("slide.svs")
except BioptimusError as err:
print(err.code) # ErrorCode.WSI_OPEN_FAILED
print(err.code.value) # "BIO-WSI-002" — the stable support code
print(err.context) # {"path": "slide.svs", "errno": 2, ...}
print(err.remediation) # "Verify the slide path points to a valid, ..."
record = err.to_dict() # JSON-friendly dict for your own logging / telemetry
if err.code is ErrorCode.WSI_UNSUPPORTED_FORMAT:
... # branch on a specific failure class
```
**Existing `except` handlers keep working.** Each typed error subclasses the built-in it replaces, so code that already catches the standard exception is unaffected:
| Typed error | Also caught by | Raised for |
| - | - | - |
| `BioptimusValueError` | `except ValueError` | Invalid input — bad path, unsupported format, malformed file |
| `BioptimusRuntimeError` | `except RuntimeError` | Invalid state — backend unavailable, model/forward failure, disk full |
| `BioptimusImportError` | `except ImportError` | A feature needs an optional extra that is not installed |
`BioptimusImportError` names the exact extra to install, so a missing optional dependency is self-explanatory:
```python theme={null}
from bioptimus.observability.errors import BioptimusImportError
try:
...
except BioptimusImportError as err:
print(err.remediation) # e.g. "Install ... pip install bioptimus-sdk[torch] ..."
```
## Error codes
Every `BioptimusError` carries a stable, hyphen-delimited code such as `BIO-WSI-002`, grouped by domain:
| Domain | Prefix | Covers |
| - | - | - |
| Whole-slide image reading | `BIO-WSI-xxx` | Opening/reading slide files, missing backends |
| Tissue masking | `BIO-MASK-xxx` | Tissue mask generation and validation |
| Tile extraction | `BIO-TILE-xxx` | Extracting tiles from a slide |
| Bulk-RNA / omics input | `BIO-RNA-xxx` | Omics file schema and content |
| Request dispatch / transport | `BIO-NET-xxx` | Talking to a remote or AWS endpoint |
| Model execution | `BIO-MODEL-xxx` | Loading or running a model |
| Output writing | `BIO-WRITE-xxx` | Writing inference outputs |
| Slide-level aggregation | `BIO-AGG-xxx` | Combining tile-level results |
| Environment / runtime | `BIO-ENV-xxx` | Disk space, GPU availability, missing dependencies |
| Configuration | `BIO-CFG-xxx` | Invalid SDK configuration |
| Unclassified | `BIO-GEN-xxx` | Fallback for anything else |
Each code has a fixed remediation string, the same one attached to its `error_code` field above and surfaced by [`verify_endpoint`](/guides/get-started/support-bundles#verify-an-endpoint)'s failure reports. The full list of codes and remediations is in the [API reference](/sdk-reference/observability/errors).
# Performance tuning
Source: https://docs.bioptimus.com/guides/get-started/performance-tuning
What to change to get more embedding/prediction throughput, and what to expect from each change.
Whole-slide inference is a pipeline: tiles are read and preprocessed on CPU, then run through the model on GPU, then written to disk. For Bioptimus embedding and prediction models, **the GPU forward pass is usually the bottleneck** — a few knobs matter a lot, and most others matter little. This page covers the client-side (SDK) knobs; for what's fixed by the deployment (precision, batching env vars), see the Inference Server's [Performance](/api-reference/performance) page.
## Quick reference
| Do this | Expected impact |
| - | - |
| Use the **fp16** checkpoint instead of fp32 | Largest single lever — noticeably faster on the same GPU, for the same slide |
| Tighten tissue masking (`mask_threshold`) | Impact scales directly with tile count — fewer tiles is proportionally less work |
| Keep concurrency reasonably high (see below) | Too low starves the GPU; past a certain point, more makes no difference |
| Leave batch size at the default | Has little effect within a normal range — not worth sweeping |
| Add or upgrade GPUs | The remaining lever once the above are already set |
Batch wait time and CPU preprocess worker count rarely matter once the GPU is the bottleneck — leave them at their defaults unless you've confirmed with the check in [Is the GPU the bottleneck?](#is-the-gpu-the-bottleneck) that the GPU is *not* saturated.
## Backend-specific knobs
```python theme={null}
from bioptimus.models.backbones import Backbone
model = Backbone(
"h1",
backend="local",
checkpoint=".../h1_optimus_cls_token_fp16.pt2", # prefer the fp16 checkpoint
device="cuda:0",
max_batch_size=128, # default is a reasonable middle ground; rarely needs changing
)
```
```python theme={null}
inferrer.predict(output_path=out, mode="embed", max_concurrency=256)
```
For a cohort (multiple slides) on a single local GPU, keep `Inference(..., workers=...)` at 1–2 — all slides share the same GPU, so higher values add threads without adding throughput. To use multiple local GPUs, run one process per GPU (`CUDA_VISIBLE_DEVICES=`) rather than raising `workers`. See [Cohorts](/guides/workflows/cohort).
`max_concurrency` is the main lever — size it to your endpoint's GPU count, not your client's CPU count:
```python theme={null}
inferrer.predict(output_path=out, mode="embed", max_concurrency=256)
```
On a **single-GPU** endpoint (e.g. `ml.g5.xlarge`, an A10G-class GPU), lower `max_concurrency` (e.g. 32) to avoid out-of-memory from too many concurrent requests. On a **multi-GPU** endpoint, a higher value (e.g. 256) is a reasonable default. See the note in [Spatial transcriptomics](/guides/workflows/spatial-transcriptomics#2-run-inference-image-only).
For a cohort, size the client-side `workers` to a small multiple of the endpoint's GPU count — too few starves the server's batching, too many just queues up requests.
Tissue masking (`mask_threshold`) and tile geometry (`size`/`stride`) are backend-agnostic — see [Preprocessing tuning knobs](/guides/workflows/preprocessing#tuning-knobs).
For ready-to-copy starting configurations (single slide vs. cohort), see [Recommended starting configurations](/guides/get-started/sdk#recommended-starting-configurations) on the Bioptimus SDK page.
As a rough reference, on a single mid-range datacenter GPU, embedding a typical whole slide (a few thousand tissue tiles) with the fp16 checkpoint takes on the order of one to a few minutes; fp32 is a few times slower. Tissue coverage and GPU model shift this substantially — measure your own workload (tiles processed ÷ wall-clock time) rather than relying on this as a guarantee.
Slower than expected, or a run that looks stalled? See [Common issues](/guides/get-started/troubleshooting).
# Bioptimus SDK
Source: https://docs.bioptimus.com/guides/get-started/sdk
The Bioptimus Python SDK — installation and the two ways to use it.
The Bioptimus Python SDK runs whole-slide inference against either an [on-premise server](/deployment/platforms/on-premise) or a [SageMaker endpoint](/deployment/platforms/aws-sagemaker). It handles WSI reading, tiling, tissue masking, bulk-RNA alignment, and concurrent dispatch.
The Bioptimus SDK talks to whichever model is deployed at the endpoint you connect to. `Backbone.available_backbones()` lists the models the Bioptimus SDK *knows how to build* (`h1`, `m-optimus`, `tissue-seg`) — to see what a server actually has loaded, check its [`/ping`](/api-reference/introduction) response.
## Installation
```bash theme={null}
pip install bioptimus-sdk
```
```bash theme={null}
pip install bioptimus_sdk--py3-none-any.whl
```
## Two ways to use it
One object configures the whole pipeline. Caches tissue masks, organizes outputs into a workspace, and is reproducible. Best for most users and for cohorts.
`Backbone` + `SlideInference` give explicit, per-slide control over the model client, mask provider, and writer.
## Connecting to a model
The `Backbone` factory is the low-level client used by both layers.
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg']
model = Backbone(Models.H1, backend="remote", base_url="http://localhost:8080")
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
print(Backbone.available_backbones()) # ['h1', 'm-optimus', 'tissue-seg']
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url="http://localhost:8080")
```
```python H-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.H1, backend="aws",
endpoint_name="h-optimus", region_name="us-east-1",
)
```
```python M-Optimus theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
model = Backbone(
Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1",
)
```
For M-Optimus, gene sets are fetched from the server automatically (`model.input_gene_names`, `model.output_gene_names`).
Getting a connection error instead of a response? See [Common issues](/guides/get-started/troubleshooting).
## Recommended starting configurations
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.inference.inference import SlideInference
model = Backbone(
"h1", backend="local",
checkpoint=".../h1_optimus_cls_token_fp16.pt2", # fp16
device="cuda:0",
max_batch_size=128,
)
inferrer = SlideInference(
wsi_path=wsi_path, model=model,
mask_provider=tissue_mask_provider, mask_threshold=0.5,
)
inferrer.predict(output_path=out, mode="embed", max_concurrency=256)
```
Use the [`Inference`](/guides/workflows/cohort) facade. Set `workers` by backend: **1–2** on a single local GPU (higher is pointless — see [Performance tuning](/guides/get-started/performance-tuning#backend-specific-knobs)); a small multiple of the endpoint's GPU count for `remote`/`aws`; one process per GPU for multiple local GPUs.
```python theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(
model_name=Models.H1, cohort=cohort,
api_url="http://localhost:8080",
tissue=True, mask_threshold=0.5,
output_path="/data/output", experiment="my_experiment", run=1,
workers=2,
)
infer.tissue()
infer.run(mode="embed")
```
Still too slow? Raise `mask_threshold` (fewer tiles), then move to a larger or additional GPU, or dig into the knobs on [Performance tuning](/guides/get-started/performance-tuning).
## Guides
One-object pipeline, workspaces, reproducible config.
Multi-slide cohorts and late-binding bulk RNA.
M-Optimus gene-expression prediction end to end.
Extract embeddings and visualize morphology.
Read slides: levels, MPP, regions, thumbnails.
Load Zarr/HDF5/NPZ and overlay genes and masks.
Connection issues, and where to look for slow or stalled runs.
## Output formats
| Format | Extension | Notes |
| - | - | - |
| `OutputFormat.ZARR` | `.zarr` | Default. Directory store, memory-efficient |
| `OutputFormat.HDF5` | `.h5` | Single file, memory-efficient |
| `OutputFormat.NPZ` | `.npz` | Accumulates in memory, compressed on close |
Every output file contains the same contents:
* **Datasets:** `outputs`, `coords`, `tissue_ratios`, `thumbnail`, `tissue_mask` — plus `input_gene_names` and `output_gene_names` for M-Optimus.
* **Metadata attributes:** `slide_name`, `tile_size`, `stride`, `mpp`, `slide_dimensions`, `slide_dimensions_at_mpp`, `num_tiles`.
See [Visualizing results](/guides/get-started/visualizing-results) to load and plot them.
# Support bundles
Source: https://docs.bioptimus.com/guides/get-started/support-bundles
Verify a deployed endpoint and build a redacted bundle to share with support.
Before running a full slide, or after something goes wrong, use these tools to check a deployment and package what happened into something you can send to Bioptimus support — nothing here leaves your environment unless you send it yourself. For where logs go and how to customize them, see [Logging](/guides/get-started/logging).
* [**Verify an endpoint**](#verify-an-endpoint) — send one synthetic tile and check the response against the model contract.
* [**Inspect your environment**](#inspect-your-environment) — capture a diagnostic snapshot for self-diagnosis before you open a ticket.
* [**Build a support bundle**](#build-a-support-bundle) — package diagnostics and logs into a redacted `.tar.gz`.
* [**Customize what gets redacted**](#customize-what-gets-redacted) — control the redaction policy on anything you export.
Every artifact below is redacted best-effort, not guaranteed PII-free — see [What gets shared](#what-gets-shared) before sharing one outside your environment.
## Verify an endpoint
Before running a full slide, send one synthetic tile through the endpoint and check the response against the model's contract:
```python Python theme={null}
from bioptimus.observability.healthcheck import verify_endpoint
report = verify_endpoint(
model="h1", # model name, e.g. "h1" or "m-optimus"
backend="local", # "remote", "aws", "local", or "huggingface"
checkpoint=".pt2", # local only
device="cuda", # "cuda" or "cpu" for the in-process backends
)
print(report.summary()) # h1 on local: ok (4 checks)
```
```bash CLI theme={null}
bioptimus verify --model h1 --backend local --checkpoint .pt2
bioptimus verify --model h1 --backend remote --base-url http://localhost:8080
bioptimus verify --model h1 --backend aws --endpoint-name --region-name
```
The CLI prints a checklist, one line per step:
```text theme={null}
h1 on local: ok (4 checks)
[PASS] client_construction (812.4 ms) — Built a local client for h1.
[PASS] metadata (0.1 ms) — Metadata reports model 'h1'.
[PASS] embedding_request (156.2 ms) — Endpoint accepted the synthetic tile in embedding mode.
[PASS] embedding_output_contract (0.0 ms) — Output is 1536 finite value(s), matching the model contract.
```
Each backend takes its own connection arguments, all passed straight through to `Backbone`:
| Backend | Required arguments | Python | CLI |
| - | - | - | - |
| `remote` | Server base URL | `base_url="http://localhost:8080"` | `--base-url http://localhost:8080` |
| `aws` | SageMaker endpoint + region | `endpoint_name=""`, `region_name=""` | `--endpoint-name --region-name ` |
| `local` | Checkpoint path + device | `checkpoint=".pt2"`, `device="cuda"` | `--checkpoint .pt2 --device cuda` |
| `huggingface` | Device (weights auto-downloaded) | `device="cuda"`, `precision="fp16"` | `--device cuda --precision fp16` |
For the `local` backend you can point at the directory holding the checkpoint (`model_dir=` / `--model-dir`) instead of naming the `.pt2` file. Add `--json` to print the full `HealthReport` instead, or `--mode embedding`/`--mode prediction` to force a specific mode.
The command exits non-zero on any failed check, so it can gate a pipeline before the expensive run starts. Each failure carries its own code and fix — see [Error codes](/guides/get-started/logging#error-codes):
```python theme={null}
for check in report.failures:
print(check.name, check.error_code) # e.g. client_construction BIO-MODEL-002
print(check.remediation) # what to do about it
```
### Read the report
`verify_endpoint` returns a `HealthReport`; `bioptimus verify --json` prints the same object. Every step is a `Check`, and nothing raises unless you ask it to:
```python theme={null}
report.ok # True when every non-skipped check passed
report.summary() # "h1 on local: ok (4 checks)"
report.checks # every Check, in execution order
report.failures # only the checks that ran and failed
report.to_dict() # JSON-friendly dict (what --json prints, what a bundle stores)
```
Each `Check` carries `name`, `passed`, `skipped`, `duration_ms`, `detail`, and — on failure — `error_code`, `remediation`, and structured `context` (for example the observed vs. expected output length).
### Verify options
| Option | Python | CLI | Purpose |
| - | - | - | - |
| Force specific modes | `modes=["embedding"]` | `--mode embedding` (repeatable) | Skip mode auto-detection |
| Override tile size | `tile_size=(224, 224)` | — | Probe how the endpoint handles another geometry |
| Seed the synthetic tile | `seed=0` | — | Make two runs byte-identical and comparable |
| Raise instead of report | `raise_on_failure=True` | *(non-zero exit)* | Turn the first failure into a `BioptimusRuntimeError` |
| Bound a slow endpoint | `timeout=30.0` | `--timeout 30` | Cap the wait on the `remote`/`aws` request |
By default `modes="auto"` resolves exactly the modes the model exposes — embedding for the plain backbones, prediction for `tissue-seg`, both for `m-optimus`, and embedding-only for the `huggingface` backend. The tile sent is a deterministic gradient-plus-noise image at the model's declared tile size, so a synthetic request is never mistaken for customer data in a server log.
## Inspect your environment
The same environment snapshot that goes into a bundle is available on its own — useful for self-diagnosis before you open a ticket. It never raises: a probe that cannot run degrades to an `{"error": ...}` entry instead of failing the snapshot.
```python theme={null}
from bioptimus.observability.probes import snapshot
snap = snapshot() # a plain, JSON-friendly dict
snap["gpu"]["cuda_available"]
snap["dependencies"]["satisfied"]
```
The snapshot is stamped with `schema_version` and `captured_at` and gathers:
| Section | Reports |
| - | - |
| `versions` | Installed versions of the SDK and key dependencies |
| `dependencies` | Whether installed packages satisfy the SDK's declared constraints (`satisfied`, `conflicts`) |
| `host` | Python, platform, interpreter, CPU/RAM, container memory limit, open-file limit |
| `env` | An allowlist of non-secret, inference-relevant environment variables |
| `gpu` | Torch/CUDA/driver versions, per-device capability and memory, context health |
| `wsi_backends` | Which WSI backends (OpenSlide, cuCIM, tifffile) import, and why not |
| `disk` | Temp-dir free space, writability, and leaked AOTInductor extractions |
| `pt2_compat` | Whether each `.pt2` checkpoint matches the host GPU/CPU architecture |
| `models` | Load state, device, and precision of loaded models |
Each probe is also callable on its own — for a fast CUDA or checkpoint check without the whole snapshot:
```python theme={null}
from bioptimus.observability.probes import probe_gpu, probe_pt2_compat
probe_gpu()["cuda_available"] # False -> run on a GPU instance
probe_pt2_compat({"h1": "h1.pt2"})[0]["compatible"] # False -> checkpoint/arch mismatch
```
A model that loads but fails at inference is almost always a version or architecture mismatch. `snap["dependencies"]["conflicts"]` and `snap["pt2_compat"]` pinpoint it in one call — the same evidence support would ask for.
## Build a support bundle
One command packages a redacted environment snapshot and logs into a `.tar.gz` — attach this to a support request:
```bash theme={null}
bioptimus support-bundle --output bundle.tar.gz
# Wrote support bundle to bundle.tar.gz
```
`--categories` selects what goes in the bundle: `diagnostics` (environment snapshot and logs, the default) and/or `usage` (model-request records only, e.g. for a billing question). Repeat the flag or comma-separate to combine them. Add a per-slide status ledger with `--workspace`, or fold in a running server's own diagnostics:
```bash theme={null}
bioptimus support-bundle --categories usage --output usage.tar.gz
bioptimus support-bundle --categories diagnostics,usage --output bundle.tar.gz
bioptimus support-bundle --workspace path/to/workspace --output bundle.tar.gz
bioptimus support-bundle --server-url http://localhost:8080 --output bundle.tar.gz
```
Or from Python:
```python theme={null}
from bioptimus.observability.support_bundle import build_support_bundle
archive = build_support_bundle(
output_path="bundle.tar.gz",
categories=["diagnostics", "usage"], # omit to get the default full diagnostics bundle
)
print(archive) # bundle.tar.gz
```
**What you can put in a bundle:**
| Contents | Python | CLI |
| - | - | - |
| Environment snapshot + logs | *default* | *default* |
| Usage records only | `categories=["usage"]` | `--categories usage` |
| `.pt2` checkpoint compatibility | `checkpoints={"h1": "h1.pt2"}` | `--checkpoint h1.pt2` |
| Per-slide status ledger | `workspace="path/"` or `ledger_file="manifest.yaml"` | `--workspace path/` or `--ledger manifest.yaml` |
| A running server's own diagnostics | `server_url="http://localhost:8080"` | `--server-url ...` |
| Endpoint verification (`healthcheck.json`) | `verify={"model": "h1", "backend": "remote", "base_url": "..."}` | — |
| Loaded-model summary | `models={"h1": model}` | — |
The last two are Python-only, since they need a live inference request or in-memory model objects. The bundle finds your log file automatically through `BIOPTIMUS_LOG_FILE` and includes its rotations; point it at a different file with `--log-file` / `log_file=`. Every option writes a single local `.tar.gz`; nothing is sent anywhere.
Both the CLI and Python **redact by default**. For local self-debugging you can keep full paths and identifiers with `--no-anonymize` (CLI) or `anonymize=False` (Python) — never share an unredacted bundle outside your environment.
### What's inside the archive
Extract the `.tar.gz` and inspect it before sending — every bundle is one directory:
```text theme={null}
bioptimus-support-/
manifest.json # schema version, timestamp, SDK version, anonymized flag, file list
diagnostics.json # the redacted environment snapshot (the sections shown above)
logs/ # each configured log file and its rotations, redacted line by line
usage.json # model-request records — only with the usage category
healthcheck.json # endpoint verification report — only with verify=
ledger.yaml # per-slide status summary — only with --workspace / --ledger
server_diagnostics.json # the server's own /diagnostics — only with --server-url
server_usage.json # the server's usage records — only with --server-url and usage
```
`manifest.json` records whether the bundle was anonymized and the exact file list, so a recipient can confirm what they received. The first three files are always present in a diagnostics bundle; the rest appear only when you request the option that produces them.
A running server exposes the same redacted data directly:
`/diagnostics` is **unauthenticated** — keep it behind the same VPN or network boundary as the rest of your deployment, never on a public network.
```bash theme={null}
curl "http://localhost:8080/diagnostics?categories=usage" -o usage.json
```
```json usage.json theme={null}
{
"usage": [
{
"event": "model_request_received",
"model_name": "h1",
"requested_model_name": "h1",
"mode": "embedding",
"endpoint": "/invocations"
}
]
}
```
On **AWS SageMaker**, the server itself runs in your own account — its logs land in **your CloudWatch**, not ours. `bioptimus support-bundle` still captures the client-side SDK logs and diagnostics; CloudWatch is the source for server-side logs on that platform.
## Customize what gets redacted
Redaction is driven by a per-field policy you can inspect and change. Each recognized field takes one of three strategies:
| Strategy | Effect | Default for |
| - | - | - |
| `Strategy.HASH` | Replace with a stable pseudonym (`sha256:...`), preserving correlation | Identifiers: `slide_id`, `patient_id`, `wsi_id`, ... |
| `Strategy.REDACT` | Replace with `[REDACTED]`, dropping the value | *(opt-in)* |
| `Strategy.KEEP` | Keep verbatim; still scrubs embedded paths and user names | Any unrecognized field |
The default policy (`DEFAULT_CONFIG`) hashes every recognized identifier and path field. To apply a different policy — for example, redact `slide_id` outright instead of hashing it — build an `AnonymizationConfig` and run your own data (a log entry, a manifest, any dict) through the redactor before you share it:
```python theme={null}
from bioptimus.observability.anonymize import (
AnonymizationConfig, Strategy, anonymize_mapping,
)
from bioptimus.observability import read_recent_logs
policy = AnonymizationConfig(overrides={"slide_id": Strategy.REDACT})
for entry in read_recent_logs(anonymize=False): # read full detail...
shareable = anonymize_mapping(entry, policy) # ...then redact with your policy
```
The building blocks are reusable on any value: `hash_identifier`, `redact_text` (paths, URIs, and slide names in free text), `scrub_user_paths` (home-directory user names), and `anonymize_value` / `anonymize_mapping` (a single keyed value, or a whole nested mapping). See the [`anonymize` reference](/sdk-reference/observability/anonymize).
## What gets shared
**Before you share a bundle, know what's in it:**
* **Hashed, not encrypted.** Identifiers and path-like fields (slide/patient IDs, `slide_path`, ...) are replaced with a stable, unsalted hash so the same slide correlates across log lines — not a cryptographic guarantee. Paths embedded in free text are redacted outright.
* **Nothing is transmitted automatically.** Every command above only writes a local file or answers a request you make yourself; sharing it with Bioptimus is a manual step.
* **Best-effort redaction.** Bioptimus makes no warranty that the output is free of PII and accepts no liability for it — review a bundle yourself before sharing it outside your environment. An unrecognized field or path shape can pass through unredacted; in particular, connection arguments such as a SageMaker `endpoint_name` or `region_name` aren't recognized identifier fields, so they pass through unchanged.
# Common issues
Source: https://docs.bioptimus.com/guides/get-started/troubleshooting
Common SDK issues: connecting to a server and diagnosing runs.
When a run fails or behaves unexpectedly, two tools narrow it down before you read a traceback:
* **Gate a deployment before the expensive run** with [`bioptimus verify`](/guides/get-started/support-bundles#verify-an-endpoint) — one synthetic tile, checked against the model contract.
* **Self-diagnose the environment** with [`snapshot()`](/sdk-reference/observability/probes#snapshot), or run a single [`probe_*`](/sdk-reference/observability/probes) for a fast, targeted check. Most issues below map to one probe — the same evidence a [support bundle](/guides/get-started/support-bundles#inspect-your-environment) carries.
Every SDK error also carries a code and a one-line fix; the full list is in [Error codes](/guides/get-started/logging#error-codes).
Before constructing a `Backbone` (or an `Inference` pipeline), confirm the on-premise server is reachable:
```python theme={null}
import requests
requests.get("http://localhost:8080/ping", timeout=5).json()
# {"status": "ok", "models": ["h1", "tissue-seg"]}
```
If this raises `ConnectionError` / connection refused or times out, nothing is serving at that URL:
* **Not started** — launch the container, mapping port 8080, then wait for models to load. See [On-premise deployment](/deployment/platforms/on-premise):
```bash theme={null}
docker run -d --name bioptimus-server --gpus all -p 8080:8080 serve
```
* **`503 {"status": "loading"}`** — models are still initialising; wait and retry.
* **Wrong URL/port** — `base_url` (and the pipeline's `api_url`) must be the server's host and port, with **no** trailing `/ping`.
For SageMaker there is no `/ping`: confirm the endpoint is `InService` and that `endpoint_name` / `region_name` are correct.
The SDK loaded but the GPU is not being used. Check what the runtime actually sees:
```python theme={null}
from bioptimus.observability.probes import probe_gpu
gpu = probe_gpu()
gpu["cuda_available"] # False -> CUDA is not usable here
gpu.get("reason") # why: PyTorch missing, or CUDA cannot be queried
```
* **`cuda_available` is `False` on a GPU host** — the NVIDIA driver is usually too old for the CUDA build PyTorch ships. `probe_gpu` reports the driver and CUDA build versions side by side even when CUDA is down.
* **A device loads but kernels fail at launch** — its compute capability is missing from PyTorch's compiled build (common after a wheel reinstall); `probe_gpu` flags that device.
* **Running on a CPU-only box** — move to a GPU instance. This surfaces as [`GPU_UNAVAILABLE`](/guides/get-started/logging#error-codes).
A model that loads and then fails at inference (`MODEL_FORWARD_FAILED`) is almost always an architecture or dependency mismatch, not a problem with the slide. Two probes pinpoint it:
```python theme={null}
from bioptimus.observability.probes import probe_pt2_compat, probe_dependencies
probe_pt2_compat({"h1": "h1.pt2"})[0]["compatible"] # False -> built for another GPU arch
probe_dependencies()["conflicts"] # non-empty -> a dependency is out of range
```
* **`.pt2` architecture mismatch** — the AOTInductor artifact is compiled for a specific GPU/CPU architecture. `probe_pt2_compat` reports `compiled_for` versus the host and the `reason` for any mismatch.
* **Dependency drift** — upgrading a pinned dependency past its supported ceiling (for example `zarr>=3` or a too-new `torch`) breaks the compiled artifacts silently. `probe_dependencies` reports each conflict as `{name, installed, required}`.
Some backends ship as optional extras to keep the base install light. Using one without its extra raises [`BioptimusImportError`](/sdk-reference/observability/errors) (a real `ImportError` subclass) tagged [`DEPENDENCY_MISSING`](/guides/get-started/logging#error-codes), with the exact command in its `remediation`:
| Feature | Install |
| - | - |
| `local` backend / PyTorch inference | `pip install bioptimus-sdk[torch]` |
| `huggingface` backend | `pip install bioptimus-sdk[huggingface]` |
| GPU-accelerated WSI decoding (cuCIM) | `pip install bioptimus-sdk[cucim]` |
OpenSlide and tiffslide readers are bundled by default. If an import still fails after installing, `probe_host()` catches the "installed it but it's not found" case — check `executable` and `in_venv` to confirm you are running the interpreter you installed into.
When a WSI fails to open, first check which readers actually imported on this host:
```python theme={null}
from bioptimus.observability.probes import probe_wsi_backends
probe_wsi_backends()
# {"openslide": {"available": True, "version": "..."},
# "cucim": {"available": False, "reason": "..."}, ...}
```
A backend reporting `available: False` carries the import `reason` — usually a missing native library. Install the matching extra (above) or convert the slide to a format one of the available backends reads.
Loading a `.pt2` extracts it into the system temp directory; if that directory is full or read-only, extraction fails (`DISK_FULL`). `probe_disk` reports the headroom and flags both failure modes:
```python theme={null}
from bioptimus.observability.probes import probe_disk
d = probe_disk()
d["warning"] # "TMP_NOT_WRITABLE", "TMP_NEARLY_FULL", or None
d["aotinductor_temp_dirs"] # leftover extractions pile up when packages aren't cleaned
```
Point `TMPDIR` at a writable volume with room, and clear leaked `aotinductor` extraction directories if the count keeps climbing.
The writer is finalizing outputs (thumbnail, tissue mask, metadata) — this is expected and brief. A run that stays stuck with an idle GPU points to I/O, not the model; see the Inference Server's [Troubleshooting](/api-reference/troubleshooting) guide.
Server responding fine but throughput lower than expected, or a run that looks stalled? That's covered in the Inference Server's [Troubleshooting](/api-reference/troubleshooting) guide. For what to change to get more throughput, see [Performance tuning](/guides/get-started/performance-tuning).
Need to share details with support? See [Logging](/guides/get-started/logging) and [Support bundles](/guides/get-started/support-bundles).
# Visualizing results
Source: https://docs.bioptimus.com/guides/get-started/visualizing-results
Load Zarr/HDF5/NPZ outputs and overlay gene expression or tissue masks.
Inference outputs are written as Zarr (default), HDF5, or NPZ — all three carry the same datasets under the same keys.
Figures in this documentation are real model outputs on open-access **TCGA** slides (e.g. `TCGA-75-7027`), shown as representative examples — not benchmarks. See [Responsible use → Data attribution](/documentation/resources/responsible-use#data-attribution).
## Load an output file
Load a Zarr output with the built-in helper; it returns the arrays and metadata as a dict.
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
preds, coords = data["outputs"], data["coords"]
gene_names = data.get("gene_names") # M-Optimus gene predictions only; None for embeddings
```
`load_zarr_output` returns the following datasets (plus a `metadata` dict). Optional entries are present only when the model and pipeline produce them:
| Key | Shape | Description |
| - | - | - |
| `outputs` | `(n_tiles, D)` | Per-tile embeddings (`D` = 1536) or gene predictions (`D` = number of output genes). |
| `coords` | `(n_tiles, 2)` | Pixel `(x, y)` location of each tile on the slide. |
| `gene_names` | `(D,)` | Output gene identifiers (M-Optimus only). |
| `thumbnail` | `(H, W, 3)` | Downsampled RGB slide overview. |
| `tissue_mask` | `(H, W)` | Boolean tissue-vs-background mask. |
| `tissue_ratios` | `(n_tiles,)` | Fraction of each tile covered by tissue. |
Metadata attributes include `slide_dimensions`, `slide_dimensions_at_mpp`, `tile_size`, `mpp`, and `num_tiles`. HDF5 (`.h5`) and NPZ (`.npz`) outputs expose the same keys via `h5py` and `np.load(allow_pickle=True)`.
## Plot with the built-in helpers
The helpers below visualize **M-Optimus gene predictions** and require the `gene_names` returned above. For H-Optimus tile embeddings (no `gene_names`), see [Tile embeddings & PCA](/guides/workflows/embeddings-pca).
`bioptimus.utils` ships the plotting helpers used in the [getting-started notebooks](https://github.com/bioptimus/m-jumpstart). Heatmap overlays and single-gene overlays are all covered by the [spatial gene panel](#spatial-gene-panel) helper below — pass a single-entry `GENE_PANEL` for one gene — so there's no need to hand-roll matplotlib.
### Spatial gene panel
Overlay one or more genes on the slide thumbnail in one call.
```python theme={null}
GENE_PANEL = {
"EPCAM": "ENSG00000119888", # epithelium / tumor
"CD3E": "ENSG00000198851", # T-cells
"CD8A": "ENSG00000153563", # cytotoxic T-cells
"MKI67": "ENSG00000148773", # proliferation
"COL1A1": "ENSG00000108821", # stroma
"SFTPC": "ENSG00000168484", # alveolar (lung)
}
utils.plot_gene_panel_overlay(preds, coords, gene_names, GENE_PANEL,
wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Spatial gene-expression panel overlaid on a slide thumbnail]()
### Image-only vs. bulk-RNA-guided
Compare the same gene predicted with and without bulk RNA context.
```python theme={null}
gene_idx = gene_names.index("ENSG00000119888") # EPCAM
utils.plot_gene_overlay_comparison(
preds_bulk, coords_bulk, preds_image_only, coords_image_only,
gene_idx=gene_idx, wsi_path=wsi_path, gene_name="EPCAM",
label_a="With bulk RNA", label_b="Without bulk RNA (image only)",
)
```
![Side-by-side EPCAM overlay: with bulk RNA vs image only]()
### Highest- and lowest-expressing tiles
Pull the tiles driving a gene's prediction for quick visual QC.
```python theme={null}
utils.plot_top_gene_tiles(preds, coords, gene_idx=gene_idx,
wsi_path=wsi_path, gene_name="EPCAM",
n_top=6, n_bottom=6)
```
![Grid of tiles with highest and lowest predicted EPCAM expression]()
# WSI processing
Source: https://docs.bioptimus.com/guides/reference/wsi-processing
Read whole-slide images: pyramid levels, MPP, regions, and thumbnails.
The Bioptimus SDK's `WSI` factory provides uniform access to whole-slide images, automatically selecting the best available backend.
**Supported backends** (auto-selected, in order of preference):
* **cuCIM** — GPU-accelerated reading, used when available.
* **OpenSlide** — CPU fallback, broad format support.
**Supported formats** include `.svs`, `.tiff`, `.tif`, `.ndpi`, `.vms`, `.vsi`, `.scn`, `.mrxs`, and `.jp2` (subject to the active backend).
## Open a slide
```python theme={null}
from bioptimus.io.wsi import WSI, Level, MPP, Magnification, MeasurementUnit
wsi = WSI("/data/wsi/tcga_coad.svs")
print(wsi.level_count) # pyramid levels (0 = highest resolution)
print(wsi.level_dimensions(Level(0))) # base dimensions
print(wsi.mpp) # microns per pixel at level 0
```
## Properties
Slides carry rich scanning metadata. Access it through `props`, and inspect the pyramid with `level_dimensions` and `level_downsample`.
```python theme={null}
props = wsi.props
print(props.SCANNER) # scanning device
print(props.OBJECTIVE_POWER) # objective magnification (e.g. 40)
print(props.MPP_X, props.MPP_Y)
# Per-level pyramid structure:
for level in range(wsi.level_count):
dims = wsi.level_dimensions(Level(level))
ds = wsi.level_downsample(Level(level))
print(f"Level {level}: {dims.width}x{dims.height} (downsample {ds:.1f}x)")
```
## Bounded vs. unbounded
Scanners capture the whole slide, but tissue occupies only part of it. `bounded=True` returns the tissue area; `bounded=False` returns the full slide.
```python theme={null}
tissue = wsi.dimensions(bounded=True) # tissue area only
full = wsi.dimensions(bounded=False) # entire slide
```
## Read a region
Specify resolution three ways — by pyramid `Level`, by physical `MPP`, or by `Magnification` — and choose pixel or micrometer units.
```python theme={null}
# By pyramid level, size in pixels (Level 2 ≈ 16× downsample), tissue-bounded:
region_px = wsi.read_region(location=(1625, 0), size=(1200, 1200),
resolution=Level(2),
measurement_unit=MeasurementUnit.PIXELS, bounded=True)
# The same window requested in micrometers (physical units):
region_um = wsi.read_region(location=(1625, 0), size=(599.0, 599.0),
resolution=Level(2),
measurement_unit=MeasurementUnit.UM, bounded=True)
img = region_px.image # PIL image
print(region_px.image.size, region_px.resolution)
```
![Region read in pixels versus micrometers, side by side]()
`MPP ≈ 10 / magnification` (40× ≈ 0.25 MPP, 20× ≈ 0.5 MPP, 10× ≈ 1.0 MPP). Bioptimus embeddings use 0.5 MPP; tissue segmentation uses 8 MPP.
## Thumbnails & associated images
```python theme={null}
thumb = wsi.get_thumbnail(size=(512, 512), bounded=False) # PIL image
images = wsi.associated_images # {'thumbnail','label','macro'}
macro = images["macro"] # slide-level macro photograph
props = wsi.props # scanner, objective, MPP_X/Y, ...
```
`size` is an upper bound: the aspect ratio is preserved, so the longer side is fit to the requested dimension and the shorter side scaled down proportionally. This slide is slightly taller than wide, so requesting `(512, 512)` yields 486×512 px rather than an exact 512×512.
![Whole-slide image thumbnail]()
![Macro associated image of the whole slide]()
For inference you don't usually call the reader directly — the [Inference pipeline](/guides/get-started/inference-pipeline) and `SlideInference` handle tiling. Use the reader for QC, custom region extraction, or building your own pipelines.
# Cohorts
Source: https://docs.bioptimus.com/guides/workflows/cohort
Run a model over a cohort of slides with shared tissue masks and a structured workspace.
A `Cohort` is the single source of truth for a multi-slide experiment. You build it from a directory of WSIs (or a manifest CSV), run a model over it, and outputs are organized into a reproducible workspace. Cohorts can also hold bulk RNA for multimodal prediction — see [Spatial transcriptomics](/guides/workflows/spatial-transcriptomics#3-add-bulk-rna-multimodal-prediction).
`m-jumpstart` includes a cohort batch-processing example.
## 1. Build a cohort
```python theme={null}
from bioptimus.data.cohort import Cohort
cohort = Cohort.from_directories(wsi_dir="/data/wsi/tcga_mini_coad")
```
```python theme={null}
from bioptimus.data.cohort import Cohort
# One row per slide. Required column: `wsi_id` (filename stem or full name).
# Optional: `patient_id`, `bulk_rna_id`, `timepoint`, `wsi_path`, `bulk_rna_path`.
# Pass `wsi_dir=` / `bulk_rna_dir=` to resolve paths from IDs, or
# `columns={...}` to map custom column names.
cohort = Cohort.from_csv("/data/cohort.csv", wsi_dir="/data/wsi/tcga_mini_coad")
```
```python theme={null}
print(cohort.summary())
print(cohort.wsi_ids)
print(cohort[0].available_modalities) # e.g. ['image']
```
## 2. Run a model over the cohort
Create one `Inference` for the cohort, then run it. Tissue masks are cached and resume automatically, so re-running only processes what's missing. Pick your backend below (only the `common` dict differs), then your model.
```python theme={null}
common = dict(
api_url="http://localhost:8080",
tissue=True, mask_threshold=0.5,
output_path="/data/output", experiment="tcga_coad", run=1,
workers=5,
)
```
```python theme={null}
common = dict(
backend="aws", endpoint_name="bioptimus-prod", region_name="us-east-1",
tissue=True, mask_threshold=0.5,
output_path="/data/output", experiment="tcga_coad", run=1,
workers=1, # ml.g5.xlarge has a single GPU; concurrent requests can OOM
)
```
```python H-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(model_name=Models.H1, cohort=cohort, variant="mini", **common)
infer.tissue() # shared masks; only computes what's missing
infer.run(mode="embed") # H-Optimus produces embeddings
infer.report() # status summary
```
```python M-Optimus theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
infer = Inference(model_name=Models.M_OPTIMUS, cohort=cohort, variant="mini", **common)
infer.tissue() # shared masks; only computes what's missing
infer.run(mode="embed")
infer.run(mode="predict") # image-only (see Spatial transcriptomics for bulk RNA)
infer.report() # status summary
```
Each output is tagged with the modalities used (e.g. `["image"]`).
## 3. Extract & save tiles (optional)
Independent of inference — useful for QC or external pipelines:
```python theme={null}
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
from bioptimus.io.wsi.factory import WSI
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
# Reuse the model's tile geometry (224×224 @ 0.5 µm/pixel) so tiles match inference.
tile_spec = Backbone(Models.H1, base_url="http://localhost:8080").model_spec.tile_spec
extractor = TileExtractor(tile_spec=tile_spec, mask_threshold=0.5)
with WSI(wsi_path) as reader:
extractor.fit_extract(reader)
extractor.save(tile_dir, image_format="png", workers=4)
extractor.to_csv(csv_dir)
```
# Tile embeddings & PCA
Source: https://docs.bioptimus.com/guides/workflows/embeddings-pca
Extract tile embeddings (H-Optimus or M-Optimus) and visualize morphology with PCA.
Both H-Optimus and M-Optimus produce a 1536-dimensional embedding per tile (the CLS token). Principal components of these embeddings reveal the dominant axes of morphological variation — often corresponding to tissue vs. background, tumor vs. stroma, and immune-rich vs. immune-poor regions.
`h1-jumpstart` — a full, runnable H-Optimus embedding example.
`m-jumpstart` — the equivalent for M-Optimus.
## 1. Configure the model
Embeddings are available from both models. Import once, then pick your backend and model — the rest of the guide is identical.
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
```
```python H-Optimus theme={null}
API_URL = "http://localhost:8080"
model = Backbone(Models.H1, backend="remote", base_url=API_URL)
tissue = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python M-Optimus theme={null}
API_URL = "http://localhost:8080"
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url=API_URL)
tissue = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python H-Optimus theme={null}
model = Backbone(Models.H1, backend="aws",
endpoint_name="h-optimus", region_name="us-east-1")
tissue = Backbone("tissue-seg", backend="aws",
endpoint_name="h-optimus", region_name="us-east-1")
```
```python M-Optimus theme={null}
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
tissue = Backbone("tissue-seg", backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
```python theme={null}
mask_provider = TiledTissueMask(model=BioptimusTissueMaskModel(backbone=tissue))
```
## 2. Extract embeddings
```python theme={null}
from bioptimus.inference.inference import SlideInference as Inference
from bioptimus.inference.writers import OutputFormat
inferrer = Inference(wsi_path="/data/wsi/tcga_coad.svs", model=model,
mask_provider=mask_provider, mask_threshold=0.5)
result_path = inferrer.predict(output_path="/data/output/tcga_coad.zarr",
output_format=OutputFormat.ZARR,
max_concurrency=256, mode="embed")
```
On a single-GPU SageMaker endpoint (`ml.g5.xlarge`), lower `max_concurrency` (e.g. 32) to avoid out-of-memory from concurrent tile requests.
## 3. PCA
Load the embeddings and fit PCA in one call — `bioptimus.utils.plot_pca_scatter` fits, plots, and returns the scores plus the fitted PCA (explained-variance ratios are printed):
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
embeddings, coords = data["outputs"], data["coords"] # (N, 1536), (N, 2)
scores, pca = utils.plot_pca_scatter(embeddings, n_components=3,
title="Embeddings PCA", cmap="plasma")
```
![Scatter plot of the first two principal components of tile embeddings]()
## 4. Spatial visualization
`plot_spatial_pca` maps the PC scores back onto tile coordinates as per-PC spatial heatmaps — each component tends to isolate a distinct tissue compartment:
```python theme={null}
utils.plot_spatial_pca(scores, coords, components=[0, 1, 2])
```
![Three spatial heatmaps, one per principal component, overlaid on tile coordinates]()
The same embeddings feed downstream models (classification, clustering, retrieval, MIL). For the high-level workflow with caching and workspaces, use the [Inference pipeline](/guides/get-started/inference-pipeline) with `infer.embed(path)`.
# Tissue masking & tiling
Source: https://docs.bioptimus.com/guides/workflows/preprocessing
Preprocess a slide before inference: segment tissue to drop background, then split it into model-sized tiles.
Before any embedding or gene-expression model runs, two preprocessing steps prepare the slide:
1. **Tissue masking** — a whole-slide image is mostly background. [Tissue segmentation](/documentation/models/tissue-segmentation) produces a binary mask so background is discarded before the expensive feature step.
2. **Tiling** — the slide is too large to process at once, so it is split into small, model-sized tiles. Only tiles that overlap tissue are kept.
Resolutions differ by stage: tissue segmentation runs at **8 µm/pixel** (coarse, 512×512 tiles), while embeddings and gene predictions run at **0.5 µm/pixel** (224×224 tiles). The mask is generated once and reused to filter the fine-resolution grid.
The [Inference pipeline](/guides/get-started/inference-pipeline) does both steps for you. Reach for the manual API below only when you need masks or tiles as standalone artifacts — for QC, custom filtering, or your own pipeline.
## The quick path: the Inference pipeline
`Inference` masks tissue, caches the result, and reuses it automatically for later `embed`/`predict` calls.
```python theme={null}
from bioptimus.inference import Inference
from bioptimus.models.types import Models
from bioptimus import utils
infer = Inference(
model_name=Models.H1,
api_url="http://localhost:8080",
tissue=True,
mask_threshold=0.5,
output_path="/data/output",
experiment="preprocessing-demo",
run=1,
)
mask = infer.tissue(wsi_path) # uint8 (H, W) array; cached to /tissue/.png
```
The mask is cached to `/tissue/.png` and reused on the next `tissue()`, `embed()`, or `predict()` call. Visualize it against the slide thumbnail with the built-in helper:
```python theme={null}
mask_png = "/data/output/preprocessing-demo/run_1/tissue/.png"
utils.plot_slide_and_mask(wsi_path, mask_png, threshold=0.5)
```
![Slide thumbnail alongside its binary tissue mask]()
## The manual path: masks and tiles as artifacts
### 1. Generate a tissue mask
Build a mask provider from the tissue-seg endpoint. It is reusable across slides — it generates a fresh mask per slide.
```python theme={null}
from bioptimus.io.wsi import WSI
from bioptimus.models.backbones import Backbone
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
tissue_backbone = Backbone("tissue-seg", base_url="http://localhost:8080")
provider = TiledTissueMask(model=BioptimusTissueMaskModel(tissue_backbone))
wsi = WSI(wsi_path)
mask = provider.generate(wsi) # uint8 (H, W) in {0, 1} over the tissue bounding box at 8 µm/pixel
```
Already have masks on disk? Use them directly instead of the endpoint. Non-zero pixels mean tissue, and files are matched to slides by filename stem (PNG, JPEG, TIFF, BMP, or `.npy`):
```python theme={null}
from bioptimus.preprocess.wsi.provider.precomputed import PrecomputedTissueMask
provider = PrecomputedTissueMask(mask_dir="masks/", suffix=".png")
```
### Bring your own segmentation (custom mask provider)
Need a different segmentation model — for example one that is robust to marker-pen artifacts, or one you already run in-house? Subclass `TissueMaskProvider` and return your own mask. `TileExtractor` and the [Inference pipeline](/guides/get-started/inference-pipeline) depend only on this interface, so any provider — algorithmic, model-based, or a wrapper around an external tool — plugs in unchanged.
```python theme={null}
import numpy as np
from bioptimus.io.wsi.interface import WSIReader
from bioptimus.observability import log_failures
from bioptimus.preprocess.wsi.provider.base import TissueMaskProvider
class CustomMaskProvider(TissueMaskProvider):
"""Wraps an external segmentation model as a tissue mask provider."""
def __init__(self, segmenter):
self._segmenter = segmenter
@log_failures
def generate(self, reader: WSIReader, *, show_progress: bool = True) -> np.ndarray:
# Run your own segmentation and return a uint8 (H, W) mask, 1 = tissue.
# The mask must cover the same extent the extractor scales it against:
# the slide's tissue-bounded region, reader.dimensions(bounded=True).
mask = self._segmenter.run(reader.path)
return (mask > 0).astype(np.uint8)
```
**What the mask must be.** A 2-D `uint8 (H, W)` array where `1` = tissue. Two rules keep it aligned with the tiling grid:
1. **Resolution is flexible.** The mask does **not** need to match the slide's pixel dimensions or any specific µm/pixel — `TileExtractor` resizes it (nearest-neighbour) to the reference level before filtering. A mask produced at 2 µm/pixel therefore aligns correctly against a 0.5 µm/pixel tile grid.
2. **Extent must match, not just aspect ratio.** The mask is stretched onto the slide's **tissue-bounded** region — `reader.dimensions(bounded=True)`. For slides with no scan-area metadata this equals the full slide, so a full-slide mask is correct. For slides that *do* carry bounds metadata (e.g. some scanner formats), `bounded=True` is a crop; a mask spanning the **whole** slide would be squashed into that crop and misalign. Produce the mask over the same region your segmenter sees, and compare `reader.dimensions(bounded=True)` with `reader.dimensions(bounded=False)` if unsure.
As a guard, if the mask's aspect ratio differs from the target region's by more than 5%, a `MASK_ASPECT_MISMATCH` warning is emitted so a transposed or wrong-slide mask does not pass silently.
Use the provider one of two ways: generate the masks up front, save them to a directory, and point `Inference` at that directory with `tissue=False` (shown next), or pass it straight to `TileExtractor(mask_provider=...)` when tiling manually (see [Tile the slide](#2-tile-the-slide) below). To confirm alignment visually, save the mask and overlay it on the slide thumbnail with `utils.plot_slide_and_mask` (see [the quick path](#the-quick-path-the-inference-pipeline) above).
### Hybrid pipeline: local mask, remote embeddings
Because masking and embedding are independent stages, you can produce tissue masks **locally** with a custom provider while sending the surviving tiles to a **remote** endpoint (e.g. AWS SageMaker) for embeddings. This is a **two-phase** workflow: first generate and save the masks, then run inference against them with `tissue=False` so the built-in `tissue-seg` model does not recompute over your masks.
**Phase 1 — generate the masks with your custom provider and save them by slide stem:**
```python theme={null}
from pathlib import Path
from PIL import Image
from bioptimus.io.wsi import WSI
provider = CustomMaskProvider(my_local_segmenter)
wsi_dir, mask_dir = Path("/data/wsis"), Path("/data/masks")
mask_dir.mkdir(parents=True, exist_ok=True)
for wsi_path in wsi_dir.glob("*.svs"):
reader = WSI(str(wsi_path))
mask = provider.generate(reader) # runs YOUR segmentation; uint8 (H, W)
# Save named by slide stem; *255 keeps it visible and round-trips through (loaded > 0).
Image.fromarray(mask * 255).save(mask_dir / f"{wsi_path.stem}.png")
```
**Phase 2 — run the pipeline against those masks, with embeddings on the remote endpoint:**
```python theme={null}
from bioptimus.inference import Inference
from bioptimus.models import Models
infer = Inference(
model_name=Models.H1,
backend="aws", # remote embeddings on SageMaker
endpoint_name="my-endpoint",
wsi_dir="/data/wsis",
mask_dir="/data/masks", # your custom masks, paired to slides by filename stem
tissue=False, # do NOT recompute masks with the built-in tissue-seg model
)
infer.find() # auto-pairs each slide with its mask by stem
infer.embed()
```
`mask_dir` pairs each mask to its slide by filename stem via the built-in `PrecomputedTissueMask` provider, and `tissue=False` ensures your masks are used as-is. The embedding backend (`backend="aws"`) is chosen independently of how the mask is produced.
Inference over a slide is atomic — if any tile fails after retries, that slide's output is discarded. The [Cohort workflow](/guides/workflows/cohort) resumes per slide, so a single failure does not restart the whole cohort.
### 2. Tile the slide
A `TileSpec` defines the tile geometry; `TileExtractor` builds the grid, filters it against the mask, and exports the tiles. Passing `mask_provider` lets the extractor generate the per-slide mask during `fit`.
```python theme={null}
from bioptimus.io.wsi import MPP, MeasurementUnit
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
spec = TileSpec(
size=(224, 224),
stride=(224, 224),
resolution=MPP(0.5),
unit=MeasurementUnit.PIXELS,
)
extractor = TileExtractor(tile_spec=spec, mask_provider=provider, mask_threshold=0.5)
tiles = extractor.fit_extract(wsi) # keeps only tiles with >= 50% tissue
extractor.save("tiles/", image_format="png", workers=4) # writes tile images
extractor.to_csv("tiles/") # writes tiles/.csv
print(f"{len(tiles)} tiles kept")
```
`to_csv` writes one row per retained tile:
| Column | Description |
| - | - |
| `slide_name` | Slide filename stem. |
| `top`, `left` | Tile top-left location in absolute slide coordinates. |
| `width`, `height` | Tile size. |
| `unit` | Measurement unit (`PIXELS` or `UM`). |
| `resolution_type`, `resolution_value` | Resolution the tile is defined at (e.g. `mpp` / `0.5`). |
| `tissue_ratio` | Fraction of the tile covered by tissue. |
`save` writes each tile image as `{slide}_x{left}_y{top}_w{width}_h{height}_{res}_mask_r{ratio}.png`.
## Tuning knobs
| Knob | Where | Effect |
| - | - | - |
| `mask_threshold` | `Inference`, `TileExtractor` | Minimum tissue fraction (0–1) to keep a tile. Lower keeps more border tiles; `None` disables filtering. |
| `size` / `stride` | `TileSpec` | Tile size and step. Equal size and stride give a non-overlapping grid; a smaller stride overlaps tiles. |
| `resolution` / `unit` | `TileSpec` | Physical resolution of tiles. Bioptimus feature models expect 224×224 at 0.5 µm/pixel. |
| `max_concurrency` | `TiledTissueMask.generate` | Concurrent requests to the tissue-seg endpoint (default 64). |
| `workers` | `TileExtractor.save` | Threads for writing tile images (default 4). |
For end-to-end inference you rarely call these directly — the [Inference pipeline](/guides/get-started/inference-pipeline) handles masking and tiling and streams results to disk. Use the manual API for QC, custom region extraction, or building your own pipeline. To read regions, thumbnails, and slide metadata, see [WSI processing](/guides/reference/wsi-processing).
For guidance on which of these knobs actually affect throughput (and by how much), see [Performance tuning](/guides/get-started/performance-tuning).
# Spatial transcriptomics
Source: https://docs.bioptimus.com/guides/workflows/spatial-transcriptomics
Predict spatial gene expression from an H&E slide with M-Optimus, end to end.
M-Optimus predicts expression of thousands of genes directly from H\&E histology — no spatial transcriptomics assay required — enabling spatial analysis on any archived slide. This guide uses the core API (`Backbone` + `SlideInference`) for explicit control; the [Inference pipeline](/guides/get-started/inference-pipeline) wraps the same steps.
See `m-jumpstart` for a full, runnable M-Optimus example.
## 1. Configure the model and tissue masking
Pick your backend — the rest of the guide is identical for both.
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
API_URL = "http://localhost:8080"
model = Backbone(Models.M_OPTIMUS, backend="remote", base_url=API_URL)
tissue_backbone = Backbone("tissue-seg", backend="remote", base_url=API_URL)
```
```python theme={null}
from bioptimus.models.backbones import Backbone
from bioptimus.models.types import Models
from bioptimus.preprocess.wsi.models.bioptimus_mask import BioptimusTissueMaskModel
from bioptimus.preprocess.wsi.provider.tiled import TiledTissueMask
model = Backbone(Models.M_OPTIMUS, backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
tissue_backbone = Backbone("tissue-seg", backend="aws",
endpoint_name="m-optimus", region_name="us-east-1")
```
```python theme={null}
mask_provider = TiledTissueMask(model=BioptimusTissueMaskModel(backbone=tissue_backbone))
print(f"{len(model.input_gene_names or [])} input genes, "
f"{len(model.output_gene_names or [])} output genes")
```
## 2. Run inference (image only)
`SlideInference` tiles the slide at 0.5 µm/pixel, dispatches tiles concurrently, and streams results to Zarr so memory stays constant regardless of slide size.
```python theme={null}
from bioptimus.inference.inference import SlideInference
from bioptimus.inference.writers import OutputFormat
inferrer = SlideInference(
wsi_path="/data/wsi/tcga_coad.svs",
model=model,
mask_provider=mask_provider,
mask_threshold=0.5,
)
result_path = inferrer.predict(
output_path="/data/output/tcga_coad.zarr",
output_format=OutputFormat.ZARR,
max_concurrency=256, # see note for single-GPU SageMaker endpoints
)
```
Tissue masking filters background tiles before prediction — on a typical TCGA slide this skips most of the \~16k tile grid. Set `mask_threshold` to the minimum tissue fraction to keep a tile (default 0.5).
On a single-GPU SageMaker endpoint (`ml.g5.xlarge`), lower `max_concurrency` (e.g. 32) to avoid out-of-memory from concurrent tile requests. On a multi-GPU on-premise server, 256 is a reasonable default. See [Performance tuning](/guides/get-started/performance-tuning) for more.
## 3. Add bulk RNA (multimodal prediction)
M-Optimus prediction can be conditioned on **bulk RNA** for a multimodal readout. Bulk RNA is attached through a [`Cohort`](/guides/workflows/cohort) via late-binding: run image-only first, then link the RNA and re-predict. Modality-aware tracking processes only the missing outputs, and image-only results are preserved separately for comparison.
Send **TPM-normalized expression values** — do **not** `log1p` them yourself. The server transforms the input and the model output for you. Genes are auto-aligned to the model's input set; genes the model expects but your file lacks are encoded as `-1` (non-measured).
The server applies two transforms around the model:
| Stage | Transform | Applied to |
| - | - | - |
| Input (preprocess) | `log1p` | your TPM-normalized expression values, before the model |
| Output (postprocess) | `expm1` | the model output, to return expression values |
```python theme={null}
from bioptimus.data.cohort import Cohort
from bioptimus.inference import Inference
from bioptimus.models.types import Models
cohort = Cohort.from_directories(wsi_dir="/data/wsi/tcga_mini_coad")
infer_m = Inference(model_name=Models.M_OPTIMUS, cohort=cohort,
api_url="http://localhost:8080", tissue=True,
output_path="/data/output", experiment="tcga_coad", run=1)
infer_m.run(mode="predict") # image-only baseline
```
Link bulk RNA and re-predict:
```python theme={null}
# Plain CSV (Ensembl IDs as columns):
cohort.link_bulk_rna("/data/rna/tcga_mini_coad")
# GDC/TCGA gene-quantification TSVs — use the tpm_unstranded column (not
# fpkm_* or raw counts) and strip Ensembl versions:
cohort.link_bulk_rna(
"/data/rna/tcga_mini_coad",
gene_column="gene_id",
value_column="tpm_unstranded", # TPM-normalized expression values (the expected scale)
strip_version=True, # ENSG00000000003.15 -> ENSG00000000003
)
infer_m.run(mode="predict", force=True) # re-predict with bulk RNA context
```
Retrieve each stage's output by modality:
```python theme={null}
out = cohort[0].outputs.get(infer_m.model_name)
img_only = out.get_stage_path("predict", ["image"])
multimodal = out.get_stage_path("predict", ["image", "bulk_rna"])
```
## 4. Inspect the output
Load either output — image-only or multimodal — with the built-in helper:
```python theme={null}
from bioptimus import utils
data = utils.load_zarr_output(result_path)
outputs, coords = data["outputs"], data["coords"] # (n_tiles, n_genes), (n_tiles, 2)
gene_names = data["gene_names"]
```
## 5. Visualize
Genes are identified by **Ensembl gene ID** (e.g. `ENSG00000198851`). Useful markers include EPCAM (`ENSG00000119888`, epithelium), CD3E (`ENSG00000198851`, T-cells), CD8A (`ENSG00000153563`, cytotoxic T-cells), MKI67 (`ENSG00000148773`, proliferation), COL1A1 (`ENSG00000108821`, stroma), and SFTPC (`ENSG00000168484`, alveolar).
`bioptimus.utils.plot_gene_panel_overlay` renders a whole panel on the slide thumbnail in one call. Load the two prediction sets from step 3 and compare them — the multimodal output uses bulk RNA, the image-only output does not:
```python theme={null}
from bioptimus import utils
GENE_PANEL = {
"EPCAM": "ENSG00000119888", # epithelium / tumor
"CD3E": "ENSG00000198851", # T-cells
"CD8A": "ENSG00000153563", # cytotoxic T-cells
"MKI67": "ENSG00000148773", # proliferation
"COL1A1": "ENSG00000108821", # stroma
"SFTPC": "ENSG00000168484", # alveolar (lung)
}
```
```python theme={null}
bulk = utils.load_zarr_output(multimodal)
utils.plot_gene_panel_overlay(bulk["outputs"], bulk["coords"], bulk["gene_names"],
GENE_PANEL, wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Six-gene spatial expression panel, image plus bulk RNA]()
```python theme={null}
image = utils.load_zarr_output(img_only)
utils.plot_gene_panel_overlay(image["outputs"], image["coords"], image["gene_names"],
GENE_PANEL, wsi_path=wsi_path, cols=3, cmap="inferno")
```
![Six-gene spatial expression panel, image only]()
A FP16 model variant is available for faster inference. Run the FP32 and FP16 outputs through the same analysis to compare the impact of quantization.
# License
Source: https://docs.bioptimus.com/license
Copyright (c) 2026 Bioptimus. All rights reserved.
## Commercial and Proprietary License
This software, including all source code, documentation, configuration files, model artifacts, and binary packages (including but not limited to `.pt2` files), is the exclusive property of Bioptimus.
**All rights are reserved.** No part of this software may be reproduced, distributed, transmitted, modified, or used in any form or by any means without the prior written permission of Bioptimus.
## Restrictions
1. **No unauthorized use.** Use of this software requires a separate license agreement or contract with Bioptimus.
2. **No reverse engineering.** Reverse engineering, decompilation, disassembly, or any other attempt to derive source code, algorithms, model weights, or training data from any binary packages — specifically including `.pt2` model files — is strictly prohibited.
3. **No redistribution.** This software may not be copied, shared, sublicensed, sold, or otherwise distributed to any third party without explicit written authorization from Bioptimus.
4. **No derivative works.** Creating derivative works based on this software or any of its components is prohibited without a license agreement.
## Disclaimer
THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL BIOPTIMUS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY ARISING FROM THE USE OF THIS SOFTWARE.
## Contact
To obtain a license or for any inquiries regarding the use of this software:
**Bioptimus** 86 90 Rue Notre Dame de Nazareth 75003 Paris, France
Email: [contact@bioptimus.com](mailto:contact@bioptimus.com) Website: [https://www.bioptimus.com/contact](https://www.bioptimus.com/contact) LinkedIn: [https://www.linkedin.com/company/bioptimus](https://www.linkedin.com/company/bioptimus)
# bioptimus.data.cohort
Source: https://docs.bioptimus.com/sdk-reference/data/cohort
Cohort manifest for pairing WSIs with bulk RNA and clinical metadata.
Provides [`Cohort`](/sdk-reference/data/cohort#cohort), a typed registry that tracks the mapping between patients, wsis, bulk RNA samples, timepoints, labels, and arbitrary clinical metadata. [`WSIRecord`](/sdk-reference/data/cohort#wsirecord) is the single source of truth for each WSI — including per-model outputs and processing status.
Construction paths:
1. **From a user-provided CSV** via [`Cohort.from_csv`](/sdk-reference/data/cohort#from_csv).
2. **Auto-matching from directories** via [`Cohort.from_directories`](/sdk-reference/data/cohort#from_directories).
3. **From a YAML manifest** via [`Cohort.load`](/sdk-reference/data/cohort#load) (for resume).
The cohort can be serialised to YAML via [`Cohort.save`](/sdk-reference/data/cohort#save) and loaded back for full reproducibility and resume support.
**Example**:
```python theme={null}
from bioptimus.data.cohort import Cohort
# Auto-match from directories:
cohort = Cohort.from_directories(
wsi_dir="/data/wsis",
bulk_rna_dir="/data/rna",
)
# Or load a saved manifest (resume):
cohort = Cohort.load(".output/brca/run_1/manifest.yaml")
# Add labels post-hoc:
cohort.add_labels({"patient_001": {"mutation": "BRAF"}})
# Iterate:
for record in cohort:
print(record.patient_id, record.wsi_path)
```
## ModelOutput
```python theme={null}
@dataclass
class ModelOutput()
```
Per-model output record for a single WSI.
**Attributes**:
* `model_name` - Name of the model that produced this output.
* `embedding_path` - Path to the embedding output file.
* `prediction_path` - Path to the prediction output file.
* `tiles_csv_path` - Path to the tile coordinates CSV.
* `tiles_dir` - Directory containing exported tile images.
* `modalities` - Input modalities used for this output (e.g. `["image"]` or `["image", "bulk_rna"]`).
* `timestamp` - ISO 8601 string of when the output was produced.
* `status` - Processing status (`"pending"`, `"done"`, `"error"`).
* `error` - Error message if status is `"error"`.
* `base_dir` - Manifest directory that relative output paths are anchored to for existence checks. Set by [`Cohort.load`](/sdk-reference/data/cohort#load) and never serialised. `None` resolves relative paths against the current working directory.
***
#### is\_done
```python theme={null}
@property
def is_done() -> bool
```
Returns `True` if status is done.
***
#### has\_embedding
```python theme={null}
@property
def has_embedding() -> bool
```
Returns `True` if an embedding path is set and exists.
***
#### has\_prediction
```python theme={null}
@property
def has_prediction() -> bool
```
Returns `True` if a prediction path is set and exists.
***
#### set\_stage\_output
```python theme={null}
def set_stage_output(stage: InferenceMode | str,
modalities: list[str],
path: Path,
timestamp: str | None = None) -> None
```
Records an output for a specific stage and modality combo.
Updates both the per-modality tracking dict and the top-level convenience fields (`embedding_path` / `prediction_path`). The combined [`InferenceMode.PREDICTION_WITH_EMBEDDING`](/sdk-reference/inference/schemas#prediction_with_embedding) stage records the single output file under both fields, since it carries the prediction and the embedding together.
The output stage as an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) (or a mode string; legacy short spellings are accepted).
Modalities used for this output.
Output file path.
ISO 8601 timestamp.
***
#### get\_stage\_path
```python theme={null}
def get_stage_path(stage: InferenceMode | str,
modalities: list[str] | None = None) -> Path | None
```
Returns the output path for a stage and modality combo.
For the combined stage both path fields point at the same file, so the embedding field is returned.
The output stage as an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) (or a mode string; legacy short spellings are accepted).
Modality combination to look up. Falls back to the top-level path when `None`.
**Returns**:
Path resolved against `base_dir`, or `None` if not
recorded.
***
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Serialises to a plain dict for YAML output.
**Returns**:
A dictionary representation of the model output.
***
#### from\_dict
```python theme={null}
@classmethod
def from_dict(cls,
data: dict[str, Any],
base_dir: Path | None = None) -> ModelOutput
```
Constructs from a plain dict (YAML round-trip).
Dictionary with serialised model output fields.
Manifest directory to anchor relative paths to for existence checks.
**Returns**:
A new [`ModelOutput`](/sdk-reference/data/cohort#modeloutput) instance.
## WSIRecord
```python theme={null}
@dataclass
class WSIRecord()
```
Single WSI entry — the single source of truth.
Tracks inputs (WSI, RNA, mask), per-model outputs, and arbitrary annotations for one WSI within a cohort.
**Attributes**:
* `patient_id` - Unique patient identifier.
* `wsi_id` - Unique WSI identifier (typically the filename stem).
* `wsi_path` - Path to the WSI file.
* `bulk_rna_id` - Identifier for the paired bulk RNA sample.
* `bulk_rna_path` - Path to the bulk RNA CSV/TSV file.
* `mask_path` - Path to a tissue mask (pre-computed or cached).
* `mask_timestamp` - When the mask was computed/discovered.
* `timepoint` - Temporal ordering label (e.g. `"t0"`, `"t1"`).
* `outputs` - Per-model output records keyed by model name.
* `labels` - Arbitrary categorical annotations.
* `metadata` - Arbitrary clinical/treatment metadata.
* `base_dir` - Manifest directory that relative paths are anchored to for existence checks. Set by [`Cohort.load`](/sdk-reference/data/cohort#load) and never serialised. `None` resolves relative paths against the current working directory.
***
#### get\_output
```python theme={null}
def get_output(model_name: str) -> ModelOutput
```
Returns the output for *model\_name*, creating if absent.
Model identifier string.
**Returns**:
The [`ModelOutput`](/sdk-reference/data/cohort#modeloutput) for the given model.
***
#### is\_stage\_done
```python theme={null}
def is_stage_done(model_name: str, stage: InferenceMode | str) -> bool
```
Checks whether a stage output exists on disk.
When `modality_outputs` are tracked, the check is scoped to the record's current `available_modalities` so that linking new data (e.g. bulk RNA) automatically surfaces pending work without requiring `force=True`. The combined stage requires both its path fields, so it is done only once both outputs exist.
Model identifier.
The output stage as an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) (or a mode string; legacy short spellings are accepted).
**Returns**:
`True` if the stage's output path(s) exist on disk.
***
#### has\_mask
```python theme={null}
@property
def has_mask() -> bool
```
Returns `True` if a mask path is set and exists on disk.
***
#### merge\_saved\_outputs
```python theme={null}
def merge_saved_outputs(saved: WSIRecord) -> int
```
Merges output/mask metadata from a saved (manifest) record.
Used to resume after a restart: outputs recorded in a previous manifest are folded into this live record, which may already have some outputs from the current process. WSI/RNA paths are left untouched so the user's current directory layout wins.
Adopts `saved.base_dir` unconditionally so that relative paths merged from *saved* keep resolving against the manifest directory rather than the process CWD, regardless of whether a mask is present (existence checks read the record's `base_dir`, not the output's).
The corresponding record loaded from a manifest.
**Returns**:
Number of output entries merged in (new models plus new
modality combinations added to an existing model output).
***
#### available\_modalities
```python theme={null}
@property
def available_modalities() -> list[str]
```
Returns the input modalities available for this record.
Always includes `"image"`. Includes `"bulk_rna"` when `bulk_rna_path` is set.
***
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Serialises to a plain dict for YAML output.
**Returns**:
A dictionary representation of the WSI record.
***
#### from\_dict
```python theme={null}
@classmethod
def from_dict(cls,
data: dict[str, Any],
base_dir: Path | None = None) -> WSIRecord
```
Constructs from a plain dict (YAML round-trip).
Dictionary with serialised WSI record fields.
Manifest directory to anchor relative paths to for existence checks.
**Returns**:
A new [`WSIRecord`](/sdk-reference/data/cohort#wsirecord) instance.
## PatientRecord
```python theme={null}
@dataclass
class PatientRecord()
```
Groups all WSIs belonging to a single patient.
**Attributes**:
* `patient_id` - Unique patient identifier.
* `wsis` - Ordered list of WSI records (by timepoint).
* `labels` - Patient-level categorical annotations.
* `metadata` - Patient-level clinical metadata.
## Cohort
```python theme={null}
class Cohort(records: list[WSIRecord] | None = None)
```
Typed cohort manifest — single source of truth for an experiment.
A cohort is an ordered collection of [`WSIRecord`](/sdk-reference/data/cohort#wsirecord) entries grouped by patient. It stores all information needed to reproduce and resume an inference run: file locations, pairing logic, timepoints, per-model outputs, labels, and clinical metadata.
Construction: Use [`from_csv`](/sdk-reference/data/cohort#from_csv), [`from_directories`](/sdk-reference/data/cohort#from_directories), or [`load`](/sdk-reference/data/cohort#load) rather than calling the constructor directly.
Initial list of [`WSIRecord`](/sdk-reference/data/cohort#wsirecord) entries.
***
#### from\_csv
```python theme={null}
@classmethod
@log_failures
def from_csv(cls,
path: str | Path,
*,
wsi_dir: str | Path | None = None,
bulk_rna_dir: str | Path | None = None,
columns: dict[str, str] | None = None) -> Cohort
```
Creates a cohort from a user-provided CSV manifest.
Required columns:
* `wsi_id`: WSI identifier (filename stem or full name).
Optional columns:
* `patient_id`: if absent, derived from `wsi_id`.
* `bulk_rna_id`: paired RNA sample identifier.
* `timepoint`: temporal label.
* `wsi_path`: explicit path override.
* `bulk_rna_path`: explicit path override.
* Any other columns are treated as labels if prefixed
with `label_` or as metadata otherwise.
When *wsi\_dir* or *bulk\_rna\_dir* are provided, paths are resolved by joining the directory with the corresponding ID (plus matching extension found on disk).
The *columns* dict maps canonical field names to the actual CSV column names. Only the keys that differ from the defaults need to be specified. For example:
Recognised canonical keys: `wsi_id`, `patient_id`, `bulk_rna_id`, `timepoint`, `wsi_path`, `bulk_rna_path`.
```python theme={null}
Cohort.from_csv(
"cohort.csv",
columns={
"wsi_id": "imageid",
"patient_id": "subject_id",
},
)
```
Path to the CSV manifest.
Directory to resolve WSI paths from.
Directory to resolve bulk RNA paths from.
Mapping of canonical field names to actual CSV column names. Unmapped fields fall back to their canonical name.
**Returns**:
A populated [`Cohort`](/sdk-reference/data/cohort#cohort).
***
#### from\_directories
```python theme={null}
@classmethod
@log_failures
def from_directories(
cls,
wsi_dir: str | Path,
bulk_rna_dir: str | Path | None = None,
*,
mask_dir: str | Path | None = None,
patient_id_fn: Callable[[str], str] | None = None) -> Cohort
```
Creates a cohort by auto-matching files from directories.
WSI and bulk RNA files are paired by **exact filename stem** equality (case-sensitive). Patient IDs default to the shared stem. When multiple wsis share the same patient ID, they are assigned sequential timepoints (`t0`, `t1`, ...) in alphabetical order.
Directory containing WSI files.
Directory containing bulk RNA files. When `None`, wsis are registered without RNA.
Directory containing pre-computed masks (PNG files whose stem matches the WSI stem).
Optional callable that maps a filename stem to a patient ID.
**Returns**:
A populated [`Cohort`](/sdk-reference/data/cohort#cohort).
***
#### save
```python theme={null}
@log_failures
def save(path: str | Path) -> Path
```
Persists the cohort manifest to a YAML file.
The YAML includes all inputs, per-model outputs, status, labels, and metadata — everything needed to resume.
Destination file path.
**Returns**:
The resolved output path.
***
#### load
```python theme={null}
@classmethod
@log_failures
def load(cls, path: str | Path) -> Cohort
```
Loads a cohort from a previously saved YAML manifest.
Path to the YAML manifest file.
**Returns**:
A fully-populated [`Cohort`](/sdk-reference/data/cohort#cohort).
***
#### upsert
```python theme={null}
def upsert(wsi_id: str,
*,
patient_id: str | None = None,
wsi_path: Path | None = None,
bulk_rna_path: Path | None = None,
mask_path: Path | None = None,
timepoint: str | None = None) -> WSIRecord
```
Inserts or updates a WSI record.
If a record with *wsi\_id* exists, updates the non-None fields. Otherwise creates a new record.
WSI identifier.
Patient identifier (defaults to wsi\_id).
Path to WSI file.
Path to bulk RNA file.
Path to tissue mask.
Timepoint label.
**Returns**:
The inserted or updated [`WSIRecord`](/sdk-reference/data/cohort#wsirecord).
***
#### get\_wsi
```python theme={null}
def get_wsi(wsi_id: str) -> WSIRecord | None
```
Returns the record for *wsi\_id*, or `None`.
Unique WSI identifier (typically the file stem).
**Returns**:
The matching [`WSIRecord`](/sdk-reference/data/cohort#wsirecord), or `None` if not found.
***
#### link\_bulk\_rna
```python theme={null}
def link_bulk_rna(bulk_rna_dir: str | Path,
*,
extensions: set[str] | None = None,
separator: str | None = None,
gene_column: str | None = None,
value_column: str | None = None,
strip_version: bool = False) -> int
```
Links bulk RNA files to existing WSI records by stem.
Scans *bulk\_rna\_dir* for recognised RNA files and pairs them with existing records by exact filename-stem equality. Records that already have a `bulk_rna_path` are skipped.
This enables late-binding of bulk RNA data: build a cohort from WSIs first, then call this method to attach RNA when it becomes available.
Directory containing bulk RNA files.
File extensions to match. Defaults to `{".csv", ".tsv"}`.
Column delimiter for parsing. When `None` the separator is inferred from the file extension.
Column name containing gene identifiers. Required (with `value_column`) for long-format files (e.g. GDC/TCGA gene quantification TSVs).
Column name containing expression values to read (e.g. `"tpm_unstranded"`).
When `True`, strips version suffixes from gene identifiers (e.g. `ENSG…00003.15` → `ENSG…00003`).
**Returns**:
Number of records that were linked.
***
#### add\_labels
```python theme={null}
def add_labels(labels: dict[str, dict[str, Any]],
*,
by: str = "patient_id") -> None
```
Adds categorical labels to records.
Mapping of `{identifier: {label_name: value}}`.
Key to match on — `"patient_id"` or `"wsi_id"`.
***
#### add\_metadata
```python theme={null}
def add_metadata(metadata: dict[str, dict[str, Any]],
*,
by: str = "patient_id") -> None
```
Adds clinical/treatment metadata to records.
Mapping of `{identifier: {field: value}}`.
Key to match on — `"patient_id"` or `"wsi_id"`.
***
#### add\_labels\_from\_csv
```python theme={null}
@log_failures
def add_labels_from_csv(path: str | Path, *, by: str = "patient_id") -> None
```
Adds labels and metadata from a CSV file.
The CSV must have a column matching *by* (default `patient_id`). Columns prefixed with `label_` are treated as labels; remaining columns as metadata.
Path to the labels CSV.
Join key column name.
***
#### num\_patients
```python theme={null}
@property
def num_patients() -> int
```
Number of unique patients in the cohort.
***
#### patients
```python theme={null}
@property
def patients() -> dict[str, PatientRecord]
```
Patient-indexed view of the cohort.
***
#### get\_patient
```python theme={null}
def get_patient(patient_id: str) -> PatientRecord
```
Returns the [`PatientRecord`](/sdk-reference/data/cohort#patientrecord) for a given patient.
Patient identifier string.
**Returns**:
The matching [`PatientRecord`](/sdk-reference/data/cohort#patientrecord).
**Raises**:
* `KeyError` - If *patient\_id* is not found.
***
#### wsi\_ids
```python theme={null}
@property
def wsi_ids() -> list[str]
```
Ordered list of all WSI IDs.
***
#### patient\_ids
```python theme={null}
@property
def patient_ids() -> list[str]
```
Ordered list of unique patient IDs.
***
#### wsi\_paths
```python theme={null}
@property
def wsi_paths() -> list[Path | None]
```
Ordered list of WSI paths (may contain `None`).
***
#### pending
```python theme={null}
def pending(model_name: str, stage: InferenceMode | str) -> list[WSIRecord]
```
Returns wsis that have not completed a stage for a model.
Checks whether the output file exists on disk. Records with missing WSI paths are excluded.
Model identifier.
The output stage as an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) (or a mode string; legacy short spellings are accepted).
**Returns**:
List of [`WSIRecord`](/sdk-reference/data/cohort#wsirecord) entries still needing
processing.
***
#### to\_csv
```python theme={null}
@log_failures
def to_csv(path: str | Path) -> Path
```
Writes input columns to a CSV file (labels + metadata).
For full persistence including outputs, use [`save`](/sdk-reference/data/cohort#save).
Destination file path.
**Returns**:
The resolved output path.
***
#### summary
```python theme={null}
def summary() -> str
```
Returns a human-readable summary of the cohort.
**Returns**:
Multi-line summary string.
# bioptimus.data.multi_modal.multi_modal_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/multi_modal/multi_modal_dataset
Multi-modal dataset for joint histology and omics inference.
Wraps a single WSI together with optional bulk RNA data, providing lazy patch access and a PyTorch-compatible `__getitem__` interface.
## MultiModalData
```python theme={null}
class MultiModalData(wsi_path: Union[str, Path],
extractor: TileExtractor,
bulk_rna_path: Union[str, Path] | None = None,
bulk_rna_gene_names: List[str] | None = None,
image_transform: Callable[[np.ndarray], Any] | None = None,
bulk_rna_transform: str | None = None,
bulk_rna_separator: str | None = None,
bulk_rna_gene_column: str | None = None,
bulk_rna_value_column: str | None = None,
bulk_rna_strip_version: bool = False,
wsi_backend: str | None = None)
```
Wraps a single Whole Slide Image with its extraction plan.
On construction the slide is opened via [`WSI`](/sdk-reference/io/wsi/factory#wsi) and the extractor is fitted + executed to produce a list of [`RegionSpec`](/sdk-reference/io/wsi/types#regionspec). Each spec describes one tile/patch location. Patches are read lazily via [`get_patch`](/sdk-reference/data/multi_modal/multi_modal_dataset#get_patch).
Path to the WSI file (any format supported by [`WSI`](/sdk-reference/io/wsi/factory#wsi)).
Optional path to bulk RNA CSV.
A *configured* [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor). A fresh `fit_extract` is called for every slide so the same extractor object can be reused across slides.
Optional callable applied to the raw `np.ndarray` patch **before** it is returned. Receives `(H, W, C)` uint8 and should return a transformed array (or tensor).
**Attributes**:
* `path` *Path* - Resolved slide path.
* `bulk_rna_path` *Path | None* - Resolved path to bulk RNA CSV.
* `reader` *WSIReader* - Open reader for the slide.
* `specs` *List\[RegionSpec]* - Extraction plan (one entry per patch).
* `slide_name` *str* - Stem of the slide filename.
**Example**:
```python theme={null}
wsi = MultiModalData("tissue.svs", extractor)
patch, meta = wsi.get_patch(0)
```
***
#### \_\_len\_\_
```python theme={null}
def __len__() -> int
```
Returns the number of patches in the extraction plan.
**Returns**:
* `int` - Total patch count for this slide.
**Example**:
```python theme={null}
>>> wsi = MultiModalData("slide.svs", extractor)
>>> len(wsi)
150
```
***
#### get\_patch
```python theme={null}
def get_patch(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Reads a single patch from the slide.
Index into `specs`.
**Returns**:
A tuple `(patch, metadata)` where *patch* is an
`np.ndarray` of shape `(H, W, C)` (or whatever the
transform returns), and *metadata* is a dict with at
minimum `source`, `x`, `y`, `width`, `height`,
`slide_name`, and `tissue_ratio`.
**Raises**:
* `IndexError` - If *idx* is out of range.
***
#### close
```python theme={null}
def close() -> None
```
Closes the underlying WSI reader and releases resources.
**Example**:
```python theme={null}
>>> wsi.close()
```
# bioptimus.data.omics.bulkrna
Source: https://docs.bioptimus.com/sdk-reference/data/omics/bulkrna
Bulk RNA-seq data reader.
Provides [`BulkRNAData`](/sdk-reference/data/omics/bulkrna#bulkrnadata) — a lightweight wrapper around a CSV/TSV of gene expression counts for a single patient sample.
**Wide format** (default) — one row per sample, genes as columns:
```text theme={null}
sample_id,gene_1,gene_2,...,gene_N
patient_001,12.3,4.5,...,0.1
```
The first column is treated as the sample identifier (index). All remaining columns are gene expression values. Only the first row is used; additional rows are ignored.
**Long format** — one row per gene, activated by setting `gene_column` and `value_column`:
```text theme={null}
gene_name,raw_count,normalized
gene_1,12.3,0.5
gene_2,4.5,0.2
```
The separator is inferred from the file extension (`.tsv` → tab, otherwise comma) but can be overridden via `separator`.
## BulkRNAData
```python theme={null}
class BulkRNAData(path: Union[str, Path],
gene_names: list[str] | None = None,
omic_transform: OmicsPreprocessor | None = None,
separator: str | None = None,
gene_column: str | None = None,
value_column: str | None = None,
strip_version: bool = False)
```
Single-sample bulk RNA-seq counts for PyTorch pipelines.
Reads a CSV or TSV with gene expression counts and stores the values as a contiguous `np.float32` array for zero-copy tensor creation via `counts`.
Supports two layouts:
* **Wide** (default): one row per sample, genes as columns.
The first column is the sample identifier (index).
* **Long**: one row per gene, activated by setting
`gene_column` and `value_column`.
Path to the CSV or TSV file.
Optional gene list to filter/align columns to. Missing genes are zero-filled, extras are dropped. When `None` all columns are kept.
Optional preprocessing transform applied when accessing `counts`.
Column delimiter. When `None` the separator is inferred from the file extension (`.tsv` → tab, otherwise comma).
Column name that contains gene identifiers. Required for long-format files.
Column name that contains expression values to read. Required for long-format files.
**Attributes**:
* `sample_name` - Sample identifier (first row index in wide format, or the file stem in long format).
* `gene_names` - Ordered list of gene column names.
***
#### num\_genes
```python theme={null}
@property
def num_genes() -> int
```
Number of gene columns.
***
#### counts
```python theme={null}
@property
def counts() -> list[float]
```
Gene counts as a list of floats.
***
#### get\_counts
```python theme={null}
def get_counts(sample_name: str | None = None) -> list[float]
```
Returns gene counts as a flat sequence.
When no `omic_transform` is configured, returns a plain `list[float]`. With a transform, returns the transform output (typically an `np.ndarray`).
Ignored (kept for backward compatibility).
**Returns**:
Gene expression values in column order.
# bioptimus.data.tile_spec
Source: https://docs.bioptimus.com/sdk-reference/data/tile_spec
Tile specification dataclass.
Defines [`TileSpec`](/sdk-reference/data/tile_spec#tilespec), a pure data structure describing tile geometry (size, stride, resolution, measurement unit). This module lives in the data layer so it can be imported by both `bioptimus.extraction` and `bioptimus.models` without creating circular dependencies.
For backward compatibility, [`TileSpec`](/sdk-reference/data/tile_spec#tilespec) is re-exported from `bioptimus.extraction.wsi.types`.
## TileSpec
```python theme={null}
@dataclass(frozen=True)
class TileSpec()
```
Defines the specifications of a tile within a WSI for extraction.
**Attributes**:
* `size` - The size of the tile as `(width, height)` or a single integer for both dimensions.
* `stride` - The stride of the tile as `(stride_width, stride_height)` or a single integer for both dimensions.
* `resolution` - The resolution at which the tile is defined.
* `unit` - The unit of measurement for the tile dimensions.
**Example**:
```python theme={null}
>>> spec = TileSpec(
... size=(256, 256), stride=(128, 128),
... resolution=Resolution(level=0),
... unit=MeasurementUnit.PIXELS,
... )
>>> print(spec.size)
(256, 256)
```
***
#### size
(width, height)
***
#### stride
(stride\_width, stride\_height)
***
#### width
```python theme={null}
@property
def width() -> int
```
Returns the tile width.
***
#### height
```python theme={null}
@property
def height() -> int
```
Returns the tile height.
***
#### stride\_width
```python theme={null}
@property
def stride_width() -> int
```
Returns the horizontal stride.
***
#### stride\_height
```python theme={null}
@property
def stride_height() -> int
```
Returns the vertical stride.
***
#### scale\_to\_reference
```python theme={null}
def scale_to_reference(downsample_ratio: float, mpp: MPP) -> TileSpec
```
Returns a new TileSpec scaled to reference-level pixels.
Ratio of the reference-level downsample to the target-resolution downsample.
Microns-per-pixel at the target resolution.
**Returns**:
A new `TileSpec` in reference-level pixels.
**Raises**:
* `ValueError` - If `unit` is not `PIXELS` or `UM`, or if the scaled size or stride rounds to zero (the requested geometry is finer than one reference-level pixel).
# bioptimus.data.wsi
Source: https://docs.bioptimus.com/sdk-reference/data/wsi
WSI data wrappers for patch-level access and PyTorch integration.
Provides [`WSIData`](/sdk-reference/data/wsi/wsi_dataset#wsidata) (single-slide wrapper) and [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset) (multi-slide collection with flat patch indexing). For PyTorch DataLoader integration, see `PytorchTileDataset`.
**Example**:
```python theme={null}
>>> from bioptimus.data.wsi import WSIData, WSIDataset, PytorchTileDataset
>>> wsi = WSIData("slide.svs", extractor)
>>> patch, meta = wsi.get_patch(0)
```
# bioptimus.data.wsi.torch_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/wsi/torch_dataset
PyTorch adapter for WSI tile datasets.
Provides [`PytorchTileDataset`](/sdk-reference/data/wsi/torch_dataset#pytorchtiledataset), a `torch.utils.data.Dataset` adapter that wraps `WSIDataset` for use with `torch.utils.data.DataLoader`. Two sampling modes are supported:
* `"sequential"` — patches in slide-load order (deterministic).
* `"random"` — same patches, but indices are shuffled each epoch.
Both modes visit every patch exactly once per epoch.
The *data* argument accepts three forms:
* A ready-made [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset).
* A directory path (`str` or `Path`) — all supported WSI files inside
it are discovered and loaded.
* A list of file paths — each path is opened as a slide.
**Example**:
```python theme={null}
>>> from bioptimus.data.wsi.torch_dataset import PytorchTileDataset
>>> ds = PytorchTileDataset(wsi_dataset, sampling="sequential")
>>> ds = PytorchTileDataset("slides/", extractor=ext, sampling="random", seed=0)
>>> loader = DataLoader(ds, batch_size=64, num_workers=4)
```
## SamplingMode
```python theme={null}
class SamplingMode(str, Enum)
```
Patch sampling strategy.
**Attributes**:
* `SEQUENTIAL` - Iterate over every patch in slide-load order.
* `RANDOM` - Shuffle the global patch indices each epoch.
**Example**:
```python theme={null}
>>> mode = SamplingMode("random")
>>> mode == SamplingMode.RANDOM
True
```
## PytorchTileDataset
```python theme={null}
class PytorchTileDataset(data: Any,
extractor: Optional[TileExtractor] = None,
sampling: Union[str, SamplingMode] = SamplingMode.SEQUENTIAL,
transform: Optional[Callable[[np.ndarray], Any]] = None,
seed: Optional[int] = None)
```
PyTorch-compatible dataset over WSI tiles with configurable ordering.
Accepts a [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset), a directory path, or a list of file paths. When a path or list is given, *extractor* is required so the slides can be opened and tiled automatically.
`"sequential"` (default) Patches are returned in the order slides were loaded — slide 0 patch 0, slide 0 patch 1, …, slide N patch M.
`"random"` The same set of patches, but the global indices are shuffled. Call [`shuffle`](/sdk-reference/data/wsi/torch_dataset#shuffle) between epochs (or at init) to re-randomise.
In both modes `len()` equals the total patch count, and every patch is visited exactly once per full iteration.
One of:
* A [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset) instance (used directly).
* A `str` or `Path` pointing to a **directory** — all supported WSI files are discovered and loaded.
* A `list` of file paths — each is opened as a slide.
- `extractor` - Required when *data* is a path or list of paths. A configured [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) used to tile each slide.
- `sampling` - `"sequential"` or `"random"` (default `"sequential"`).
- `transform` - Optional callable applied to the `np.ndarray` `(H, W, C)` patch.
- `seed` - Optional RNG seed for reproducible shuffling.
**Raises**:
* `ValueError` - If *data* is a path/list but *extractor* is not provided.
* `FileNotFoundError` - If a directory path contains no supported WSIs.
**Example**:
```python theme={null}
ds = PytorchTileDataset(wsi_dataset, sampling="sequential")
ds = PytorchTileDataset("slides/", extractor=ext, sampling="random", seed=0)
ds = PytorchTileDataset(["a.svs", "b.svs"], extractor=ext)
```
***
#### shuffle
```python theme={null}
def shuffle(seed: Optional[int] = None) -> None
```
Re-shuffles the global index mapping.
Call this between epochs to get a different ordering. Has no effect when `sampling` is `"sequential"`.
Optional RNG seed for reproducibility.
**Example**:
```python theme={null}
>>> ds = PytorchTileDataset(dataset, sampling="random", seed=42)
>>> ds.shuffle(seed=123) # new ordering for next epoch
```
***
#### \_\_len\_\_
```python theme={null}
def __len__() -> int
```
Returns the total number of patches in the dataset.
**Returns**:
* `int` - Total patch count across all slides.
**Example**:
```python theme={null}
>>> len(ds)
500
```
***
#### \_\_getitem\_\_
```python theme={null}
def __getitem__(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Returns a `(patch, metadata)` tuple for a global patch index.
Global patch index (0-based).
**Returns**:
Tuple\[Any, Dict\[str, Any]]: The patch array (or transformed
output) and a metadata dictionary.
**Example**:
```python theme={null}
>>> patch, meta = ds[0]
>>> meta["slide_name"]
'slide_001'
```
# bioptimus.data.wsi.wsi_dataset
Source: https://docs.bioptimus.com/sdk-reference/data/wsi/wsi_dataset
WSI Data & Dataset Module.
Provides [`WSIData`](/sdk-reference/data/wsi/wsi_dataset#wsidata) (single-slide wrapper) and [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset) (multi-slide collection) that bridge the bioptimus I/O + extraction layers into structures ready for feature extraction or training.
Typical usage:
```python theme={null}
from bioptimus.data.wsi import WSIData, WSIDataset
from bioptimus.extraction.wsi.tile_extraction import TileExtractor
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.io.wsi.types import Level, MeasurementUnit
spec = TileSpec(size=(256, 256), stride=(256, 256),
resolution=Level(0), unit=MeasurementUnit.PIXELS)
extractor = TileExtractor(tile_spec=spec, mask_threshold=0.5)
dataset = WSIDataset.from_paths(
paths=["slide_1.svs", "slide_2.svs"],
extractor=extractor,
)
# Iterate over all patches (flat indexing across slides)
for i in range(len(dataset)):
patch, meta = dataset[i] # np.ndarray (H, W, C), dict
```
## WSIData
```python theme={null}
class WSIData(path: Union[str, Path],
extractor: TileExtractor,
transform: Optional[Callable[[np.ndarray], Any]] = None)
```
Wraps a single Whole Slide Image with its extraction plan.
On construction the slide is opened via [`WSI`](/sdk-reference/io/wsi/factory#wsi) and the extractor is fitted + executed to produce a list of [`RegionSpec`](/sdk-reference/io/wsi/types#regionspec). Each spec describes one tile/patch location. Patches are read lazily via [`get_patch`](/sdk-reference/data/wsi/wsi_dataset#get_patch).
Path to the WSI file (any format supported by [`WSI`](/sdk-reference/io/wsi/factory#wsi)).
A *configured* [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor). A fresh `fit_extract` is called for every slide so the same extractor object can be reused across slides.
Optional callable applied to the raw `np.ndarray` patch **before** it is returned. Receives `(H, W, C)` uint8 and should return a transformed array (or tensor).
**Attributes**:
* `path` *Path* - Resolved slide path.
* `reader` *WSIReader* - Open reader for the slide.
* `specs` *List\[RegionSpec]* - Extraction plan (one entry per patch).
* `slide_name` *str* - Stem of the slide filename.
**Example**:
```python theme={null}
wsi = WSIData("tissue.svs", extractor)
patch, meta = wsi.get_patch(0)
```
***
#### \_\_len\_\_
```python theme={null}
def __len__() -> int
```
Returns the number of patches in the extraction plan.
**Returns**:
* `int` - Total patch count for this slide.
**Example**:
```python theme={null}
>>> wsi = WSIData("slide.svs", extractor)
>>> len(wsi)
150
```
***
#### get\_patch
```python theme={null}
def get_patch(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Reads a single patch from the slide.
Index into `specs`.
**Returns**:
A tuple `(patch, metadata)` where *patch* is an
`np.ndarray` of shape `(H, W, C)` (or whatever the
transform returns), and *metadata* is a dict with at
minimum `source`, `x`, `y`, `width`, `height`,
`slide_name`, and `tissue_ratio`.
**Raises**:
* `IndexError` - If *idx* is out of range.
***
#### close
```python theme={null}
def close() -> None
```
Closes the underlying WSI reader and releases resources.
**Example**:
```python theme={null}
>>> wsi.close()
```
## WSIDataset
```python theme={null}
class WSIDataset(data_list: List[WSIData])
```
A collection of [`WSIData`](/sdk-reference/data/wsi/wsi_dataset#wsidata) wrappers with flat patch indexing.
Supports two access patterns:
* **Slide-level** — `dataset.slides[i]` returns a [`WSIData`](/sdk-reference/data/wsi/wsi_dataset#wsidata).
* **Patch-level** (flat) — `dataset[k]` maps a global patch index
across all slides and returns `(patch, metadata)`.
The flat indexing uses a cumulative-sum lookup (`O(log N)` via `bisect.bisect`) so random access is fast regardless of how many slides are loaded.
Pre-built [`WSIData`](/sdk-reference/data/wsi/wsi_dataset#wsidata) instances.
**Example**:
```python theme={null}
dataset = WSIDataset.from_paths(paths, extractor)
len(dataset) # total patch count across all slides
patch, meta = dataset[42]
slide = dataset.slides[0]
```
***
#### from\_paths
```python theme={null}
@classmethod
def from_paths(
cls,
paths: Sequence[Union[str, Path]],
extractor: TileExtractor,
transform: Optional[Callable[[np.ndarray], Any]] = None) -> WSIDataset
```
Creates a dataset by opening and extracting every slide.
Iterable of WSI file paths.
Shared [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) (re-fitted per slide).
Optional callable applied to each extracted tile array.
**Returns**:
A new [`WSIDataset`](/sdk-reference/data/wsi/wsi_dataset#wsidataset).
**Example**:
```python theme={null}
dataset = WSIDataset.from_paths(
glob.glob("slides/*.svs"), extractor
)
```
***
#### \_\_len\_\_
```python theme={null}
def __len__() -> int
```
Returns the total number of patches across all slides.
**Returns**:
* `int` - Global patch count.
**Example**:
```python theme={null}
>>> len(dataset)
500
```
***
#### \_\_getitem\_\_
```python theme={null}
def __getitem__(idx: int) -> Tuple[Any, Dict[str, Any]]
```
Returns `(patch, metadata)` for a global patch index.
Global patch index (0-based).
**Raises**:
* `IndexError` - If *idx* is out of range.
***
#### num\_slides
```python theme={null}
@property
def num_slides() -> int
```
Returns the number of loaded slides.
**Returns**:
* `int` - Slide count.
**Example**:
```python theme={null}
>>> dataset.num_slides
3
```
***
#### slide\_patch\_counts
```python theme={null}
def slide_patch_counts() -> Dict[str, int]
```
Returns a mapping of slide names to their patch counts.
**Returns**:
Dict\[str, int]: `{slide_name: patch_count}` for every slide.
**Example**:
```python theme={null}
>>> dataset.slide_patch_counts()
{'slide_001': 150, 'slide_002': 350}
```
***
#### close
```python theme={null}
def close() -> None
```
Closes all underlying WSI readers.
**Example**:
```python theme={null}
>>> dataset.close()
```
# bioptimus.extraction.wsi
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi
WSI extraction sub-package.
Provides tile-based and mega-tile-based region extraction from Whole Slide Images with tissue-mask filtering and multiple export formats.
Classes: TileExtractor: Grid-based tile extractor with tissue-mask filtering. TileSpec: Tile size, stride, resolution, and unit specification. MegaTileExtractor: Mega-tile (grid-of-tiles) extractor. MegaTileSpec: Mega-tile grid shape, layout, stride, and ratio bounds. MegaTileRegion: Container for an extracted mega tile and its tiles. GridLayout: Rectangular or hexagonal layout enum.
**Example**:
```python theme={null}
>>> from bioptimus.extraction.wsi import TileExtractor, TileSpec
>>> from bioptimus.extraction.wsi import MegaTileExtractor, MegaTileSpec, GridLayout
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> spec = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> extractor = TileExtractor(tile_spec=spec)
```
# bioptimus.extraction.wsi.megatile_extraction
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/megatile_extraction
WSI Mega-Tile Extraction Module.
This module provides the [`MegaTileExtractor`](/sdk-reference/extraction/wsi/megatile_extraction#megatileextractor) class, which groups individual tile positions from a Whole Slide Image into *mega tiles* — fixed- size grids of `rows × cols` tiles. It builds on the tile-level tissue filtering from [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) and adds a second filtering stage: only mega tiles where the fraction of tissue-positive tiles falls within a configurable `[min_valid_tiles_ratio, max_valid_tiles_ratio]` band are kept.
The class follows the same **scikit-learn–style** lifecycle as [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor):
1. **Configure** — instantiate with a [`MegaTileSpec`](/sdk-reference/extraction/wsi/types#megatilespec), optional tissue mask, and threshold.
2. **Fit** — call [`fit`](/sdk-reference/extraction/wsi/megatile_extraction#fit) with an open `WSIReader`.
3. **Extract** — call [`extract`](/sdk-reference/extraction/wsi/megatile_extraction#extract) (or the shortcut [`fit_extract`](/sdk-reference/extraction/wsi/megatile_extraction#fit_extract)). Results are stored in `megatile_regions_` and also returned.
4. **Export** — call [`to_csv`](/sdk-reference/extraction/wsi/megatile_extraction#to_csv) or [`to_json`](/sdk-reference/extraction/wsi/megatile_extraction#to_json) to persist the results.
Typical usage example:
```python theme={null}
>>> from bioptimus.extraction.wsi.megatile_extraction import MegaTileExtractor
>>> from bioptimus.extraction.wsi.types import MegaTileSpec, TileSpec, GridLayout
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> tile = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> mega = MegaTileSpec(megatile_shape=(5, 5), tile_spec=tile,
... grid_layout=GridLayout.RECTANGULAR,
... min_valid_tiles_ratio=0.5)
>>> extractor = MegaTileExtractor(megatile_spec=mega,
... mask=tissue_mask, mask_threshold=0.5)
>>> with SomeWSIReader("path/to/slide.svs") as reader:
... megatiles = extractor.fit_extract(reader)
... extractor.to_csv("output/")
... extractor.to_json("output/")
```
## MegaTileExtractor
```python theme={null}
class MegaTileExtractor(megatile_spec: MegaTileSpec,
mask: Union[np.ndarray, None] = None,
mask_provider: Union[TissueMaskProvider, None] = None,
mask_threshold: Union[float, None] = 0.5,
save_mask: bool = False,
max_megatiles: Union[int, None] = None)
```
Extracts mega tiles (grids of tiles) from a WSI, filtered by tissue.
A *mega tile* is a rectangular or hexagonal grid of individual tiles. Extraction proceeds in two stages:
1. **Tile-level filtering** — a binary tissue mask and
`mask_threshold` determine which individual tile positions contain sufficient tissue.
2. **Mega-tile-level filtering** — only mega tiles where the fraction of
valid (tissue-positive) tiles falls within `[min_valid_tiles_ratio, max_valid_tiles_ratio]` are retained.
The extractor follows the same lifecycle as [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor):
Or via method chaining:
**Configuration attributes** (set at init):
```python theme={null}
extractor = MegaTileExtractor(megatile_spec=spec, mask=mask)
extractor.fit(reader)
megatiles = extractor.extract()
extractor.to_csv("output/")
```
```python theme={null}
megatiles = MegaTileExtractor(megatile_spec=spec).fit_extract(reader)
```
**Attributes**:
* `megatile_spec` *MegaTileSpec* - Mega-tile grid shape, per-tile spec, layout, stride, and valid-ratio bounds.
* `mask` *np.ndarray | None* - Binary tissue mask.
* `mask_provider` *TissueMaskProvider | None* - Per-slide mask generator.
* `mask_threshold` *float | None* - Per-tile tissue fraction threshold.
* `save_mask` *bool* - Whether to store per-tile masks.
* `max_megatiles` *int | None* - Optional cap on returned mega tiles.
**Fitted attributes** (populated by [`fit`](/sdk-reference/extraction/wsi/megatile_extraction#fit) / [`extract`](/sdk-reference/extraction/wsi/megatile_extraction#extract)):
**Attributes**:
* `source_` *WSIReader* - Bound WSI reader.
* `slide_name_` *str* - Stem of the slide filename.
* `megatile_regions_` *List\[MegaTileRegion]* - Extracted mega tiles.
***
#### fit
```python theme={null}
@log_failures
def fit(source: WSIReader) -> "MegaTileExtractor"
```
Binds the extractor to an open WSI reader.
Stores the reader and slide name. If a `mask_provider` is set, its [`generate`](/sdk-reference/extraction/wsi/megatile_extraction#generate) method is called to create a per-slide tissue mask.
An open WSI reader.
**Returns**:
* `MegaTileExtractor` - `self` for method chaining.
**Example**:
```python theme={null}
>>> extractor.fit(reader)
>>> extractor.slide_name_
'slide_001'
```
***
#### extract
```python theme={null}
@log_failures
def extract(source: Optional[WSIReader] = None) -> List[MegaTileRegion]
```
Extracts mega-tile regions from a WSI.
The method operates in five stages:
1. **Reference-level setup** — as in [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor).
2. **Mask scaling** — resize tissue mask to the reference level.
3. **Tile-spec scaling** — convert tile dimensions to reference-level
pixels.
4. **Per-tile tissue evaluation** — build an integral image and
compute the tissue fraction for every candidate tile position.
5. **Mega-tile grouping** — slide a `rows × cols` window (in tile
units) across the tile grid, collect per-tile validity, and filter by `[min_valid_tiles_ratio, max_valid_tiles_ratio]`.
Results are stored in `megatile_regions_` **and** returned.
An open WSI reader. When given, the extractor is fitted automatically. Otherwise [`fit`](/sdk-reference/extraction/wsi/megatile_extraction#fit) must have been called.
**Returns**:
* `List[MegaTileRegion]` - Mega-tile regions that pass filtering.
**Raises**:
* `RuntimeError` - If no source is provided and [`fit`](/sdk-reference/extraction/wsi/megatile_extraction#fit) has not been called.
**Example**:
```python theme={null}
>>> megatiles = extractor.fit(reader).extract()
>>> len(megatiles)
12
```
***
#### fit\_extract
```python theme={null}
@log_failures
def fit_extract(source: WSIReader) -> List[MegaTileRegion]
```
Convenience method: [`fit`](/sdk-reference/extraction/wsi/megatile_extraction#fit) + [`extract`](/sdk-reference/extraction/wsi/megatile_extraction#extract) in one call.
An open WSI reader.
**Returns**:
* `List[MegaTileRegion]` - Extracted mega-tile regions.
**Example**:
```python theme={null}
>>> megatiles = extractor.fit_extract(reader)
>>> len(megatiles)
12
```
***
#### to\_csv
```python theme={null}
@log_failures
def to_csv(output_dir: Union[str, Path],
regions: Optional[List[MegaTileRegion]] = None) -> Path
```
Saves mega-tile metadata to a CSV file named after the slide.
Each row represents one *tile* within a mega tile. Mega-tile–level columns (`megatile_index`, `megatile_top`, etc.) allow grouping.
Destination directory (created if needed).
Override the stored regions. Defaults to `megatile_regions_`.
**Returns**:
* `Path` - Path of the written CSV file.
**Raises**:
* `RuntimeError` - If no regions are available.
**Example**:
```python theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.to_csv("output/")
PosixPath('output/slide_001.csv')
```
***
#### to\_json
```python theme={null}
@log_failures
def to_json(output_dir: Union[str, Path],
regions: Optional[List[MegaTileRegion]] = None,
indent: int = 2) -> Path
```
Saves mega-tile metadata to a JSON file named after the slide.
The output is a JSON array of mega-tile objects, each containing the mega-tile location, grid info, valid ratio, and a nested `tiles` array with per-tile metadata.
Destination directory (created if needed).
Override the stored regions.
JSON pretty-printing indent.
**Returns**:
* `Path` - Path of the written JSON file.
**Raises**:
* `RuntimeError` - If no regions are available.
**Example**:
```python theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.to_json("output/")
PosixPath('output/slide_001_megatiles.json')
```
# bioptimus.extraction.wsi.tile_extraction
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/tile_extraction
WSI Tile Extraction Module.
This module provides the [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) class, which generates a grid of tile region specifications from a Whole Slide Image (WSI). It uses a binary tissue mask to filter out background tiles and supports resolution scaling via [`Level`](/sdk-reference/io/wsi/types#level), [`Magnification`](/sdk-reference/io/wsi/types#magnification), or [`MPP`](/sdk-reference/io/wsi/types#mpp).
The class follows a **scikit-learn–style** API:
1. **Configure** — instantiate with a [`TileSpec`](/sdk-reference/data/tile_spec#tilespec), optional tissue mask, and threshold.
2. **Fit** — call [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) with an open `WSIReader` to bind the extractor to a specific slide.
3. **Extract** — call [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract) (or the shortcut [`fit_extract`](/sdk-reference/extraction/wsi/tile_extraction#fit_extract)) to compute tile positions. Results are stored in `region_specs_` and also returned.
4. **Export** — call [`save`](/sdk-reference/extraction/wsi/tile_extraction#save), [`to_csv`](/sdk-reference/extraction/wsi/tile_extraction#to_csv), or [`to_json`](/sdk-reference/extraction/wsi/tile_extraction#to_json) to persist the results. These methods use the fitted state so you do not need to pass the reader or regions again.
Typical usage example:
```python theme={null}
>>> from bioptimus.extraction.wsi.tile_extraction import TileExtractor
>>> from bioptimus.extraction.wsi.types import TileSpec
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> spec = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> extractor = TileExtractor(tile_spec=spec, mask=tissue_mask, mask_threshold=0.5)
>>> with SomeWSIReader("path/to/slide.svs") as reader:
... extractor.fit(reader).extract()
... extractor.to_csv("output/") # writes output/slide.csv
... extractor.to_json("output/") # writes output/slide.json
... extractor.save("tiles/")
```
## TileExtractor
```python theme={null}
class TileExtractor(tile_spec: TileSpec,
mask: Union[np.ndarray, None] = None,
mask_provider: Union[TissueMaskProvider, None] = None,
mask_threshold: Union[float, None] = 0.5,
save_mask: bool = False,
coordinates: Union[Sequence[tuple[int, int]], None] = None)
```
Extracts a grid of tile regions from a WSI, filtered by a tissue mask.
The extractor follows a **scikit-learn–style** lifecycle:
Or more concisely via method chaining:
**Configuration attributes** (set at init, do not change after fitting):
```python theme={null}
extractor = TileExtractor(tile_spec=spec, mask=mask)
extractor.fit(reader) # bind to a slide
tiles = extractor.extract() # compute tile positions
extractor.save("out/") # write tile images
extractor.to_csv("tiles.csv") # export metadata
```
```python theme={null}
tiles = TileExtractor(tile_spec=spec, mask=mask).fit_extract(reader)
```
**Attributes**:
* `tile_spec` *TileSpec* - Tile size, stride, resolution, and measurement unit.
* `mask` *np.ndarray | None* - Binary tissue mask (non-zero = tissue).
* `mask_provider` *TissueMaskProvider | None* - Provider that generates a per-slide mask during [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit). When set, takes precedence over the static *mask* array.
* `mask_threshold` *float | None* - Minimum mean mask value for a tile to be kept.
* `save_mask` *bool* - Whether to include the per-tile mask array in each [`RegionMaskSpec`](/sdk-reference/io/wsi/types#regionmaskspec).
**Fitted attributes** (populated by [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) / [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract)):
**Attributes**:
* `source_` *WSIReader* - The WSI reader bound by [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit).
* `slide_name_` *str* - Stem of the slide filename (e.g. `"slide_001"`).
* `region_specs_` *List\[RegionSpec]* - Tile regions produced by [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract). Empty until extraction is run.
***
#### fit
```python theme={null}
@log_failures
def fit(source: WSIReader) -> "TileExtractor"
```
Binds the extractor to an open WSI reader.
Stores the reader and slide name so that subsequent calls to [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract), [`save`](/sdk-reference/extraction/wsi/tile_extraction#save), [`to_csv`](/sdk-reference/extraction/wsi/tile_extraction#to_csv), and [`to_json`](/sdk-reference/extraction/wsi/tile_extraction#to_json) do not need the reader passed again.
If a `mask_provider` is set, its [`generate`](/sdk-reference/preprocess/wsi/provider/base#generate) method is called here so that each slide receives its own tissue mask automatically.
Following scikit-learn convention, fitted attributes are suffixed with an underscore (`source_`, `slide_name_`, `region_specs_`).
An **open** WSI reader for the target slide.
**Returns**:
* `TileExtractor` - `self`, to allow method chaining (e.g. `extractor.fit(reader).extract()`).
**Example**:
```python theme={null}
>>> extractor.fit(reader)
>>> extractor.slide_name_
'slide_001'
```
***
#### extract
```python theme={null}
@log_failures
def extract(source: Optional[WSIReader] = None) -> List[RegionSpec]
```
Extracts tile region specs from a WSI.
If *source* is provided it is used directly (and the extractor is automatically [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) to it). If omitted, the previously fitted source is used — call [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) first in that case.
The method operates in four stages:
1. **Reference-level setup** — identifies the lowest-resolution
pyramid level and computes the relative downsample between it and the target resolution.
2. **Mask scaling** — resizes the tissue mask (or creates an
all-tissue mask) to match the reference level's bounded dimensions.
3. **Tile-spec scaling** — converts tile size and stride from the
target resolution (pixels or microns) into reference-level pixel coordinates.
4. **Grid walk** — iterates over the reference-level grid, filters by
`mask_threshold`, and maps surviving tile coordinates back to the target resolution.
In coordinate mode (`coordinates` set), the grid walk and tissue masking are skipped: one [`RegionSpec`](/sdk-reference/io/wsi/types#regionspec) is emitted per supplied `(x, y)` position.
Results are stored in `region_specs_` **and** returned.
An open WSI reader. When given, the extractor is fitted to it automatically. When `None` (default), the extractor must already be fitted.
**Returns**:
* `List[RegionSpec]` - Region specifications for every tile that passes the tissue-mask threshold (grid mode) or one per supplied coordinate (coordinate mode).
**Raises**:
* `RuntimeError` - If no *source* is provided and [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) has not been called.
* `ValueError` - If `tile_spec.unit` is not `MeasurementUnit.PIXELS` or `MeasurementUnit.UM`.
* `BioptimusValueError` - In coordinate mode, if any coordinate is negative or would place a tile outside the slide bounds.
**Example**:
```python theme={null}
>>> extractor.fit(reader)
>>> tiles = extractor.extract()
>>> tiles[0].location
RegionLocation(top=1024, left=512)
```
***
#### fit\_extract
```python theme={null}
@log_failures
def fit_extract(source: WSIReader) -> List[RegionSpec]
```
Convenience method: [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit) + [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract) in one call.
An open WSI reader.
**Returns**:
* `List[RegionSpec]` - Extracted tile region specifications.
**Example**:
```python theme={null}
>>> tiles = extractor.fit_extract(reader)
>>> len(tiles)
42
```
***
#### to\_csv
```python theme={null}
@log_failures
def to_csv(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None) -> Path
```
Saves region metadata to a CSV file named after the slide.
The file is written as `/.csv`, reusing the `slide_name_` captured during [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit).
Uses the internally stored `region_specs_` by default. Pass *regions* explicitly to override.
Each row contains the **slide name**, location, shape, resolution, and tissue ratio. Mask arrays are **not** included — use [`save`](/sdk-reference/extraction/wsi/tile_extraction#save) to persist tile images instead.
Destination directory. Created (including parents) if it does not exist.
Region specifications to serialise. Defaults to `region_specs_`.
**Returns**:
* `Path` - The resolved path of the written CSV file (e.g. `output/slide_001.csv`).
**Raises**:
* `RuntimeError` - If the extractor has not been fitted or no regions are available.
**Example**:
```python theme={null}
>>> extractor.fit_extract(reader) # slide file is 'abc.svs'
>>> extractor.to_csv("output/")
PosixPath('output/abc.csv')
```
***
#### to\_json
```python theme={null}
@log_failures
def to_json(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None,
indent: int = 2) -> Path
```
Saves region metadata to a JSON file named after the slide.
The file is written as `/.json`, reusing the `slide_name_` captured during [`fit`](/sdk-reference/extraction/wsi/tile_extraction#fit).
Uses the internally stored `region_specs_` by default. Pass *regions* explicitly to override.
The output is a JSON array of objects, one per [`RegionSpec`](/sdk-reference/io/wsi/types#regionspec), containing the slide name, location, shape, resolution, and tissue ratio. Mask arrays are **not** included.
Destination directory. Created (including parents) if it does not exist.
Region specifications to serialise. Defaults to `region_specs_`.
Number of spaces for pretty-printing. Defaults to `2`. Set to `0` or `None` for compact output.
**Returns**:
* `Path` - The resolved path of the written JSON file (e.g. `output/slide_001.json`).
**Raises**:
* `RuntimeError` - If the extractor has not been fitted or no regions are available.
**Example**:
```python theme={null}
>>> extractor.fit_extract(reader) # slide file is 'abc.svs'
>>> extractor.to_json("output/")
PosixPath('output/abc.json')
```
***
#### save
```python theme={null}
@log_failures
def save(output_dir: Union[str, Path],
regions: Optional[List[RegionSpec]] = None,
workers: int = 4,
image_format: str = "png") -> None
```
Saves extracted tile images to disk using multi-threaded I/O.
Uses the fitted `source_` and internally stored `region_specs_` by default. Pass *regions* explicitly to override the region list.
Each tile is read from the WSI via [`WSIReader.read_region`](/sdk-reference/io/wsi/interface#read_region) and written to *output\_dir* with a metadata-rich filename:
Where `{res}` encodes the resolution (e.g. `l_0`, `mpp_0.25`, `mag_40.0x`) and `{ratio}` is the tissue fraction rounded to two decimal places.
**Note:** The tile coordinates produced by [`extract`](/sdk-reference/extraction/wsi/tile_extraction#extract) are **absolute** slide coordinates, so this method reads regions with `bounded=False` to avoid double-offsetting.
```text theme={null}
{slide_stem}_x{left}_y{top}_w{width}_h{height}_{res}_mask_r{ratio}.{fmt}
```
Directory where tile images will be written. Created (including parents) if it does not exist.
Region specifications to save. Defaults to `region_specs_`.
Maximum number of threads for parallel I/O. Defaults to `4`.
Image file extension / format accepted by `PIL.Image.save` (e.g. `"png"`, `"jpeg"`, `"tiff"`). Defaults to `"png"`.
**Returns**:
None
**Raises**:
* `RuntimeError` - If the extractor has not been fitted, no regions are available, or one or more tiles fail to save.
**Example**:
```python theme={null}
>>> extractor.fit_extract(reader)
>>> extractor.save("output/tiles")
```
# bioptimus.extraction.wsi.types
Source: https://docs.bioptimus.com/sdk-reference/extraction/wsi/types
Tile and mega-tile specification types for WSI extraction.
Defines the [`TileSpec`](/sdk-reference/data/tile_spec#tilespec) and [`MegaTileSpec`](/sdk-reference/extraction/wsi/types#megatilespec) dataclasses as well as the [`GridLayout`](/sdk-reference/extraction/wsi/types#gridlayout) enum and the [`MegaTileRegion`](/sdk-reference/extraction/wsi/types#megatileregion) container.
[`TileSpec`](/sdk-reference/data/tile_spec#tilespec) Encapsulates tile size, stride, target resolution, and measurement unit. Used by `TileExtractor`.
[`GridLayout`](/sdk-reference/extraction/wsi/types#gridlayout) Enum for rectangular vs. hexagonal tiling grids.
[`MegaTileSpec`](/sdk-reference/extraction/wsi/types#megatilespec) Describes a *mega tile* — a grid of smaller tiles — including grid dimensions, per-tile specification, layout, stride, and valid-tile-ratio bounds. Used by `MegaTileExtractor`.
[`MegaTileRegion`](/sdk-reference/extraction/wsi/types#megatileregion) Immutable container returned by the mega-tile extractor. Holds the top-left location, grid shape, per-tile [`RegionSpec`](/sdk-reference/io/wsi/types#regionspec) list, tile-availability mask, and aggregate tissue statistics.
**Example**:
```python theme={null}
>>> from bioptimus.extraction.wsi.types import TileSpec, MegaTileSpec, GridLayout
>>> from bioptimus.io.wsi.types import Level, MeasurementUnit
>>> tile = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> mega = MegaTileSpec(megatile_shape=(5, 5), tile_spec=tile,
... grid_layout=GridLayout.RECTANGULAR)
>>> mega.num_tiles
25
```
## GridLayout
```python theme={null}
class GridLayout(str, Enum)
```
Layout strategy for arranging tiles within a mega tile.
**Attributes**:
* `RECTANGULAR` - Standard row-major rectangular grid.
* `HEXAGONAL` - Offset hex grid — odd rows are shifted right by half a tile width, producing a honeycomb pattern.
**Example**:
```python theme={null}
>>> GridLayout.RECTANGULAR.value
'RECTANGULAR'
```
## MegaTileSpec
```python theme={null}
@dataclass(frozen=True)
class MegaTileSpec()
```
Defines the specification of a mega tile — a grid of smaller tiles.
A mega tile groups `rows × cols` individual tiles into a single logical region that can be extracted from a WSI. The per-tile geometry is delegated to [`TileSpec`](/sdk-reference/data/tile_spec#tilespec).
**Attributes**:
* `megatile_shape` *tuple\[int, int]* - Number of tiles as `(rows, cols)`.
* `tile_spec` *TileSpec* - Size, stride, resolution, and unit for each individual tile.
* `grid_layout` *GridLayout* - Rectangular or hexagonal arrangement. Defaults to `RECTANGULAR`.
* `megatile_stride` *tuple\[int, int] | None* - Stride expressed in **number of tiles** as `(stride_rows, stride_cols)`. `None` defaults to `megatile_shape` (no overlap).
* `min_valid_tiles_ratio` *float* - Minimum fraction of tiles that must pass the tissue-mask threshold for the mega tile to be kept. Defaults to `0.5`.
* `max_valid_tiles_ratio` *float | None* - Optional upper bound on the valid-tile fraction. `None` means no upper limit.
**Example**:
```python theme={null}
>>> tile = TileSpec(size=(256, 256), stride=(256, 256),
... resolution=Level(0), unit=MeasurementUnit.PIXELS)
>>> mega = MegaTileSpec(megatile_shape=(5, 5), tile_spec=tile)
>>> mega.rows, mega.cols
(5, 5)
>>> mega.num_tiles
25
```
***
#### megatile\_shape
(rows, cols) in number of tiles
***
#### megatile\_stride
(stride\_rows, stride\_cols)
***
#### rows
```python theme={null}
@property
def rows() -> int
```
Number of tile rows in the mega tile.
***
#### cols
```python theme={null}
@property
def cols() -> int
```
Number of tile columns in the mega tile.
***
#### num\_tiles
```python theme={null}
@property
def num_tiles() -> int
```
Total number of tiles in the mega tile grid.
***
#### effective\_stride
```python theme={null}
@property
def effective_stride() -> tuple[int, int]
```
Returns the stride, defaulting to `megatile_shape` when not set.
**Returns**:
tuple\[int, int]: `(stride_rows, stride_cols)` in tiles.
## MegaTileRegion
```python theme={null}
@dataclass(frozen=True)
class MegaTileRegion()
```
A group of tiles forming one extracted mega tile.
Returned by `MegaTileExtractor`.
**Attributes**:
* `location` *RegionLocation* - Absolute slide coordinates of the top-left corner of the mega tile.
* `megatile_spec` *MegaTileSpec* - The specification that produced this mega tile (grid shape, tile spec, layout, stride, and valid-ratio bounds).
* `tile_specs` *List\[RegionSpec]* - Per-tile region specifications, ordered row-major (`tile_specs[row * cols + col]`).
* `tile_availability` *np.ndarray* - Boolean array of shape `(rows, cols)` indicating which tiles passed the tissue threshold.
* `valid_ratio` *float* - Fraction of tiles that are valid (tissue).
**Example**:
```python theme={null}
>>> mega.grid_shape
(5, 5)
>>> mega.valid_ratio
0.72
>>> mega.tile_availability.sum()
18
```
***
#### grid\_shape
```python theme={null}
@property
def grid_shape() -> tuple[int, int]
```
`(rows, cols)` — number of tiles in each direction.
***
#### grid\_layout
```python theme={null}
@property
def grid_layout() -> GridLayout
```
Layout used during extraction.
***
#### rows
```python theme={null}
@property
def rows() -> int
```
Number of tile rows.
***
#### cols
```python theme={null}
@property
def cols() -> int
```
Number of tile columns.
***
#### num\_valid\_tiles
```python theme={null}
@property
def num_valid_tiles() -> int
```
Count of tiles that passed the tissue threshold.
***
#### num\_tiles
```python theme={null}
@property
def num_tiles() -> int
```
Total tile slots in the grid.
# bioptimus.inference
Source: https://docs.bioptimus.com/sdk-reference/inference
Inference orchestration — slide-level and multi-slide pipelines.
# bioptimus.inference.aggregate
Source: https://docs.bioptimus.com/sdk-reference/inference/aggregate
Slide-level aggregation utilities.
Small, dependency-light helpers that reduce a written tile-output file (the `(N_tiles, D)` `outputs` array produced by `bioptimus.inference.writers`) to a single slide-level vector.
Two convenience entry points build on the same mean-pool:
* [`slide_embedding`](/sdk-reference/inference/aggregate#slide_embedding) — averages tile embeddings into one slide embedding.
* [`pseudobulk`](/sdk-reference/inference/aggregate#pseudobulk) — averages predicted spatial gene expression into a single bulk RNA-seq-like profile, paired with the output gene names.
These operate on the *output of* `SlideInference.predict`/`.embed` (or the `Inference` facade), so no model or server is needed to aggregate.
## Pseudobulk
```python theme={null}
@dataclass(frozen=True, eq=False)
class Pseudobulk()
```
A predicted pseudobulk gene-expression profile for one slide.
Pairs the aggregated expression vector with the gene names it is indexed by, so callers can align it to a reference panel without a separate lookup.
`eq=False` keeps identity equality: a value-based `__eq__` would compare the numpy `expression` field with `==` and raise on truthiness.
**Attributes**:
* `expression` - The pseudobulk expression vector of shape `(G,)` as `float32`.
* `gene_names` - Ordered gene identifiers for each entry of `expression`, or `None` when the output has no gene names.
***
#### slide\_reduce
```python theme={null}
def slide_reduce(outputs: np.ndarray, pool: Pool = "mean") -> np.ndarray
```
Reduces per-tile outputs to one slide-level vector by the chosen pooling.
The `(N_tiles, D)` per-tile outputs.
The pooling strategy over the tile axis. One of `"mean"`, `"max"`, `"sum"`, or `"median"`. Defaults to `"mean"`.
**Returns**:
The slide-level vector of shape `(D,)` as `float32`.
**Raises**:
* `BioptimusValueError` - If `pool` is not a supported strategy, or `outputs` is not a 2-D `(N_tiles, D)` array or has no tiles.
***
#### slide\_mean
```python theme={null}
def slide_mean(outputs: np.ndarray) -> np.ndarray
```
Mean-pools per-tile outputs into one slide-level vector.
Thin alias for `slide_reduce(outputs, pool="mean")`, kept for convenience and backward compatibility.
The `(N_tiles, D)` per-tile outputs.
**Returns**:
The slide-level vector of shape `(D,)` as `float32`.
**Raises**:
* `BioptimusValueError` - If `outputs` is not a 2-D `(N_tiles, D)` array or has no tiles.
***
#### slide\_embedding
```python theme={null}
def slide_embedding(path: str | Path,
output_format: OutputFormat | None = None,
*,
pool: Pool = "mean",
save: bool = False) -> np.ndarray
```
Reduces a slide's tile embeddings to one slide-level embedding.
Reads the per-tile embeddings from a file written in embedding mode (the embedding is the `outputs` array) or in the combined `PREDICTION_WITH_EMBEDDING` mode (the embedding is the separate `embeddings` array, alongside the `outputs` prediction). The `embeddings` array is preferred when present.
Path to a tile-output file written in embedding or combined mode.
The on-disk format. Inferred from the extension when omitted.
The pooling strategy over tiles (`"mean"`, `"max"`, `"sum"`, or `"median"`). Defaults to `"mean"`.
When `True`, saves the vector next to the input as `.slide.npy`. Defaults to `False`.
**Returns**:
The slide embedding of shape `(D,)` as `float32`.
***
#### pseudobulk
```python theme={null}
def pseudobulk(path: str | Path,
output_format: OutputFormat | None = None,
*,
pool: Pool = "mean",
save: bool = False) -> Pseudobulk
```
Reduces a slide's predicted spatial expression to a pseudobulk profile.
Path to a tile-output file written in `predict` mode.
The on-disk format. Inferred from the extension when omitted.
The pooling strategy over tiles (`"mean"`, `"max"`, `"sum"`, or `"median"`). Defaults to `"mean"`.
When `True`, saves the vector next to the input as `.slide.npy`. Defaults to `False`.
**Returns**:
A [`Pseudobulk`](/sdk-reference/inference/aggregate#pseudobulk) pairing the `(G,)` expression vector with the
output gene names (`None` if the file has none).
# bioptimus.inference.inference
Source: https://docs.bioptimus.com/sdk-reference/inference/inference
Generic WSI inference orchestrator.
Provides [`Inference`](/sdk-reference/inference/inference#inference), a high-level facade that wires together model construction, tissue segmentation, tile extraction, and per-tile prediction via a remote model endpoint. The internal `_SlideInference` class handles single-WSI processing.
## Inference
```python theme={null}
class Inference(model_name: Models | str,
api_url: str | None = None,
*,
cohort: Cohort | None = None,
wsi_dir: str | Path | None = None,
bulk_rna_dir: str | Path | None = None,
mask_dir: str | Path | None = None,
tissue: bool = True,
mask_threshold: float = 0.5,
output_path: str | Path = _DEFAULT_OUTPUT_PATH,
output_format: OutputFormat | str = OutputFormat.ZARR,
max_concurrency: int = 256,
workers: int | None = None,
experiment: str | None = None,
run: int | None = None,
variant: str | None = None,
description: str | None = None,
backend: str = 'remote',
wsi_backend: str | None = None,
coordinates: Sequence[tuple[int, int]] | None = None,
**kwargs: Any)
```
High-level multi-WSI inference facade.
Wraps model construction, tissue segmentation, tile extraction, and per-tile prediction behind a simple API. A single instance is reusable across many WSIs.
The `Cohort` is the single source of truth — all inputs, outputs, and processing status live there. The manifest is auto-saved after each WSI completes, enabling seamless resume on restart.
Model identifier. Use [`Models`](/sdk-reference/models/types#models) enum values or a plain string matching a YAML config.
Base URL for the HTTP backend. Required when *backend* is `"remote"`.
A pre-built [`Cohort`](/sdk-reference/data/cohort#cohort) manifest. If not provided, one is created on first use (ad-hoc mode).
Directory containing WSI files.
Directory containing bulk RNA files.
Directory containing pre-computed tissue masks.
Whether to auto-compute tissue masks via the `tissue-seg` model. Defaults to `True`.
Minimum tissue fraction to keep a tile.
Root directory for the structured workspace.
On-disk format for predictions/embeddings.
Maximum concurrent HTTP requests.
Number of WSIs to process in parallel. When `None` (the default), it falls back to `max(1, gpu_count)` from the local CUDA device count. This is meaningful for the `local` backend; for the `remote` and `aws` backends the GPUs live server-side, so a warning is logged and this should be set explicitly.
Experiment / study name for directory nesting.
Run number for repeated experiments.
Sub-variant label (e.g. stain method, threshold).
Free-text note saved in the config.
`"remote"` (HTTP) or `"aws"` (SageMaker).
Optional explicit tile positions as `(x, y)` top-left, slide-origin pixel pairs at the model tile-spec resolution. When supplied, tiles are extracted at exactly these locations (one per coordinate) and tissue masking is bypassed, for every slide this instance processes. Intended for predicting at predefined locations (e.g. PseudoVisium / hex-pooled / Visium spots) and benchmarking against spatial-transcriptomics ground truth. Spot tables report spot *centres*; convert to top-left first (subtract half the spot size in x and y). Coordinate mode targets a single WSI at a time.
Backend-specific arguments forwarded to `Backbone`.
**Example**:
```python theme={null}
from bioptimus.models import Models
from bioptimus.inference import Inference
from bioptimus.data.cohort import Cohort
cohort = Cohort.from_directories("/data/wsis", "/data/rna")
infer = Inference(
model_name=Models.H1,
api_url="http://localhost:8000",
cohort=cohort,
experiment="brca-cohort-a",
run=1,
)
infer.run() # processes all pending WSIs
```
***
#### from\_workspace
```python theme={null}
@classmethod
@log_failures
def from_workspace(cls, workspace: str | Path, **overrides: Any) -> Inference
```
Resumes an [`Inference`](/sdk-reference/inference/inference#inference) from a saved workspace.
Loads both the config YAML and the cohort manifest, allowing seamless resume of an interrupted run.
Path to the workspace directory.
Keyword overrides applied on top of the config values.
**Returns**:
A fully-initialised [`Inference`](/sdk-reference/inference/inference#inference) with cohort state.
***
#### save\_config
```python theme={null}
@log_failures
def save_config(path: str | Path | None = None) -> Path
```
Persists the current configuration as a YAML file.
Destination file path. Defaults to the workspace.
**Returns**:
Path to the written config file.
***
#### cohort
```python theme={null}
@property
def cohort() -> Cohort
```
The cohort manifest (single source of truth).
***
#### wsi\_dir
```python theme={null}
@property
def wsi_dir() -> Path | None
```
Directory containing WSI files, or `None` if not set.
***
#### model\_name
```python theme={null}
@property
def model_name() -> str
```
Name of the model this instance targets.
***
#### workspace
```python theme={null}
@property
def workspace() -> Path
```
Resolved experiment workspace root directory.
***
#### manifest\_path
```python theme={null}
@property
def manifest_path() -> Path
```
Path to the cohort manifest YAML in the workspace.
***
#### find
```python theme={null}
@log_failures
def find(wsi_dir: str | Path | None = None,
bulk_rna_dir: str | Path | None = None,
mask_dir: str | Path | None = None,
*,
patient_id_fn: Callable[[str], str] | None = None) -> Cohort
```
Discovers WSI, bulk RNA, and mask files; builds the cohort.
Scans directories for supported files, auto-pairs them, and stores the result as the instance cohort.
Directory to scan for WSI files.
Directory for bulk RNA files.
Directory for pre-computed masks.
Optional callable that maps a filename stem to a patient ID.
**Returns**:
The constructed [`Cohort`](/sdk-reference/data/cohort#cohort).
**Raises**:
* `ValueError` - If no WSI directory is available.
***
#### run
```python theme={null}
def run(mode: InferenceMode | str = InferenceMode.EMBEDDING,
*,
force: bool = False) -> list[Path | None]
```
Processes all pending WSIs in the cohort.
Automatically resumes — WSIs whose output already exists on disk are skipped unless *force* is `True`.
The `InferenceMode` to run (`EMBEDDING` or `PREDICTION`); a mode string is also accepted.
Reprocess all WSIs regardless of status.
**Returns**:
List of output paths, one per WSI in cohort order.
**Raises**:
* `BioptimusValueError` - If this instance has `coordinates` set (the full-cohort form is incompatible with the single-WSI contract).
***
#### predict
```python theme={null}
@log_failures
def predict(wsi_path: str | Path | list[str | Path] | None = None,
*,
bulk_rna_path: str | Path | None = None,
force: bool = False) -> Path | None | list[Path | None]
```
Runs prediction for one or more WSIs.
When called without arguments, processes the full cohort. Single paths are auto-registered in the cohort.
When this instance was constructed with `coordinates=`, tiles are extracted at those fixed locations for the single processed WSI.
Single WSI path, list of paths, or `None` to process the full cohort.
Optional bulk RNA CSV override.
Reprocess even if already done.
**Returns**:
The output path for a single WSI (raising on failure), or a list of
output paths for a list input or the full cohort, with `None` in
the slots of slides that failed (logged, recorded in the manifest,
and skipped so the batch completes).
**Raises**:
* `BioptimusValueError` - If this instance has `coordinates` set and a list input or the full-cohort form is requested.
***
#### embed
```python theme={null}
@log_failures
def embed(wsi_path: str | Path | list[str | Path] | None = None,
*,
bulk_rna_path: str | Path | None = None,
force: bool = False) -> Path | None | list[Path | None]
```
Runs embedding extraction for one or more WSIs.
When called without arguments, processes the full cohort. Single paths are auto-registered in the cohort.
Single WSI path, list of paths, or `None` to process the full cohort.
Optional bulk RNA CSV override.
Reprocess even if already done.
**Returns**:
The output path for a single WSI (raising on failure), or a list of
output paths for a list input or the full cohort, with `None` in
the slots of slides that failed (logged, recorded in the manifest,
and skipped so the batch completes).
**Raises**:
* `BioptimusValueError` - If this instance has `coordinates` set and a list input or the full-cohort form is requested.
***
#### tissue
```python theme={null}
@log_failures
def tissue(wsi_path: str | Path | list[str | Path] | None = None,
*,
force: bool = False) -> np.ndarray | None | list[np.ndarray | None]
```
Computes (or loads cached) tissue masks.
When called without arguments, processes all WSIs in the cohort that don't yet have masks.
Single path, list, or `None` for full cohort.
Recompute even if a cached mask exists.
**Returns**:
The mask for a single path (raising on failure), or a list of masks
for the full cohort or a list input, with `None` in the slots of
slides whose mask computation failed (logged and skipped).
***
#### status
```python theme={null}
def status() -> str
```
Returns and prints a processing status summary.
**Returns**:
A human-readable summary of cohort processing progress.
***
#### failures
```python theme={null}
@property
def failures() -> list[tuple[str, str, str]]
```
Returns WSIs with errors as (wsi\_id, model, error).
***
#### report
```python theme={null}
def report() -> str
```
Prints and returns a failure summary.
**Returns**:
A formatted string listing all failed WSIs, or a success message.
# bioptimus.inference.schemas
Source: https://docs.bioptimus.com/sdk-reference/inference/schemas
Request and response schemas for model inference.
These Pydantic models define the JSON payloads exchanged between clients and a deployed model endpoint (e.g. SageMaker or a local server). Each request/response pair corresponds to a single tile.
## InferenceMode
```python theme={null}
class InferenceMode(enum.StrEnum)
```
Inference modes shared across the server, clients, and pipeline.
The values are the on-wire strings (the SageMaker `/invocations` `mode` field and the REST endpoint keys), so this enum is the single source of truth for mode identifiers without changing the wire contract. Because it is a `StrEnum`, members compare, hash, and serialize exactly like their string value, so they are drop-in wherever a mode string was used.
***
#### coerce\_inference\_mode
```python theme={null}
def coerce_inference_mode(value: InferenceMode | str) -> InferenceMode
```
Normalizes a mode or stage argument to an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode).
Accepts an [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) member unchanged, a canonical on-wire string (for example `"prediction"`), or a legacy short spelling (for example `"predict"`). A legacy spelling is translated to its canonical member and emits a `DeprecationWarning`, so the enum is the single source of truth while pre-enum callers keep working.
The mode or stage to normalize.
**Returns**:
The corresponding [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode) member.
**Raises**:
* `BioptimusValueError` - If `value` is neither a member, a canonical value, nor a known legacy spelling.
## ModelRequest
```python theme={null}
class ModelRequest(BaseModel)
```
Payload sent to the model endpoint for a single tile.
The tile image is base64-encoded for JSON transport. Bulk RNA counts can optionally be included for multimodal models.
**Attributes**:
* `image_data` - Base64-encoded tile image (PNG).
* `bulk_rna` - Optional bulk RNA counts vector.
* `slide_name` - Stem of the source slide filename.
* `x` - Tile x-coordinate in the slide.
* `y` - Tile y-coordinate in the slide.
* `width` - Tile width in pixels.
* `height` - Tile height in pixels.
* `tissue_ratio` - Fraction of tissue in the tile.
* `patch_idx` - Index of this tile in the extraction plan.
***
#### image
```python theme={null}
@property
def image() -> Image.Image
```
Decodes and returns the tile image as PIL.
## ModelResponse
```python theme={null}
class ModelResponse(BaseModel)
```
Payload returned by the model endpoint for a single tile.
Carries the model output together with the full tile metadata from the originating request, so that downstream writers can persist everything without needing a separate schema.
**Attributes**:
* `output` - Model output vector for this tile.
* `slide_name` - Stem of the source slide filename.
* `x` - Tile x-coordinate in the slide.
* `y` - Tile y-coordinate in the slide.
* `width` - Tile width in pixels.
* `height` - Tile height in pixels.
* `tissue_ratio` - Fraction of tissue in the tile.
* `patch_idx` - Index of this tile in the extraction plan.
* `resolution` - Extraction resolution (MPP) if available.
* `embedding` - The tile embedding, returned alongside `output` only for the `prediction_with_embedding` mode (both come from the same forward pass). `None` for the single-output modes.
## SageMakerRequest
```python theme={null}
class SageMakerRequest(ModelRequest)
```
Extended request for the SageMaker `/invocations` endpoint.
Adds a `model_name` field so the server can dispatch to the correct model and a `mode` field to select the scheduler (see [`InferenceMode`](/sdk-reference/inference/schemas#inferencemode)).
# bioptimus.inference.writers
Source: https://docs.bioptimus.com/sdk-reference/inference/writers
Streaming writers for persisting tile-level predictions.
Provides a [`PredictionWriter`](/sdk-reference/inference/writers#predictionwriter) protocol and three concrete implementations ([`ZarrWriter`](/sdk-reference/inference/writers#zarrwriter), [`HDF5Writer`](/sdk-reference/inference/writers#hdf5writer), [`NPZWriter`](/sdk-reference/inference/writers#npzwriter)) that save per-tile outputs and metadata to disk in a memory-efficient way.
Zarr and HDF5 writers stream results row-by-row so that the full output tensor never needs to reside in memory. The NPZ writer accumulates results in Python lists and flushes on close.
## OutputFormat
```python theme={null}
class OutputFormat(str, enum.Enum)
```
Supported on-disk formats for prediction output.
## PredictionWriter
```python theme={null}
class PredictionWriter(path: Path)
```
Abstract base for streaming prediction writers.
***
#### open
```python theme={null}
@abstractmethod
def open(num_tiles: int, embedding_dim: int) -> None
```
Pre-allocates storage once the embedding dim is known.
Total number of tiles to be written.
Dimensionality of each prediction vector.
***
#### write
```python theme={null}
@abstractmethod
def write(response: ModelResponse) -> None
```
Writes a single tile's model response.
The model response to persist.
***
#### set\_thumbnail
```python theme={null}
@abstractmethod
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail image.
RGB image array of shape `(H, W, 3)`.
***
#### set\_metadata
```python theme={null}
@abstractmethod
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores slide-level metadata alongside the arrays.
Key-value pairs (slide\_name, tile\_size, etc.).
***
#### set\_tissue\_mask
```python theme={null}
@abstractmethod
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask used during extraction.
Binary mask array of shape `(H, W)`, dtype `uint8`.
***
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores input and output gene name lists.
Default implementation is a no-op. Subclasses that support gene annotation should override this.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
***
#### close
```python theme={null}
@abstractmethod
def close() -> None
```
Flushes and finalises the output file.
***
#### abort
```python theme={null}
def abort() -> None
```
Releases resources and removes any partial output on failure.
Called when inference fails before completion so that no misleading, unfinalized output file is left on disk. Safe to call whether or not [`open`](/sdk-reference/inference/writers#open) succeeded, and independent of how many tiles were written.
## ZarrWriter
```python theme={null}
class ZarrWriter(PredictionWriter)
```
Streams tile predictions into a Zarr directory store.
***
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Pre-allocates Zarr datasets for outputs, coordinates, and tissue ratios.
Total number of tiles to be written.
Dimensionality of each prediction vector.
***
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the Zarr datasets.
The model response to persist.
***
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail as a Zarr dataset.
RGB image array of shape `(H, W, 3)`.
***
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata as Zarr root attributes.
Key-value pairs (slide\_name, tile\_size, etc.).
***
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask as a Zarr dataset.
Binary mask array of shape `(H, W)`, dtype `uint8`.
***
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name arrays as Zarr string datasets.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
***
#### close
```python theme={null}
def close() -> None
```
Logs output path (Zarr stores are flushed on write).
## HDF5Writer
```python theme={null}
class HDF5Writer(PredictionWriter)
```
Streams tile predictions into an HDF5 file.
***
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Opens the HDF5 file and pre-allocates datasets.
Total number of tiles to be written.
Dimensionality of each prediction vector.
***
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the HDF5 datasets.
The model response to persist.
***
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores a slide thumbnail as an HDF5 dataset.
RGB image array of shape `(H, W, 3)`.
***
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata as HDF5 file-level attributes.
Key-value pairs (slide\_name, tile\_size, etc.).
***
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask as an HDF5 dataset.
Binary mask array of shape `(H, W)`, dtype `uint8`.
***
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name arrays as HDF5 string datasets.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
***
#### close
```python theme={null}
def close() -> None
```
Closes the HDF5 file handle and flushes to disk.
***
#### abort
```python theme={null}
def abort() -> None
```
Closes the open HDF5 handle before removing the partial file.
## NPZWriter
```python theme={null}
class NPZWriter(PredictionWriter)
```
Accumulates tile predictions in memory, saves as compressed npz.
***
#### open
```python theme={null}
def open(num_tiles: int, embedding_dim: int) -> None
```
Allocates in-memory arrays for accumulating results.
Total number of tiles to be written.
Dimensionality of each prediction vector.
***
#### write
```python theme={null}
def write(response: ModelResponse) -> None
```
Writes a single tile response into the in-memory arrays.
The model response to persist.
***
#### set\_thumbnail
```python theme={null}
def set_thumbnail(thumbnail: NDArray) -> None
```
Stores the slide thumbnail for later NPZ serialization.
RGB image array of shape `(H, W, 3)`.
***
#### set\_metadata
```python theme={null}
def set_metadata(metadata: dict[str, Any]) -> None
```
Stores metadata for later NPZ serialization.
Key-value pairs (slide\_name, tile\_size, etc.).
***
#### set\_tissue\_mask
```python theme={null}
def set_tissue_mask(mask: NDArray) -> None
```
Stores the tissue mask for later NPZ serialization.
Binary mask array of shape `(H, W)`, dtype `uint8`.
***
#### set\_gene\_names
```python theme={null}
def set_gene_names(input_genes: list[str] | None = None,
output_genes: list[str] | None = None) -> None
```
Stores gene name lists for later NPZ serialization.
Ordered Ensembl IDs for bulk RNA input.
Ordered Ensembl IDs for predicted output.
***
#### close
```python theme={null}
def close() -> None
```
Flushes all accumulated arrays to a compressed `.npz` file.
***
#### create\_writer
```python theme={null}
def create_writer(fmt: OutputFormat, path: Path) -> PredictionWriter
```
Creates a writer for the requested format.
Output format.
Destination file or directory path.
**Returns**:
An initialised (but not yet opened) [`PredictionWriter`](/sdk-reference/inference/writers#predictionwriter).
# bioptimus.io.omics
Source: https://docs.bioptimus.com/sdk-reference/io/omics
Omics I/O utilities.
# bioptimus.io.wsi
Source: https://docs.bioptimus.com/sdk-reference/io/wsi
WSI (Whole Slide Image) module for reading digital pathology slides.
This module provides a unified interface for reading WSI files across different backends (OpenSlide, cuCIM, TiffFile) with automatic backend selection.
Backend implementations live in the private `._backends` package and should not be imported directly. Use [`WSI`](/sdk-reference/io/wsi/factory#wsi) to obtain a reader.
# bioptimus.io.wsi.factory
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/factory
Factory for WSI reader instantiation with automatic backend selection.
Provides the [`WSI`](/sdk-reference/io/wsi/factory#wsi) convenience class that selects the best available backend (cuCIM, OpenSlide, TiffSlide) based on the file extension and installed libraries.
***
#### get\_extension
```python theme={null}
def get_extension(path: str) -> str
```
Extracts and normalizes the file extension from a path.
File path or filename string.
**Returns**:
Lowercase file extension including the leading dot.
## WSI
```python theme={null}
class WSI(path: str, backend: Optional[Union[Backend, str]] = Backend.AUTO)
```
Factory class for creating WSI readers with automatic backend selection.
Usage:
```python theme={null}
>>> reader = WSI("slide.svs", backend=WSI.Backend.AUTO)
>>> reader = WSI("slide.svs", backend=WSI.Backend.OPENSLIDE)
```
## Backend
```python theme={null}
class Backend(str, Enum)
```
Supported backend identifiers and the AUTO selection flag.
***
#### list
```python theme={null}
@classmethod
def list(cls) -> List[str]
```
Returns the list of valid backend string values (excluding AUTO).
***
#### supported\_extensions
```python theme={null}
@classmethod
def supported_extensions(cls) -> List[str]
```
Returns the list of all supported file extensions.
**Returns**:
List of extension strings (e.g. `[".svs", ".tiff", ...]`).
***
#### available\_backends
```python theme={null}
@classmethod
def available_backends(cls) -> Dict[str, bool]
```
Returns the availability status of all physical backends.
**Returns**:
Mapping of backend name to availability boolean.
# bioptimus.io.wsi.interface
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/interface
WSI Reader Base Module.
This module provides the `WSIReader` abstract base class, which standardizes access to Whole Slide Images (WSI). It handles coordinate translations, resolution scaling (Level, Magnification, MPP), and tissue-aware cropping.
Typical usage example:
with OpenSlideReader("path/to/slide.svs") as reader:
# Get tissue-only dimensions
dims = reader.dimensions(bounded=True)
# Read a region relative to the slide origin
region = reader.read\_region((0, 0), (512, 512), Level(0), bounded=False)
## WSIReader
```python theme={null}
class WSIReader(path: str)
```
Abstract Base Class for all WSI Backends.
Standardizes the interface for reading digital pathology slides across different vendors. It manages metadata, pyramid levels, and tissue bounding box transformations.
**Attributes**:
* `path` - Path object pointing to the slide file.
* `props` - Metadata properties extracted from the slide.
***
#### open\_slide
```python theme={null}
@abstractmethod
def open_slide() -> None
```
Opens the file handle for the slide.
**Raises**:
* `NotImplementedError` - If not implemented by subclass.
***
#### level\_count
```python theme={null}
@property
@abstractmethod
def level_count() -> int
```
Number of pyramid levels available in the slide.
**Returns**:
* `int` - Number of pyramid levels.
**Raises**:
* `NotImplementedError` - If not implemented by subclass.
***
#### mpp
```python theme={null}
@property
def mpp() -> Union[MPP, None]
```
Native microns per pixel (MPP) value.
**Returns**:
* `Optional[MPP]` - The MPP value if found in metadata, else None.
**Example**:
```python theme={null}
>>> reader.mpp
0.25
```
***
#### downsample\_dimensions
```python theme={null}
def downsample_dimensions(downsample_factor: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Calculates slide dimensions at a specific downsample factor.
The ratio to scale down by.
Whether to return tissue bounds or full slide size.
**Returns**:
Union\[WSIDims, WSIBounds]: Scaled slide dimensions or tissue bounds.
**Example**:
```python theme={null}
>>> reader.downsample_dimensions(4.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.downsample_dimensions(4.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
***
#### level\_dimensions
```python theme={null}
def level_dimensions(level: Level,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns the dimensions of a specific pyramid level.
The target pyramid level (0 is highest resolution).
If True, returns tissue area at that level.
**Returns**:
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified level.
**Raises**:
* `ValueError` - If level is out of range.
**Example**:
```python theme={null}
>>> reader.level_dimensions(Level(2), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.level_dimensions(Level(2), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
***
#### magnification\_dimensions
```python theme={null}
def magnification_dimensions(
magnification: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific optical magnification.
Target magnification (e.g., 40x, 20x, 10x).
If True, returns tissue area at that magnification.
**Returns**:
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified magnification.
**Example**:
```python theme={null}
>>> reader.magnification_dimensions(10.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.magnification_dimensions(10.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
***
#### mpp\_dimensions
```python theme={null}
def mpp_dimensions(mpp: float,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific MPP (microns per pixel).
Target microns per pixel.
If True, returns tissue area at that MPP.
**Returns**:
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified MPP.
**Example**:
```python theme={null}
>>> reader.mpp_dimensions(1.0, bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.mpp_dimensions(1.0, bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
***
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: bool = True,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns full or bounded slide dimensions at Level 0.
Acts as the `bounded` toggle when a bool.
If True, returns tissue bounds.
***
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: Level,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a pyramid level.
Target pyramid level.
If True, returns tissue bounds.
***
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: Magnification,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a magnification.
Target optical magnification.
If True, returns tissue bounds.
***
#### dimensions
```python theme={null}
@overload
def dimensions(resolution: MPP,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions at a target MPP.
Target microns per pixel.
If True, returns tissue bounds.
***
#### dimensions
```python theme={null}
def dimensions(resolution: Union[Resolution, bool] = True,
bounded: bool = True) -> Union[WSIDims, WSIBounds]
```
Returns slide dimensions for a specific resolution.
Desired resolution (Level, Magnification, or MPP).
If True, returns tissue area at that resolution.
**Returns**:
Union\[WSIDims, WSIBounds]: Dimensions or bounds at specified resolution.
**Example**:
```python theme={null}
>>> reader.dimensions(Resolution(level=2), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(level=2), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
>>> reader.dimensions(Resolution(magnification=10.0), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(magnification=10.0), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
>>> reader.dimensions(Resolution(mpp=1.0), bounded=False)
WSIDims(width=25000, height=20000)
>>> reader.dimensions(Resolution(mpp=1.0), bounded=True)
WSIBounds(x=1250, y=500, width=25000, height=20000)
```
***
#### get\_downsample\_factor
```python theme={null}
def get_downsample_factor(resolution: Resolution) -> float
```
Calculates the downsample factor for a given resolution.
The target resolution (Level, Magnification, or MPP).
**Returns**:
* `float` - Downsample factor for the given resolution.
**Example**:
```python theme={null}
>>> reader.get_downsample_factor(Resolution(level=2))
4.0
>>> reader.get_downsample_factor(Resolution(magnification=10.0))
4.0
>>> reader.get_downsample_factor(Resolution(mpp=1.0))
4.0
```
***
#### level\_downsample
```python theme={null}
@abstractmethod
def level_downsample(level: int) -> float
```
Returns downsample factor for a pyramid level.
Pyramid level index.
**Returns**:
* `float` - Downsample factor for the given level.
**Raises**:
* `NotImplementedError` - If not implemented by subclass.
***
#### mpp\_downsample
```python theme={null}
def mpp_downsample(mpp: float) -> float
```
Calculates downsample needed for target MPP.
Target microns per pixel.
**Returns**:
* `float` - Downsample factor for the target MPP.
**Raises**:
* `ValueError` - If metadata is missing or target is too high-res.
**Example**:
```python theme={null}
>>> reader.mpp_downsample(1.0)
4.0
```
***
#### magnification\_downsample
```python theme={null}
def magnification_downsample(magnification: float) -> float
```
Calculates downsample needed for target magnification.
Target magnification power.
**Returns**:
* `float` - Downsample factor for the target magnification.
**Raises**:
* `ValueError` - If metadata is missing or target is out of range.
**Example**:
```python theme={null}
>>> reader.magnification_downsample(10.0)
4.0
```
***
#### props
```python theme={null}
@property
def props() -> WSIProps
```
Slide metadata properties.
**Returns**:
* `WSIProps` - Metadata extracted from the slide.
**Example**:
```python theme={null}
>>> reader.props.OBJECTIVE_POWER
40
```
***
#### get\_best\_level\_for\_downsample
```python theme={null}
def get_best_level_for_downsample(downsample: float) -> int
```
Finds highest pyramid level smaller than target downsample factor.
Target scaling factor.
**Returns**:
* `int` - Best pyramid level index for the downsample.
**Raises**:
* `ValueError` - If downsample is too small.
**Example**:
```python theme={null}
>>> reader.get_best_level_for_downsample(4.0)
2
```
***
#### get\_downsample\_scaling\_and\_level
```python theme={null}
def get_downsample_scaling_and_level(
resolution: Resolution) -> tuple[float, float, int]
```
Determines the downsample factor, scaling factor, and best pyramid level for a given resolution.
Desired resolution of the slide. Can be one of:
* Level: integer pyramid level
* Magnification: desired optical magnification
* MPP: microns per pixel
**Returns**:
* `tuple` - (downsample\_factor (float), scaling\_factor (float), best\_level (int))
* downsample\_factor: Factor by which the base level is downsampled.
* scaling\_factor: Additional scaling needed at the chosen level.
* best\_level: Pyramid level that best matches the resolution.
**Raises**:
* `ValueError` - If required slide properties (objective power or MPP) are unavailable.
* `TypeError` - If the resolution type is unsupported.
**Example**:
```python theme={null}
>>> reader.get_downsample_scaling_and_level(Level(2))
(4.0, 1.0, 2)
```
***
#### read\_region
```python theme={null}
def read_region(location: tuple[int, int],
size: tuple[int, int],
resolution: Resolution,
measurement_unit: MeasurementUnit = MeasurementUnit.PIXELS,
bounded: bool = True) -> Region
```
Reads a region from the slide.
(x, y) at Level 0.
(width, height) at target resolution.
Resolution level/mpp/magnification.
Unit for the region shape
pixels).
If True, (0,0) is tissue top-left. Else slide top-left.
**Returns**:
* `Region` - Object containing the image and metadata for the requested area.
**Example**:
```python theme={null}
>>> region = reader.read_region((0, 0), (512, 512), Level(0), bounded=False)
>>> region = reader.read_region((0, 0), (512, 512), Level(0), bounded=True)
```
***
#### is\_supported\_file
```python theme={null}
@abstractmethod
def is_supported_file() -> bool
```
Checks if backend supports this file extension.
**Returns**:
* `bool` - True if supported, False otherwise.
**Raises**:
* `NotImplementedError` - If not implemented by subclass.
***
#### close
```python theme={null}
@abstractmethod
def close() -> None
```
Closes slide file handle.
**Returns**:
None
**Raises**:
* `NotImplementedError` - If not implemented by subclass.
***
#### \_\_enter\_\_
```python theme={null}
def __enter__() -> "WSIReader"
```
Context manager entry.
**Returns**:
* `WSIReader` - The reader instance with slide file open.
**Example**:
```python theme={null}
>>> with SomeWSIReader("slide.svs") as reader:
... dims = reader.dimensions()
```
***
#### \_\_exit\_\_
```python theme={null}
def __exit__(exc_type: Any, exc_value: Any, traceback: Any) -> None
```
Context manager exit.
Exception type.
Exception value.
Exception traceback.
**Returns**:
None
**Example**:
```python theme={null}
>>> with SomeWSIReader("slide.svs") as reader:
... pass
```
***
#### get\_thumbnail
```python theme={null}
def get_thumbnail(size: tuple[int, int], bounded: bool = True) -> Image.Image
```
Generates a slide thumbnail.
Max (width, height) for result.
If True, crop to tissue. Else show full slide (incl. white space).
**Returns**:
* `Image.Image` - RGB thumbnail image.
**Example**:
```python theme={null}
>>> reader.get_thumbnail((1024, 1024), bounded=False)
>>> reader.get_thumbnail((1024, 1024), bounded=True)
```
# bioptimus.io.wsi.metadata
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/metadata
WSI Metadata Properties Module.
This module defines the schema and container for metadata extracted from Whole Slide Images (WSI). It provides a unified structure for handling physical, optical, and structural properties across different scanner vendors and file formats.
## WSIProps
```python theme={null}
@dataclass(frozen=True)
class WSIProps()
```
Whole Slide Image (WSI) properties container.
This class stores essential metadata required for WSI processing pipelines. It supports attribute access and dictionary-style key-based lookup with an automated fallback to raw vendor metadata.
**Attributes**:
* `DIMENSIONS` \_WSIDims \_ - Full slide width and height.
* `BOUNDS` \_WSIBounds \_ - The tissue-containing region of the slide.
* `LEVELS` *int | None* - The number of resolution levels in the pyramid.
* `MPP_X` *MPP | None* - Microns per pixel in the X dimension.
* `MPP_Y` *MPP | None* - Microns per pixel in the Y dimension.
* `OBJECTIVE_POWER` *Magnification | None* - The magnification power of the objective lens.
* `VENDOR` *str | None* - The name of the scanner vendor or backend.
* `FILE_PATH` *Path* - The filesystem path to the slide file.
* `RAW` *dict\[str, Any] | None* - A dictionary containing all unprocessed metadata from the slide.
**Example**:
```python theme={null}
>>> dims = WSIDims(50000, 30000)
>>> props = WSIProps(DIMENSIONS=dims, VENDOR="Hamamatsu")
>>> print(props.DIMENSIONS.width)
50000
>>> print(props["VENDOR"])
'Hamamatsu'
```
***
#### \_\_getitem\_\_
```python theme={null}
def __getitem__(key: str) -> Any
```
Allows dictionary-style access to properties.
Tries to retrieve the value from the class attributes first. If not found, it attempts to look up the key in the 'RAW' metadata dictionary.
The name of the property (e.g., 'VENDOR') or raw metadata key (e.g., 'tiff.ImageDescription') to retrieve.
**Returns**:
* `Any` - The value associated with the key.
**Raises**:
* `KeyError` - If the key is not a valid field of the class and is not present in the raw metadata dictionary.
# bioptimus.io.wsi.types
Source: https://docs.bioptimus.com/sdk-reference/io/wsi/types
Core types for representing resolutions and regions in Whole Slide Images (WSIs).
This module defines lightweight, immutable classes for commonly used WSI attributes such as pyramid levels, physical pixel size (MPP), and objective magnification.
Classes:
Level: Pyramid level index (0 = highest resolution).
MPP: Microns per pixel (µm/pixel).
Magnification: Objective power (e.g., 20x).
WSIDims: Width and height dimensions.
WSIBounds: A bounding box for tissue regions.
MeasurementUnit: Enum for pixels or microns.
RegionLocation: X/Y coordinates of a region.
RegionShape: Dimensions and units of a region.
RegionSpec: Full specification (Location + Shape + Resolution).
Region: Container for PIL image data and its RegionSpec.
## Level
```python theme={null}
class Level(index: int)
```
Represents a pyramid level index in a WSI.
Level 0 is the highest resolution. Higher indices are lower resolutions.
The pyramid level index (0-based).
**Raises**:
* `ValueError` - If `index` is negative.
**Example**:
```python theme={null}
>>> level = Level(0) # highest resolution
>>> int(level)
0
>>> level
Level 0
>>> Level(-1)
Traceback (most recent call last):
...
ValueError: Level index must be >= 0, got -1.
```
**Notes**:
* Level 0 is the highest resolution.
* Higher indices correspond to lower resolution levels in the pyramid.
* This class stores the level index as a primitive integer and is immutable.
## MPP
```python theme={null}
class MPP(value: float)
```
Physical size of a pixel in microns per pixel (µm/pixel) in a whole slide image (WSI).
MPP values indicate the real-world size of one image pixel and must be a finite number strictly greater than 0 (NaN and infinity are rejected).
The pixel size in microns to represent as MPP.
**Raises**:
* `ValueError` - If `value` is not a finite positive number.
**Example**:
```python theme={null}
>>> mpp = MPP(0.25) # 0.25 µm/pixel
>>> float(mpp)
0.25
>>> MPP(-1)
Traceback (most recent call last):
...
ValueError: MPP must be a finite positive number, got -1.
```
## Magnification
```python theme={null}
class Magnification(value: float)
```
Objective power of a microscope used to acquire a whole slide image (WSI).
Magnification indicates the target resolution (e.g., 20x, 40x). It is always stored as a finite positive float (NaN and infinity are rejected).
Typical magnifications:
* 10x: low-resolution overview
* 20x: standard diagnostic resolution
* 40x: high-resolution for detailed analysis
The objective power.
**Raises**:
* `ValueError` - If `value` is not a finite positive number.
**Example**:
```python theme={null}
>>> mag = Magnification(40) # 40x objective
>>> float(mag)
40.0
>>> mag
40.00x
>>> Magnification(20.0)
20.00x
>>> Magnification(-10)
Traceback (most recent call last):
...
ValueError: Magnification must be a finite positive number, got -10.0.
```
## WSIDims
```python theme={null}
class WSIDims(NamedTuple)
```
Dimensions of a slide or level at full resolution.
**Attributes**:
* `width` *int* - The width of the image in pixels.
* `height` *int* - The height of the image in pixels.
**Example**:
```python theme={null}
>>> dims = WSIDims(width=10000, height=20000)
>>> print(dims.width)
10000
```
## WSIBounds
```python theme={null}
class WSIBounds(NamedTuple)
```
The non-empty region (Bounding Box) of the slide in pixels.
This represents the "tissue-containing" area of a slide, allowing users to ignore large background areas.
**Attributes**:
* `x` *int* - The X coordinate of the top-left corner.
* `y` *int* - The Y coordinate of the top-left corner.
* `width` *int* - The width of the bounding box.
* `height` *int* - The height of the bounding box.
**Example**:
```python theme={null}
>>> bounds = WSIBounds(x=100, y=100, width=500, height=500)
>>> print(bounds.dims)
WSIDims(width=500, height=500)
```
***
#### dims
```python theme={null}
@property
def dims() -> WSIDims
```
Extracts width and height as a WSIDims object.
**Returns**:
* `WSIDims` - The dimensions of the bounding box.
**Example**:
```python theme={null}
>>> bounds = WSIBounds(x=10, y=20, width=100, height=200)
>>> bounds.dims
WSIDims(width=100, height=200)
```
## MeasurementUnit
```python theme={null}
class MeasurementUnit(str, Enum)
```
Enumeration of measurement units for WSI tiles and regions.
***
#### has\_value
```python theme={null}
@classmethod
def has_value(cls, value: str) -> bool
```
Checks if a string is a valid MeasurementUnit.
The string to check.
**Returns**:
* `bool` - True if valid, False otherwise.
**Example**:
```python theme={null}
>>> MeasurementUnit.has_value("PIXELS")
True
>>> MeasurementUnit.has_value("INCHES")
False
```
## RegionLocation
```python theme={null}
@dataclass(frozen=True)
class RegionLocation()
```
Defines the location of a region within a WSI.
**Attributes**:
* `top` *int* - The top coordinate of the top-left corner of the region.
* `left` *int* - The left coordinate of the top-left corner of the region.
* `width` *int* - The width of the region in pixels.
* `height` *int* - The height of the region in pixels.
**Example**:
```python theme={null}
>>> location = RegionLocation(top=100, left=200)
>>> print(location.top)
100
```
***
#### x
```python theme={null}
@property
def x() -> int
```
Alias for left coordinate.
***
#### y
```python theme={null}
@property
def y() -> int
```
Alias for top coordinate.
## RegionShape
```python theme={null}
@dataclass(frozen=True)
class RegionShape()
```
Defines the shape of a region within a WSI.
**Attributes**:
* `width` *int* - The width of the region in pixels.
* `height` *int* - The height of the region in pixels.
* `unit` *MeasurementUnit* - The unit of measurement for the shape dimensions (e.g., PIXELS, UM).
**Example**:
```python theme={null}
>>> shape = RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS)
>>> print(shape.unit)
PIXELS
>>> print(shape.width)
500
```
***
#### shape
```python theme={null}
@property
def shape() -> tuple[int, int]
```
Returns the width and height as a tuple.
**Returns**:
tuple\[int, int]: A tuple containing (width, height).
***
#### to\_um
```python theme={null}
def to_um(mpp: MPP) -> "RegionShape"
```
Converts pixels to microns.
The pixel count to convert.
The microns-per-pixel scale.
**Returns**:
* `float` - The length in microns.
**Example**:
```python theme={null}
>>> unit = MeasurementUnit.PIXELS
>>> unit.pixels_to_um(100, MPP(0.5))
50.0
```
***
#### to\_pixels
```python theme={null}
def to_pixels(mpp: MPP) -> "RegionShape"
```
Converts microns to pixels.
The length in microns to convert.
The microns-per-pixel scale.
**Returns**:
* `float` - The length in pixels.
**Example**:
```python theme={null}
>>> unit = MeasurementUnit.UM
>>> unit.um_to_pixels(50, MPP(0.5))
100.0
```
## RegionMaskSpec
```python theme={null}
@dataclass(frozen=True)
class RegionMaskSpec()
```
Defines the mask applied/corresponding to the region e.g., during extraction can be binary or multiclass mask.
**Attributes**:
* `ratio` *float* - Pixels belonging to the mask relative to the total region area.
* `mask` *np.ndarray | None* - The mask applied to the region, or `None` if mask storage was not requested.
## RegionSpec
```python theme={null}
@dataclass(frozen=True)
class RegionSpec()
```
Defines the specifications of a region within a WSI.
**Attributes**:
* `location` *RegionLocation* - The location of the region within the slide.
* `shape` *RegionShape* - The shape of the region.
* `resolution` *Resolution* - The resolution at which the region is defined (Level, MPP, or Magnification).
**Example**:
```python theme={null}
>>> location = RegionLocation(top=100, left=200)
>>> spec = RegionSpec(location=location, shape=RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS), resolution=Level(0))
>>> print(spec.location.left)
200
```
## Region
```python theme={null}
@dataclass(frozen=True)
class Region()
```
Defines a region of interest within a WSI, including its image data and specifications. This class encapsulates both the pixel data of a region and its metadata, such as location, shape, and resolution.
**Attributes**:
* `image` *PIL.Image.Image* - The pixel data of the region as a PIL Image.
* `spec` *RegionSpec* - The specifications of the region, including location, shape, resolution, and mask.
**Example**:
```python theme={null}
>>> from PIL import Image
>>> img = Image.new('RGB', (500, 500)) # create a blank image for demonstration
>>> location = RegionLocation(top=100, left=200)
>>> shape = RegionShape(width=500, height=500, unit=MeasurementUnit.PIXELS)
>>> mask_spec = RegionMaskSpec(mask=np.zeros((500, 500)), ratio=0.0)
>>> spec = RegionSpec(location=location, shape=shape, resolution=Level(0), mask_spec=mask_spec)
>>> region = Region(image=img, spec=spec)
>>> print(region.spec.location.left)
200
```
***
#### location
```python theme={null}
@property
def location() -> RegionLocation
```
Convenience property to access the region's location directly.
***
#### shape
```python theme={null}
@property
def shape() -> tuple[int, int]
```
Convenience property to access the region's shape directly.
***
#### resolution
```python theme={null}
@property
def resolution() -> Resolution
```
Convenience property to access the region's resolution directly.
***
#### measurement\_unit
```python theme={null}
@property
def measurement_unit() -> MeasurementUnit
```
Convenience property to access the region's measurement unit directly.
***
#### mask
```python theme={null}
@property
def mask() -> Union[np.ndarray, None]
```
Convenience property to access the region's mask directly.
**Returns**:
np.ndarray | None: The mask array, or `None` if mask storage
was not requested during extraction.
***
#### mask\_ratio
```python theme={null}
@property
def mask_ratio() -> float
```
Convenience property to access the region's mask ratio directly.
# bioptimus.models
Source: https://docs.bioptimus.com/sdk-reference/models
Model abstractions — backbones, types, and endpoint clients.
# bioptimus.models.backbones
Source: https://docs.bioptimus.com/sdk-reference/models/backbones
Backbone feature-extractor models (H-Optimus-1, M-Optimus, Tissue-Seg, …).
Model specifications are loaded from YAML config files in the `configs/` directory. The [`Backbone`](/sdk-reference/models/backbones#backbone) factory wires them together with a pluggable client so that a single `Backbone(...)` call works for any backend (HTTP, AWS, GCP, …).
**Example**:
```python theme={null}
backbone = Backbone("h1", base_url="http://localhost:8080")
backbone = Backbone("h1", backend="aws", endpoint_name="prod")
```
## Backbone
```python theme={null}
class Backbone(model: str = 'h1',
backend: Backend | str = Backend.REMOTE,
**kwargs: Any)
```
Factory for creating backbone endpoint models.
Model identity (spec, endpoint path) is read from YAML config files. The factory pairs each config with a pluggable client backend to produce a ready-to-use [`EndpointModel`](/sdk-reference/models/endpoint_model#endpointmodel).
Usage:
```python theme={null}
>>> Backbone("h1", base_url="http://localhost:8080")
>>> Backbone("h1", backend="aws", endpoint_name="ep")
>>> Backbone("h1", backend="huggingface")
>>> Backbone.available_backbones()
['h0', 'h0-mini', 'h1', 'm-optimus', 'tissue-seg']
>>> Backbone.available_backbones(backend="huggingface")
['h0', 'h0-mini', 'h1']
```
## Backend
```python theme={null}
class Backend(str, Enum)
```
Supported inference backends.
***
#### available\_backbones
```python theme={null}
@classmethod
def available_backbones(cls,
backend: Backend | str | None = None) -> List[str]
```
Returns registered backbone names, optionally filtered by backend.
If given, restrict the result to models the backend can serve. Only the `"huggingface"` backend is restrictive — it serves the embedding backbones that declare a `huggingface` build recipe (`h0-mini`, `h0`, `h1`). The `"remote"`, `"aws"` and `"local"` backends can serve any registered model, so they return the full list.
**Returns**:
Sorted list of backbone names.
**Raises**:
* `KeyError` - If `backend` is not a recognised backend.
# bioptimus.models.clients
Source: https://docs.bioptimus.com/sdk-reference/models/clients
Client protocol for model endpoint communication.
Defines the [`Client`](/sdk-reference/models/clients#client) protocol that all client backends (HTTP, AWS SageMaker, GCP Vertex AI, Azure ML, …) must satisfy. Each client encapsulates connection details and knows how to send a JSON request body and return the JSON response body.
## Client
```python theme={null}
@runtime_checkable
class Client(Protocol)
```
Protocol for model endpoint communication.
Implementations are fully configured at construction time (URL, endpoint name, credentials, etc.) so that `predict` / `embed` only need the serialized body.
***
#### predict
```python theme={null}
def predict(body: str) -> str
```
Sends a prediction request synchronously.
***
#### embed
```python theme={null}
def embed(body: str) -> str
```
Sends an embedding request synchronously.
***
#### predict\_with\_embedding
```python theme={null}
def predict_with_embedding(body: str) -> str
```
Send a combined prediction+embedding request synchronously.
Returns both the prediction and the embedding from a single forward pass. Backends that cannot produce both (e.g. embedding-only models) may raise `NotImplementedError`.
***
#### metadata
```python theme={null}
def metadata() -> str
```
Fetches model metadata synchronously.
***
#### predict\_async
```python theme={null}
async def predict_async(body: str, session: Any = None) -> str
```
Sends a prediction request asynchronously.
***
#### embed\_async
```python theme={null}
async def embed_async(body: str, session: Any = None) -> str
```
Sends an embedding request asynchronously.
***
#### predict\_with\_embedding\_async
```python theme={null}
async def predict_with_embedding_async(body: str, session: Any = None) -> str
```
Send a combined prediction+embedding request asynchronously.
# bioptimus.models.clients.aws
Source: https://docs.bioptimus.com/sdk-reference/models/clients/aws
AWS SageMaker client.
Sends JSON payloads to a SageMaker endpoint via the `sagemaker-runtime` `invoke_endpoint` API. The `model_name` field is injected into every request body for multi-model dispatch.
## AWSClient
```python theme={null}
class AWSClient(endpoint_name: str,
model_name: str,
region_name: str | None = None,
boto_session: boto3.Session | None = None,
timeout: float = 60.0)
```
Sends JSON payloads to an AWS SageMaker endpoint.
Name of the SageMaker endpoint.
Model identifier injected into the request body for SageMaker multi-model dispatch.
AWS region. When *None*, uses the default from the environment or *boto\_session*.
Optional pre-configured `boto3.Session` (e.g. with explicit credentials or a custom profile). When *None*, a default session is created using the standard credential chain.
Read timeout in seconds.
**Example**:
```python theme={null}
client = AWSClient(
endpoint_name="h1-prod",
model_name="h1",
)
resp_json = client.predict(request_json)
```
***
#### predict
```python theme={null}
@log_failures
def predict(body: str) -> str
```
Invokes the SageMaker endpoint for prediction.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### embed
```python theme={null}
@log_failures
def embed(body: str) -> str
```
Invokes the SageMaker endpoint for embedding.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### predict\_with\_embedding
```python theme={null}
@log_failures
def predict_with_embedding(body: str) -> str
```
Invokes the SageMaker endpoint for the combined prediction+embedding mode.
Returns both the gene prediction and the tile embedding from a single forward pass (the embedding is a free byproduct of the prediction backbone).
Serialized JSON request payload.
**Returns**:
Response body as a JSON string, with `embedding` populated
alongside `output`.
***
#### metadata
```python theme={null}
@log_failures
def metadata() -> str
```
Fetches model metadata via `/invocations`.
Sends a minimal request with `mode` set to `"metadata"` so the server returns model metadata instead of running inference.
**Returns**:
Response body as a JSON string.
***
#### predict\_async
```python theme={null}
@log_failures
async def predict_async(body: str, session: Any = None) -> str
```
Predicts asynchronously via `run_in_executor`.
`boto3` is not natively async, so the synchronous `predict` call is delegated to a thread pool.
Serialized JSON payload.
Unused. Accepted for interface compatibility with the `Client` protocol.
**Returns**:
Response body as a string.
***
#### embed\_async
```python theme={null}
@log_failures
async def embed_async(body: str, session: Any = None) -> str
```
Embeds asynchronously via `run_in_executor`.
Serialized JSON payload.
Unused. Accepted for interface compatibility.
**Returns**:
Response body as a string.
***
#### predict\_with\_embedding\_async
```python theme={null}
@log_failures
async def predict_with_embedding_async(body: str, session: Any = None) -> str
```
Combined prediction+embedding asynchronously via `run_in_executor`.
Serialized JSON payload.
Unused. Accepted for interface compatibility.
**Returns**:
Response body as a string.
# bioptimus.models.clients.dispatch
Source: https://docs.bioptimus.com/sdk-reference/models/clients/dispatch
Shared helpers for concurrent per-tile dispatch failure handling.
These utilities are transport-agnostic and shared by every code path that fans a batch of tile requests out concurrently (the whole-slide inference pipeline and the standalone PyTorch adapter). They centralize how a batch summarizes and surfaces the tiles that failed after their per-request retry budget was exhausted, so the failure policy stays consistent and is defined in exactly one place.
***
#### summarize\_indices
```python theme={null}
def summarize_indices(indices: list[int],
limit: int = MAX_LOGGED_INDICES) -> str
```
Formats a possibly-long list of tile indices for error messages.
The tile indices to format.
Maximum number of indices to list before truncating.
**Returns**:
A comma-separated string, truncated with a count if too long.
***
#### dispatch\_failure\_error
```python theme={null}
def dispatch_failure_error(failures: list[tuple[int, BaseException]],
total_tiles: int,
*,
action: str = "Dispatch",
detail: str | None = None) -> BioptimusRuntimeError
```
Builds a typed error summarizing per-tile dispatch failures.
Constructs (but does not raise) a `BioptimusRuntimeError` tagged with `DISPATCH_FAILED`, chaining the lowest-indexed failure as the cause. Callers raise the returned error with `raise dispatch_failure_error(...) from ...` or simply `raise`; the cause is already attached.
`(tile_index, exception)` pairs for each failed tile.
Total number of tiles in the batch.
Verb describing the operation for the message, for example `"Dispatch"` or `"Prediction"`.
Optional extra clause inserted before the first error, for example `"No output written."`.
**Returns**:
The constructed error, with the first failure set as its cause.
# bioptimus.models.clients.hf
Source: https://docs.bioptimus.com/sdk-reference/models/clients/hf
Hugging Face client for in-process `timm` models.
Loads an embedding backbone (`h0-mini`, `h0`, `h1`) from the Hugging Face Hub via `timm` and exposes the same JSON-string transport contract as the HTTP, AWS, and local clients — so it plugs into `EndpointModel` unchanged.
No running server, Docker image, or AWS account is required — only network access to the Hub and acceptance of each model's gated license. The build recipe (Hub repository and `timm` keyword arguments) lives in the SDK model configs under the `huggingface` block (see `bioptimus.models.config_loader`); only the plain embedding backbones declare such a block.
Pooling is derived from the model output itself: backbones that already return a pooled `(N, embedding_dim)` feature (`h0`, `h1`) are used as-is, while token-sequence outputs (`h0-mini`) are pooled according to the config `output_type` (e.g. `cls+patch_mean`).
***
#### hf\_supported\_models
```python theme={null}
def hf_supported_models() -> list[str]
```
Returns the sorted model names that can be built from the Hub.
A model is supported when its SDK config declares a `huggingface` block.
**Returns**:
Sorted list of supported model names.
## HuggingFaceClient
```python theme={null}
class HuggingFaceClient(config: ModelConfig,
*,
device: str = 'cuda',
dtype: torch.dtype = torch.float16,
model: torch.nn.Module | None = None)
```
Runs a Hugging Face Hub embedding backbone in-process.
The build recipe (Hub repository and `timm` keyword arguments) is taken wholesale from the model's SDK config, so the config stays the single source of truth and inconsistent spec/recipe combinations cannot be constructed. Prefer [`build_hf_client`](/sdk-reference/models/clients/hf#build_hf_client) (or `Backbone(..., backend="huggingface")`) as the supported construction path.
The model's SDK config. Must declare a `huggingface` build recipe (`config.huggingface`).
Torch device string.
Inference dtype. On CUDA, reduced-precision dtypes (`float16`/`bfloat16`) enable autocast; `float32` runs in full precision.
Optional preloaded model, primarily for tests.
**Raises**:
* `ValueError` - If `config` has no `huggingface` build recipe.
***
#### predict
```python theme={null}
def predict(body: str) -> str
```
Runs prediction synchronously.
Embedding-only Hugging Face backbones share the `predict` and `embed` output contract so generic callers remain compatible.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
**Returns**:
Serialized [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) JSON string.
***
#### predict\_with\_embedding
```python theme={null}
def predict_with_embedding(body: str) -> str
```
Not supported for embedding-only Hugging Face backbones.
The combined mode returns a spatial-transcriptomics prediction alongside the embedding, but Hugging Face backbones are pure embedders with no prediction head, so there is no second output to pair.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
**Returns**:
Never returns; this method always raises.
**Raises**:
* `NotImplementedError` - Always, since these backbones produce only an embedding.
***
#### embed
```python theme={null}
def embed(body: str) -> str
```
Runs embedding synchronously.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
**Returns**:
Serialized [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) JSON string.
***
#### metadata
```python theme={null}
def metadata() -> str
```
Returns model metadata as a JSON string.
**Returns**:
JSON string with the model name, version, weights source, output
configuration, and the *effective* runtime precision (derived
from the dtype the client runs with, not the config default).
***
#### predict\_async
```python theme={null}
async def predict_async(body: str, session: Any = None) -> str
```
Runs prediction asynchronously via the event loop executor.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
Unused; accepted for Client-protocol parity.
**Returns**:
Serialized [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) JSON string.
***
#### embed\_async
```python theme={null}
async def embed_async(body: str, session: Any = None) -> str
```
Runs embedding asynchronously via the event loop executor.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
Unused; accepted for Client-protocol parity.
**Returns**:
Serialized [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) JSON string.
***
#### predict\_with\_embedding\_async
```python theme={null}
async def predict_with_embedding_async(body: str, session: Any = None) -> str
```
Not supported for embedding-only Hugging Face backbones.
Serialized [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) JSON string.
Unused; accepted for Client-protocol parity.
**Returns**:
Never returns; this method always raises.
**Raises**:
* `NotImplementedError` - Always; see [`predict_with_embedding`](/sdk-reference/models/clients/hf#predict_with_embedding).
***
#### build\_hf\_client
```python theme={null}
def build_hf_client(model_name: str,
device: str = "cuda",
*,
precision: str = "fp32",
hf_token: str | None = None) -> HuggingFaceClient
```
Builds an in-process client backed by a Hugging Face backbone.
Reads the model's SDK config (spec + `huggingface` build recipe) and constructs a ready-to-use [`HuggingFaceClient`](/sdk-reference/models/clients/hf#huggingfaceclient).
Registry key (`"h0-mini"`, `"h0"` or `"h1"`).
Torch device string (`"cuda"`, `"cuda:0"`, `"cpu"`).
Inference precision — `"fp32"` (default) or `"fp16"`.
Optional Hugging Face access token for gated models. When omitted, a cached login or the `HF_TOKEN` environment variable is used.
**Returns**:
A ready-to-use [`HuggingFaceClient`](/sdk-reference/models/clients/hf#huggingfaceclient).
**Raises**:
* `KeyError` - If the model has no SDK config.
* `ValueError` - If the model is not a supported embedding backbone or if `precision` is invalid.
# bioptimus.models.clients.http
Source: https://docs.bioptimus.com/sdk-reference/models/clients/http
HTTP client for direct API endpoints.
Sends JSON payloads to a model server via plain HTTP POST using `requests` (synchronous) and `aiohttp` (asynchronous).
***
#### is\_transient
```python theme={null}
def is_transient(exc: BaseException) -> bool
```
Returns whether an exception is a transient failure worth retrying.
Transient failures include connection resets, timeouts, and HTTP responses carrying a retryable server-side status code (e.g. 503 from momentary GPU back-pressure). Deterministic failures such as HTTP 400 (malformed request) are not retried.
When the exception carries an HTTP status, that status is authoritative: a retryable code is transient and any other code is not. The status is read from `.status` (`aiohttp.ClientResponseError`) or, failing that, from `.response.status_code` (`requests.HTTPError`). Only when no status is available does the exception type decide, so a `requests` error — whose base class is `OSError` — is not blindly retried.
The exception raised while dispatching a request.
**Returns**:
`True` if the failure is transient and should be retried.
***
#### tile\_retry\_logger
```python theme={null}
def tile_retry_logger(
logger: logging.Logger,
tile_index: int,
*,
max_retries: int = DEFAULT_MAX_RETRIES
) -> Callable[[int, BaseException, float], None]
```
Builds an `on_retry` callback that logs a per-tile dispatch retry.
Returns the callback expected by [`retry_sync`](/sdk-reference/models/clients/http#retry_sync) / [`retry_async`](/sdk-reference/models/clients/http#retry_async), emitting a structured `tile_dispatch_retry` warning. Shared by every per-tile dispatch path so the retry log event and fields stay identical.
The logger to emit on.
Index of the tile being retried, recorded in the event.
Retry budget, used to report the total attempt count.
**Returns**:
A callback taking `(attempt, exc, delay)` that logs the retry.
***
#### retry\_sync
```python theme={null}
def retry_sync(
attempt: Callable[[], T],
*,
max_retries: int = DEFAULT_MAX_RETRIES,
base_delay: float = DEFAULT_RETRY_BASE_DELAY,
is_retryable: Callable[[BaseException], bool] = is_transient,
sleep: Callable[[float], None] = time.sleep,
on_retry: Callable[[int, BaseException, float], None] | None = None
) -> T
```
Runs *attempt*, retrying transient failures with exponential backoff.
Synchronous twin of [`retry_async`](/sdk-reference/models/clients/http#retry_async) for threaded (non-async) callers, such as the `TorchModel` thread-pool dispatch. Shares the same backoff policy, defaults, and `on_retry` contract so both paths behave identically.
Zero-argument callable that performs one attempt.
Number of retries after the initial attempt (so the total number of attempts is `max_retries + 1`).
Seed for the exponential backoff (`base_delay * 2**n`).
Predicate deciding whether a raised exception is transient and worth retrying.
Blocking sleep function; injectable so tests can skip real waits.
Optional callback invoked before each retry with the zero-based attempt number, the exception, and the delay.
**Returns**:
The result of the first successful *attempt*.
**Raises**:
* `BaseException` - Re-raises the last exception when the retry budget is exhausted or the exception is not retryable.
***
#### retry\_async
```python theme={null}
async def retry_async(
attempt: Callable[[], Awaitable[T]],
*,
max_retries: int = DEFAULT_MAX_RETRIES,
base_delay: float = DEFAULT_RETRY_BASE_DELAY,
is_retryable: Callable[[BaseException], bool] = is_transient,
sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
on_retry: Callable[[int, BaseException, float], None] | None = None
) -> T
```
Runs *attempt*, retrying transient failures with exponential backoff.
The retry mechanism is transport-agnostic: the work to perform, the logging, and (optionally) the sleep function are all supplied by the caller, keeping this function reusable and easy to test.
Zero-argument coroutine factory that performs one attempt.
Number of retries after the initial attempt (so the total number of attempts is `max_retries + 1`).
Seed for the exponential backoff (`base_delay * 2**n`).
Predicate deciding whether a raised exception is transient and worth retrying.
Async sleep function; injectable so tests can skip real waits.
Optional callback invoked before each retry with the zero-based attempt number, the exception, and the delay.
**Returns**:
The result of the first successful *attempt*.
**Raises**:
* `BaseException` - Re-raises the last exception when the retry budget is exhausted or the exception is not retryable.
## HTTPClient
```python theme={null}
class HTTPClient(base_url: str, endpoints: Dict[str, str], timeout: float = 30.0)
```
Sends JSON payloads over HTTP.
Fully configured at construction — `predict` / `embed` only need the serialized body.
Root URL of the model server, e.g. `"http://localhost:8080"`.
Mapping of endpoint type to path. At least one key must be present; see the example below.
HTTP request timeout in seconds.
**Example**:
```python theme={null}
client = HTTPClient(
"http://localhost:8080",
endpoints={
"prediction": "/api/predict/m-optimus",
"embedding": "/api/embed/m-optimus",
},
)
pred_json = client.predict(request_json)
emb_json = client.embed(request_json)
```
***
#### close
```python theme={null}
def close() -> None
```
Closes the underlying HTTP session and releases connections.
***
#### predict
```python theme={null}
@log_failures
def predict(body: str) -> str
```
POST to the prediction endpoint.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### embed
```python theme={null}
@log_failures
def embed(body: str) -> str
```
POST to the embedding endpoint.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### predict\_with\_embedding
```python theme={null}
@log_failures
def predict_with_embedding(body: str) -> str
```
POST to the combined prediction+embedding endpoint.
Returns both the gene prediction and the tile embedding from a single forward pass. Only models that expose both prediction and embedding configure this endpoint; others raise `NotImplementedError`.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string, with `embedding` populated
alongside `output`.
***
#### metadata
```python theme={null}
@log_failures
def metadata() -> str
```
GET the metadata endpoint.
**Returns**:
Response body as a JSON string.
***
#### predict\_async
```python theme={null}
@log_failures
async def predict_async(body: str, session: Any = None) -> str
```
POST to the prediction endpoint asynchronously.
Serialized JSON request payload.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Response body as a JSON string.
***
#### embed\_async
```python theme={null}
@log_failures
async def embed_async(body: str, session: Any = None) -> str
```
POST to the embedding endpoint asynchronously.
Serialized JSON request payload.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Response body as a JSON string.
***
#### predict\_with\_embedding\_async
```python theme={null}
@log_failures
async def predict_with_embedding_async(body: str, session: Any = None) -> str
```
POST to the combined prediction+embedding endpoint asynchronously.
Serialized JSON request payload.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Response body as a JSON string, with `embedding` populated
alongside `output`.
# bioptimus.models.clients.local
Source: https://docs.bioptimus.com/sdk-reference/models/clients/local
In-process model inference client.
Satisfies the `Client` protocol by running model inference directly in the current process, without requiring the separate FastAPI microservice. This enables local GPU inference for notebooks, CI/CD pipelines, and researcher workflows with zero infrastructure overhead.
## Serialization contract
The [`Client`](/sdk-reference/models/clients#client) protocol exchanges JSON strings (`str`). The `InProcessClient` deserializes each request body into a `ModelRequest`, runs the server-model pipeline (preprocess → GPU forward → postprocess), and returns a `ModelResponse` serialized to JSON.
This double-serialization preserves drop-in compatibility with the remote backends. The overhead is negligible compared to eliminating network I/O.
## Batching
Two modes are supported:
* **Dynamic batching** (`dynamic_batching=True`, default): An internal `BatchScheduler` accumulates concurrent `predict_async` / `embed_async` calls into GPU batches, exactly matching the behavior of the FastAPI server. One scheduler is created per asyncio event loop (to support multi-WSI processing across threads).
* **Explicit batching** (`dynamic_batching=False`): Callers use `predict_batch` / `embed_batch` to submit pre-formed batches. Async calls run at batch-size-1 (no accumulation).
## InProcessClient
```python theme={null}
class InProcessClient(model: BaseServerModel,
max_batch_size: int = 48,
max_wait_ms: float = 30.0,
dynamic_batching: bool = True)
```
Runs model inference in-process using a loaded .pt2 checkpoint.
Satisfies the `Client` protocol.
A loaded server-model instance (or subclass).
Maximum tiles per GPU forward pass for dynamic batching.
Maximum milliseconds to wait for additional requests before dispatching a partial batch.
When *True* (default), async calls are accumulated into GPU batches via an internal scheduler. When *False*, each async call runs immediately at batch size 1.
**Example**:
```python theme={null}
from bioptimus.runtime import build_server_model
from bioptimus.models.clients.local import InProcessClient
model = build_server_model("h1", "/path/to/h1.pt2", device="cuda")
client = InProcessClient(model)
response_json = client.embed(request_json)
```
***
#### predict
```python theme={null}
@log_failures
def predict(body: str) -> str
```
Runs a prediction request synchronously.
Deserializes the JSON body, runs the server-model pipeline (prepare\_sample → forward\_batch), and returns the response as a JSON string.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### embed
```python theme={null}
@log_failures
def embed(body: str) -> str
```
Runs an embedding request synchronously.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string.
***
#### predict\_with\_embedding
```python theme={null}
@log_failures
def predict_with_embedding(body: str) -> str
```
Runs a combined prediction+embedding request synchronously.
Returns both the gene prediction and the tile embedding from a single forward pass (the embedding is a free byproduct of the prediction backbone), so callers needing both avoid a second inference.
Serialized JSON request payload.
**Returns**:
Response body as a JSON string, with `embedding` populated
alongside `output`.
***
#### metadata
```python theme={null}
@log_failures
def metadata() -> str
```
Returns model metadata as a JSON string.
**Returns**:
JSON-serialized model metadata dictionary.
***
#### predict\_async
```python theme={null}
@log_failures
async def predict_async(body: str, session: Any = None) -> str
```
Runs a prediction request asynchronously.
When dynamic batching is enabled, the request enters a per-loop batch scheduler. Otherwise it runs immediately.
Serialized JSON request payload.
Unused. Accepted for interface compatibility.
**Returns**:
Response body as a JSON string.
***
#### embed\_async
```python theme={null}
@log_failures
async def embed_async(body: str, session: Any = None) -> str
```
Runs an embedding request asynchronously.
Serialized JSON request payload.
Unused. Accepted for interface compatibility.
**Returns**:
Response body as a JSON string.
***
#### predict\_with\_embedding\_async
```python theme={null}
@log_failures
async def predict_with_embedding_async(body: str, session: Any = None) -> str
```
Runs a combined prediction+embedding request asynchronously.
When dynamic batching is enabled, the request enters the combined-mode per-loop batch scheduler. Otherwise it runs immediately.
Serialized JSON request payload.
Unused. Accepted for interface compatibility.
**Returns**:
Response body as a JSON string, with `embedding` populated
alongside `output`.
***
#### predict\_batch
```python theme={null}
@log_failures
def predict_batch(bodies: list[str]) -> list[str]
```
Runs a batch of prediction requests synchronously.
Bypasses dynamic batching — the caller controls the batch composition directly.
List of serialized JSON request payloads.
**Returns**:
List of response JSON strings, positionally aligned.
***
#### embed\_batch
```python theme={null}
@log_failures
def embed_batch(bodies: list[str]) -> list[str]
```
Runs a batch of embedding requests synchronously.
List of serialized JSON request payloads.
**Returns**:
List of response JSON strings, positionally aligned.
***
#### predict\_with\_embedding\_batch
```python theme={null}
@log_failures
def predict_with_embedding_batch(bodies: list[str]) -> list[str]
```
Runs a batch of combined prediction+embedding requests synchronously.
Bypasses dynamic batching — the caller controls the batch composition directly.
List of serialized JSON request payloads.
**Returns**:
List of response JSON strings, positionally aligned, each with
`embedding` populated alongside `output`.
***
#### from\_checkpoint
```python theme={null}
@classmethod
@log_failures
def from_checkpoint(cls,
model_name: str,
checkpoint: str | Path,
device: str = "cuda",
*,
assets_root: str | Path | None = None,
max_batch_size: int = 48,
max_wait_ms: float = 30.0,
dynamic_batching: bool = True) -> InProcessClient
```
Creates an InProcessClient from a checkpoint path.
Convenience factory that combines [`build_server_model`](/sdk-reference/runtime/loader#build_server_model) with client construction.
Model registry key (e.g. `"h1"`).
Path to the `.pt2` file.
Torch device string.
Assets directory for models that need it.
Maximum batch size for dynamic batching.
Maximum wait time for batch accumulation.
Whether to enable dynamic batching.
**Returns**:
A ready-to-use [`InProcessClient`](/sdk-reference/models/clients/local#inprocessclient).
# bioptimus.models.config_loader
Source: https://docs.bioptimus.com/sdk-reference/models/config_loader
Model configuration loader.
Reads model definition YAML files from the `configs/` directory and builds `ModelSpec` instances. This is the single source of truth for all model specifications.
## EndpointPaths
```python theme={null}
@dataclass(frozen=True)
class EndpointPaths()
```
Parsed endpoint routes for a model.
Each field corresponds to an optional endpoint type declared in the model's YAML config under `endpoints:`.
## HuggingFaceConfig
```python theme={null}
@dataclass(frozen=True)
class HuggingFaceConfig()
```
Hugging Face Hub build recipe for an embedding backbone.
Declares how to build the model directly from the Hub via `timm` for the in-process `huggingface` backend. Present only on configs of models that can be served this way.
**Attributes**:
* `repo_id` - Hugging Face Hub repository ID (without the `hf-hub:` prefix), e.g. `"bioptimus/H-optimus-0"`.
* `timm_kwargs` - Raw keyword arguments forwarded to `timm.create_model`. String values are treated as dotted import paths (e.g. `"timm.layers.SwiGLUPacked"`) and resolved to the referenced object at build time via [`resolve_dotted_path`](/sdk-reference/models/config_loader#resolve_dotted_path); other values are passed through unchanged.
***
#### resolve\_dotted\_path
```python theme={null}
def resolve_dotted_path(path: str) -> Any
```
Resolves a dotted import path to the object it references.
Imports the containing module and returns the named attribute, e.g. `"timm.layers.SwiGLUPacked"` imports `timm.layers` and returns `SwiGLUPacked`. Kept lazy (imports happen only when called) so that backends other than `huggingface` never import `timm`.
A dotted path of the form `"module.submodule.attribute"`.
**Returns**:
The object referenced by `path`.
**Raises**:
* `ValueError` - If `path` does not contain a module component.
* `ImportError` - If the module cannot be imported.
* `AttributeError` - If the attribute does not exist on the module.
## ModelConfig
```python theme={null}
@dataclass(frozen=True)
class ModelConfig()
```
A fully parsed model configuration.
Combines the [`ModelSpec`](/sdk-reference/models/types#modelspec) with routing metadata that is not part of the spec itself.
***
#### endpoint\_path
```python theme={null}
@property
def endpoint_path() -> str
```
Primary endpoint path (backward compatibility).
***
#### genes\_endpoint\_path
```python theme={null}
@property
def genes_endpoint_path() -> str | None
```
Genes/metadata endpoint path (backward compatibility).
***
#### load\_model\_config
```python theme={null}
def load_model_config(path: Path) -> ModelConfig
```
Loads a single model config YAML file.
Path to the YAML file.
**Returns**:
A [`ModelConfig`](/sdk-reference/models/config_loader#modelconfig) with the parsed spec and routing.
***
#### load\_all\_model\_configs
```python theme={null}
def load_all_model_configs(
configs_dir: Path | None = None) -> Dict[str, ModelConfig]
```
Discovers and loads all model configs from a directory.
Scans for `*.yaml` files and returns a dict keyed by `model_name`.
Directory to scan. Defaults to the package-bundled `configs/` directory.
**Returns**:
Mapping of model name to [`ModelConfig`](/sdk-reference/models/config_loader#modelconfig).
# bioptimus.models.endpoint_model
Source: https://docs.bioptimus.com/sdk-reference/models/endpoint_model
Client-agnostic endpoint model.
Provides a single [`EndpointModel`](/sdk-reference/models/endpoint_model#endpointmodel) that pairs a `ModelSpec` with any `Client` implementation (HTTP, AWS SageMaker, GCP, Azure, …). This eliminates the need for per-backend model subclasses.
## EndpointModel
```python theme={null}
class EndpointModel(model_spec: ModelSpec,
transport: Client,
input_gene_names: list[str] | None = None,
output_gene_names: list[str] | None = None)
```
Model endpoint backed by a pluggable client.
Combines a [`ModelSpec`](/sdk-reference/models/types#modelspec) (what the model expects and produces) with a [`Client`](/sdk-reference/models/clients#client) (how to reach it).
Supports both synchronous (`predict` / `embed`) and asynchronous (`predict_async` / `embed_async`) dispatch.
Model specification describing input requirements and output shape.
A configured [`Client`](/sdk-reference/models/clients#client) instance (e.g. [`HTTPClient`](/sdk-reference/models/clients/http#httpclient), [`AWSClient`](/sdk-reference/models/clients/aws#awsclient)).
Optional ordered Ensembl IDs expected as bulk RNA input (M-Optimus ).
Optional ordered Ensembl IDs of predicted output genes (M-Optimus ).
**Example**:
```python theme={null}
from bioptimus.models.clients.http import HTTPClient
client = HTTPClient(
"http://localhost:8080",
endpoints={"embedding": "/api/embed/h1"},
)
model = EndpointModel(H1_SPEC, client)
response = model(request)
```
***
#### model\_spec
```python theme={null}
@property
def model_spec() -> ModelSpec
```
Model specification including tile and output config.
***
#### input\_gene\_names
```python theme={null}
@property
def input_gene_names() -> list[str] | None
```
Ordered Ensembl IDs expected as bulk RNA input.
***
#### output\_gene\_names
```python theme={null}
@property
def output_gene_names() -> list[str] | None
```
Ordered Ensembl IDs of predicted output genes.
***
#### predict
```python theme={null}
@log_failures
def predict(request: ModelRequest) -> ModelResponse
```
Sends a prediction request to the endpoint.
Tile request payload to send.
**Returns**:
Parsed model response with predictions.
***
#### embed
```python theme={null}
@log_failures
def embed(request: ModelRequest) -> ModelResponse
```
Sends an embedding request to the endpoint.
Tile request payload to send.
**Returns**:
Parsed model response with embeddings.
***
#### predict\_with\_embedding
```python theme={null}
@log_failures
def predict_with_embedding(request: ModelRequest) -> ModelResponse
```
Send a combined prediction+embedding request to the endpoint.
Returns both the gene prediction (`output`) and the tile embedding (`embedding`) from a single forward pass, so callers needing both avoid a second inference.
Tile request payload to send.
**Returns**:
Parsed model response with `embedding` populated alongside
`output`.
***
#### predict\_async
```python theme={null}
@log_failures
async def predict_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Sends a prediction request asynchronously.
Tile request payload to send.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Parsed model response with predictions.
***
#### embed\_async
```python theme={null}
@log_failures
async def embed_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Sends an embedding request asynchronously.
Tile request payload to send.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Parsed model response with embeddings.
***
#### predict\_with\_embedding\_async
```python theme={null}
@log_failures
async def predict_with_embedding_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Send a combined prediction+embedding request asynchronously.
Tile request payload to send.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
Parsed model response with `embedding` populated alongside
`output`.
***
#### \_\_call\_\_
```python theme={null}
def __call__(request: ModelRequest) -> ModelResponse
```
Alias for [`predict`](/sdk-reference/models/endpoint_model#predict).
# bioptimus.models.types
Source: https://docs.bioptimus.com/sdk-reference/models/types
Model specification types.
Defines [`ModelSpec`](/sdk-reference/models/types#modelspec) which encapsulates a model's identity, input tile requirements, normalisation parameters, output contract, and compute hints.
Reuses `TileSpec` for tile geometry so the same spec drives both the extraction pipeline and the model's expectations.
**Example**:
```python theme={null}
from bioptimus.extraction.wsi.types import TileSpec
from bioptimus.io.wsi.types import MPP, MeasurementUnit
from bioptimus.models.types import ModelSpec
spec = ModelSpec(
model_name="h1",
version="1.0.0",
tile_spec=TileSpec(
size=(224, 224), stride=(224, 224),
resolution=MPP(0.5), unit=MeasurementUnit.PIXELS,
),
embedding_dim=1536,
output_type="cls+patch_mean",
num_prefix_tokens=1,
patch_size=14,
mean=(0.707, 0.579, 0.704),
std=(0.212, 0.230, 0.178),
weights_source="bioptimus/H-optimus-1",
)
```
## Models
```python theme={null}
class Models(str, Enum)
```
Known model identifiers.
Values correspond to the `model_name` field in each YAML config under `bioptimus/models/configs/`.
## ModelSpec
```python theme={null}
@dataclass(frozen=True)
class ModelSpec()
```
Describes a backbone model's input and output contract.
Combines tile extraction requirements (via [`TileSpec`](/sdk-reference/data/tile_spec#tilespec)) with preprocessing, architecture, and output metadata so that data pipelines can prepare inputs and interpret outputs correctly without manual configuration.
This is **framework-agnostic** — it contains only plain Python types and can be consumed by PyTorch, TensorFlow, JAX, or any other framework.
Groups: **Identity & provenance** — who is this model? **Input / preprocessing** — what does the model expect? **Architecture** — structural hints for generic code. **Output** — how to interpret the raw model output. **Compute** — precision & hardware hints.
**Attributes**:
* `model_name` - Unique identifier / registry key (e.g. `"h1"`).
* `version` - Model version string (e.g. `"1.0.0"`). Ensures embeddings extracted with v1 are not mixed with v2.
* `weights_source` - HuggingFace repo, URL, or local path to weights (e.g. `"bioptimus/H-optimus-1"`). `None` = no auto-download.
* `license` - SPDX identifier or short description (e.g. `"proprietary"`, `"apache-2.0"`).
* `tile_spec` - Tile geometry the model expects (size, stride, resolution, measurement unit). Directly reusable by `TileExtractor`.
* `num_channels` - Number of input image channels (default 3 for RGB).
* `mean` - Per-channel normalisation mean (channel order matches input).
* `std` - Per-channel normalisation std.
* `interpolation` - Resize interpolation mode string (e.g. `"bicubic"`, `"bilinear"`).
* `antialias` - Whether the resize operation should use antialiasing.
* `color_space` - Expected colour space of the input image (`"RGB"`, `"BGR"`, `"HED"`).
* `stain_normalization` - Stain normalisation method applied *before* the model, or `None` for no stain normalisation. (e.g. `"macenko"`, `"reinhard"`, `"vahadane"`).
* `architecture` - Architecture family (`"vit"`, `"swin"`, `"resnet"`, `"convnext"`, …).
* `patch_size` - ViT patch size (e.g. 14, 16). Determines the number of output tokens = `(tile_size / patch_size)²`. `None` for non-ViT architectures.
* `num_prefix_tokens` - Number of non-spatial prefix tokens before patch tokens in the output sequence (CLS, register tokens, …). E.g. DINOv2-reg = 5, most ViTs = 1.
* `embedding_dim` - Dimensionality of the *final* output feature vector (after any post-processing like CLS + mean pooling).
* `output_type` - How the raw model output is consumed: `"cls"` | `"patch_mean"` | `"cls+patch_mean"` | `"dense"` | `"token_sequence"`.
* `output_spatial_dims` - For dense / segmentation models that output a spatial feature map, e.g. `(16, 16)`. `None` for pooled-output models.
* `precision` - Recommended inference precision (`"fp32"`, `"fp16"`, `"bf16"`).
# bioptimus.observability
Source: https://docs.bioptimus.com/sdk-reference/observability
Public observability helpers for logging, errors, context, and diagnostics.
A dependency-light, standard-library-only foundation shared by the SDK, the in-process local backend, and the FastAPI server. The package root exposes only the stable user-facing entry points:
* [`ErrorCode`](/sdk-reference/observability/errors#errorcode) / [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) — typed, coded errors;
* correlation-context helpers ([`bind_context`](/sdk-reference/observability/context#bind_context), [`get_context`](/sdk-reference/observability/context#get_context), [`new_trace_id`](/sdk-reference/observability/context#new_trace_id));
* structured logging helpers ([`configure_logging`](/sdk-reference/observability/logging#configure_logging), [`default_log_file`](/sdk-reference/observability/logging#default_log_file), [`log_event`](/sdk-reference/observability/logging#log_event), [`log_failures`](/sdk-reference/observability/logging#log_failures), [`read_recent_logs`](/sdk-reference/observability/log_export#read_recent_logs), [`read_usage_records`](/sdk-reference/observability/log_export#read_usage_records)).
Lower-level formatters, anonymizers, and diagnostic probes are available from their implementation modules for internal composition, but they are not re-exported here as top-level SDK API. Endpoint verification (`bioptimus.observability.healthcheck.verify_endpoint`) is likewise imported from its module: unlike the helpers above it needs the model stack, so re-exporting it would pull those imports into every consumer of this package.
# bioptimus.observability.anonymize
Source: https://docs.bioptimus.com/sdk-reference/observability/anonymize
PII anonymization for logs and support-bundle artifacts.
Whole-slide-image paths, filesystem paths, and object-store URIs frequently embed patient identifiers (for example `TCGA-A6-1234_patientX.svs`). Before any log line, manifest, or bundle component leaves the customer environment it is passed through this module. By default every recognized field (`ANONYMIZED_FIELDS`) is replaced by a stable hash, so a given slide keeps a consistent pseudonym across every artifact and "which slide failed" correlation survives. The digest is unsalted and truncated by design, so this is pseudonymization for support triage, not cryptographic anonymity: a party who can enumerate the identifier space can confirm a value (see ADR-005). The per-field policy is configurable via [`AnonymizationConfig`](/sdk-reference/observability/anonymize#anonymizationconfig): a field can instead be redacted entirely or kept verbatim. Paths and URIs embedded in free text are always redacted. Nothing is ever transmitted automatically — the helpers only rewrite values in place.
Entry points:
* [`AnonymizationConfig`](/sdk-reference/observability/anonymize#anonymizationconfig), the policy controlling each field's
[`Strategy`](/sdk-reference/observability/anonymize#strategy) (`DEFAULT_CONFIG` hashes every recognized field);
* [`AnonymizingFilter`](/sdk-reference/observability/anonymize#anonymizingfilter), a `logging.Filter` that scrubs log records
in place; and
* the pure helpers [`hash_identifier`](/sdk-reference/observability/anonymize#hash_identifier), [`redact_text`](/sdk-reference/observability/anonymize#redact_text),
[`scrub_user_paths`](/sdk-reference/observability/anonymize#scrub_user_paths), [`anonymize_value`](/sdk-reference/observability/anonymize#anonymize_value), and [`anonymize_mapping`](/sdk-reference/observability/anonymize#anonymize_mapping), reused by the support-bundle builder to scrub files on disk.
**Warnings**:
This redaction is best-effort, not a guarantee. It recognizes a fixed set
of field names and path/URI shapes; PII in an unrecognized field, an
unusual identifier format, or an unexpected free-text form can pass through
unredacted. The hash used is unsalted, truncated pseudonymization, not
cryptographic anonymity. Bioptimus makes no warranty that the output is
free of PII and accepts no liability for it. Audit the redacted output
internally before sharing it outside your environment.
***
#### REDACTION\_MARKER
Replacement token substituted for every redacted value.
***
#### USER\_PATH\_MARKER
Placeholder substituted for the user name in a home-directory path, so a diagnostic path keeps its structure (for example `.venv/bin/python3`) without exposing whose account it is.
***
#### HASH\_PREFIX
Prefix marking a value that has been replaced by a one-way hash pseudonym.
***
#### HASH\_FIELDS
Identifier fields that act as correlation keys. Hashing (rather than redacting) these preserves the "which slide failed" link across every log line, manifest, and ledger entry without directly exposing the underlying identity (though the unsalted, truncated hash is pseudonymization, not cryptographic anonymity).
***
#### PII\_FIELDS
Path-like fields. Their values embed directory structure and user names beyond the bare identity; hashing keeps them correlatable while removing the layout. Configure them to [`Strategy.REDACT`](/sdk-reference/observability/anonymize#redact) to drop them instead.
***
#### ANONYMIZED\_FIELDS
Every field anonymized by default. Each is hashed unless an [`AnonymizationConfig`](/sdk-reference/observability/anonymize#anonymizationconfig) override selects a different strategy.
## Strategy
```python theme={null}
class Strategy(str, Enum)
```
How a recognized field's value is anonymized.
***
#### HASH
Replace with a stable one-way hash, preserving cross-artifact correlation.
***
#### REDACT
Replace with `REDACTION_MARKER`, removing the value entirely.
***
#### KEEP
Keep verbatim; only embedded paths, URIs, and user names are scrubbed.
## AnonymizationConfig
```python theme={null}
@dataclass(frozen=True)
class AnonymizationConfig()
```
Per-field anonymization policy.
The `default` strategy applies to every field in `ANONYMIZED_FIELDS`; `overrides` selects a different strategy for named fields. `DEFAULT_CONFIG` hashes every recognized field, so all identifiers and paths become stable pseudonyms while remaining correlatable.
**Attributes**:
* `default` - Strategy applied to recognized fields without an override.
* `overrides` - Field-name to strategy mapping that wins over `default`.
***
#### strategy\_for
```python theme={null}
def strategy_for(key: str) -> Strategy
```
Returns the strategy for `key`.
The field name being anonymized.
**Returns**:
The override for `key` if one exists, the `default` strategy for
any recognized field, otherwise [`Strategy.KEEP`](/sdk-reference/observability/anonymize#keep).
***
#### DEFAULT\_CONFIG
Default policy: hash every recognized field.
***
#### redact\_text
```python theme={null}
def redact_text(text: str) -> str
```
Redacts paths, URIs, and slide file names from a free-text string.
Replaces any path or bare file name ending in a known whole-slide-image extension, plus generic absolute paths and `s3://`/`file://` URIs, with `REDACTION_MARKER`. Non-matching text is returned unchanged.
Arbitrary text that may embed slide paths or file names.
**Returns**:
The text with slide path and file-name occurrences redacted.
***
#### scrub\_user\_paths
```python theme={null}
def scrub_user_paths(text: str) -> str
```
Masks the user name embedded in home-directory paths.
Replaces the account component of a `/home/` or `/Users/` path with `USER_PATH_MARKER`, leaving the rest of the path intact so its diagnostic value (for example a virtual-environment or project layout) survives. `/root` and paths outside a home directory are left unchanged.
Arbitrary text that may embed home-directory paths, for example an interpreter path or a temp-directory location.
**Returns**:
The text with the user name in any home-directory path masked.
***
#### hash\_identifier
```python theme={null}
def hash_identifier(value: object) -> str
```
Returns a stable pseudonym for an identifier value.
Hashes the string form of *value* with SHA-256 and returns `HASH_PREFIX` followed by the leading digest characters. Values that already match the SDK pseudonym shape are returned unchanged, making anonymization idempotent when logs were redacted at emission and again at export. The result is deterministic, so the same identifier maps to the same pseudonym across every log line and bundle artifact, preserving the "which slide failed" correlation. The digest is unsalted and truncated by design (stable pseudonyms, short log lines), so it is pseudonymization for support triage, not cryptographic anonymity: a party who can enumerate the identifier space can confirm a value (see ADR-005).
The identifier to pseudonymize, for example a slide or patient ID. Non-string values are hashed by their `str` form.
**Returns**:
A pseudonym of the form `"sha256:"`.
***
#### anonymize\_value
```python theme={null}
def anonymize_value(key: str,
value: Any,
config: AnonymizationConfig = DEFAULT_CONFIG) -> Any
```
Anonymizes a single key/value pair.
The strategy for `key` is resolved from `config`: a recognized field is hashed ([`hash_identifier`](/sdk-reference/observability/anonymize#hash_identifier)) or redacted, and a kept field keeps its value with embedded slide paths, absolute paths, URIs, and home-directory user names scrubbed. A `None` value is never hashed.
The field name associated with `value`.
The field value to inspect.
The anonymization policy. Defaults to `DEFAULT_CONFIG`, which hashes every recognized field.
**Returns**:
The hashed, redacted, or scrubbed value depending on the strategy.
***
#### anonymize\_mapping
```python theme={null}
def anonymize_mapping(
data: Mapping[str, Any],
config: AnonymizationConfig = DEFAULT_CONFIG) -> dict[str, Any]
```
Recursively anonymizes a mapping.
Applies [`anonymize_value`](/sdk-reference/observability/anonymize#anonymize_value) to every entry, recursing into nested mappings and sequences. The input is not mutated.
The mapping to anonymize, for example a manifest or ledger.
The anonymization policy. Defaults to `DEFAULT_CONFIG`.
**Returns**:
A new mapping with PII values anonymized.
## AnonymizingFilter
```python theme={null}
class AnonymizingFilter(config: AnonymizationConfig = DEFAULT_CONFIG, name: str = '')
```
A logging filter that anonymizes PII in records in place.
Scrubs the structured-field payload attached by `bioptimus.observability.logging.log_event`, the record message, and any positional message arguments, so PII never reaches a handler's formatter. Recognized fields are hashed or redacted per the configured [`AnonymizationConfig`](/sdk-reference/observability/anonymize#anonymizationconfig). The filter always returns `True` — it never drops records.
***
#### filter
```python theme={null}
def filter(record: logging.LogRecord) -> bool
```
Anonymizes PII in the given record.
The log record to scrub. It is mutated in place.
**Returns**:
`True` always, so the record continues through the handler.
# bioptimus.observability.context
Source: https://docs.bioptimus.com/sdk-reference/observability/context
Correlation-ID context propagation for the SDK pipeline.
Carries lightweight, thread- and task-local context (a `trace_id` for a whole run, plus `run_id`, `slide_id`, and the current pipeline `stage`) down the execution chain without threading parameters through every function. Structured logging reads this context so every log line is automatically tagged with `run -> slide -> request` correlation fields.
The context lives in a single `contextvars.ContextVar`, so it is isolated per thread and per `asyncio` task. That makes it safe under the SDK's `ThreadPoolExecutor` fan-out (one slide per worker) and its `asyncio` tile dispatch (one task per request).
***
#### new\_trace\_id
```python theme={null}
def new_trace_id() -> str
```
Generates a new, time-sortable correlation ID.
Produces a 26-character ULID-style identifier: a 48-bit millisecond timestamp followed by 80 bits of randomness, encoded with Crockford base32. The leading timestamp makes IDs lexicographically sortable by creation time.
**Returns**:
A 26-character uppercase Crockford base32 string.
***
#### get\_context
```python theme={null}
def get_context() -> dict[str, str]
```
Returns a copy of the currently bound correlation context.
**Returns**:
A mapping of the bound context fields (for example `trace_id`,
`run_id`, `slide_id`, `stage`). Empty when nothing is bound.
***
#### bind\_context
```python theme={null}
@contextmanager
def bind_context(**fields: str | None) -> Iterator[None]
```
Binds correlation fields for the duration of the `with` block.
Merges the given fields onto the current context, yields, then restores the previous context on exit (including on error). Fields whose value is `None` are ignored, which makes optional binding ergonomic.
Correlation fields to bind, for example `trace_id`, `run_id`, `slide_id`, or `stage`.
**Returns**:
A context manager that yields `None`; the merged context is active
only inside the `with` block and is restored on exit.
# bioptimus.observability.errors
Source: https://docs.bioptimus.com/sdk-reference/observability/errors
Typed error codes and the unified SDK exception.
The SDK raises a single exception type, [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror), tagged with a machine-readable [`ErrorCode`](/sdk-reference/observability/errors#errorcode). Low-level code raises the typed error with structured `context`; this module owns the actionable remediation text for each code. The system boundary (for example the per-slide handler) logs it once with the attached code, so whoever reads the log gets an instant explanation and a direct fix.
Each [`ErrorCode`](/sdk-reference/observability/errors#errorcode) member carries two pieces of information:
* its value is the stable, dotted support code (for example `"BIO-WSI-001"`) used in logs, dashboards, and customer communication; and
* its name maps to a human-readable `event` slug (for example `"wsi_backend_unavailable"`) via [`ErrorCode.event`](/sdk-reference/observability/errors#event).
## ErrorCode
```python theme={null}
class ErrorCode(StrEnum)
```
Stable, machine-readable error codes for SDK failures.
The member value is the dotted support code embedded in logs and shared with customers. The member name doubles as a structured-log `event` slug through `event`.
***
#### event
```python theme={null}
@property
def event() -> str
```
Returns the structured-log event slug for this code.
**Returns**:
The lowercased member name, for example `"wsi_backend_unavailable"`.
***
#### remediation
```python theme={null}
@property
def remediation() -> str | None
```
Returns the central remediation hint for this code, if one exists.
***
#### remediation\_for
```python theme={null}
def remediation_for(code: ErrorCode | str) -> str | None
```
Returns the central remediation hint for an error code.
An [`ErrorCode`](/sdk-reference/observability/errors#errorcode) member or its string value.
**Returns**:
The remediation string when the code is known, otherwise `None`.
## BioptimusError
```python theme={null}
class BioptimusError(code: ErrorCode,
*,
message: str | None = None,
context: dict[str, Any] | None = None,
remediation: str | None = None)
```
Unified, typed exception raised across the SDK.
Wraps a machine-readable [`ErrorCode`](/sdk-reference/observability/errors#errorcode) together with structured diagnostic `context` and an actionable `remediation` hint. It is designed to be raised at low levels (`raise BioptimusError(...) from exc`) and logged exactly once at a system boundary.
***
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Returns the structured representation for logging or serialization.
**Returns**:
A mapping with the error code, event slug, message, context, and
remediation, suitable for inclusion in a structured log record.
## BioptimusValueError
```python theme={null}
class BioptimusValueError(BioptimusError, ValueError)
```
A [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) that is also a `ValueError`.
Raised where the SDK historically raised a plain `ValueError` (for example an unsupported format or invalid input). Inheriting from both keeps existing `except ValueError` handlers working while exposing the typed [`code`](/sdk-reference/observability/errors#code), `context`, and `remediation`.
## BioptimusRuntimeError
```python theme={null}
class BioptimusRuntimeError(BioptimusError, RuntimeError)
```
A [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) that is also a `RuntimeError`.
Raised where the SDK historically raised a plain `RuntimeError` (for example an unavailable backend or a failed operation). Inheriting from both keeps existing `except RuntimeError` handlers working while exposing the typed [`code`](/sdk-reference/observability/errors#code), `context`, and `remediation`.
## BioptimusImportError
```python theme={null}
class BioptimusImportError(BioptimusError, ImportError)
```
A [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) that is also an `ImportError`.
Raised when a feature needs an optional dependency shipped as a `bioptimus-sdk` extra (for example `bioptimus-sdk[torch]`) that is not installed. Inheriting from both keeps existing `except ImportError` handlers working while exposing the typed [`code`](/sdk-reference/observability/errors#code), `context`, and `remediation` that name the missing extra and the exact install command.
***
#### wrap\_as
```python theme={null}
@contextmanager
def wrap_as(code: ErrorCode,
*,
message: str,
error_cls: type[BioptimusError] = BioptimusRuntimeError,
**context: Any) -> Iterator[None]
```
Translates an unexpected exception in the block into a typed error.
Centralizes the recurring "wrap a low-level failure as a typed error" boundary: an already-typed [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) propagates unchanged, while any other exception is re-raised as `error_cls` with `code`, `message`, and `context` augmented with the original exception's type and detail. This mirrors `bioptimus.observability.fsboundary.fs_read`, but for arbitrary operations rather than filesystem reads.
The [`ErrorCode`](/sdk-reference/observability/errors#errorcode) to tag the wrapped failure with.
The human-readable message for the wrapped error.
The typed error class to raise. Defaults to [`BioptimusRuntimeError`](/sdk-reference/observability/errors#bioptimusruntimeerror); pass [`BioptimusValueError`](/sdk-reference/observability/errors#bioptimusvalueerror) for input-validation failures.
Structured context fields recorded on the wrapped error, in addition to the automatically added `error_type` and `detail`.
**Returns**:
A context manager that re-raises unexpected exceptions as `error_cls`.
# bioptimus.observability.fsboundary
Source: https://docs.bioptimus.com/sdk-reference/observability/fsboundary
Filesystem boundary helpers that translate raw I/O failures into typed errors.
This module centralizes the policy for converting low-level `OSError` and parser exceptions into `BioptimusError` with a stable `ErrorCode`, structured `context`; remediation text lives centrally with the error code.
The two context managers are applied at the exact boundary call (`open`, `yaml.safe_load`, `Path.mkdir`) so that domain logic stays outside the `try` and the translation policy lives in one place rather than being copy-pasted into every reader and writer. The approach mirrors how mature libraries centralize exception translation, for example httpx's `map_httpcore_exceptions` context manager and PyTorch's `_check_seekable` boundary validator.
To keep this module dependency-light (standard library only), parser exception types that are not built in (for example `yaml.YAMLError`) are not imported here; callers pass them via the `malformed` argument of [`fs_read`](/sdk-reference/observability/fsboundary#fs_read).
***
#### fs\_read
```python theme={null}
@contextmanager
def fs_read(
path: str | Path,
*,
description: str,
malformed: tuple[type[BaseException], ...] = ()
) -> Iterator[None]
```
Translates read and parse failures within the block into typed errors.
Wraps the boundary call that reads or parses *path*. Raw filesystem and parser exceptions are converted into a [`BioptimusValueError`](/sdk-reference/observability/errors#bioptimusvalueerror) with code [`ErrorCode.INVALID_CONFIG`](/sdk-reference/observability/errors#invalid_config), attaching the path and errno to the context and an errno-specific remediation hint when available. Already-typed [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) instances raised inside the block propagate unchanged so that domain errors keep their original code.
The filesystem path being read, used for structured context.
A short noun phrase naming the artifact, for example `"cohort CSV manifest"`. It is interpolated into the message.
Additional, non-built-in exception types (for example `yaml.YAMLError` or `csv.Error`) that signal malformed content and should be reported as [`ErrorCode.INVALID_CONFIG`](/sdk-reference/observability/errors#invalid_config). Built-in `UnicodeDecodeError` is always treated as malformed.
**Returns**:
A context manager that translates read and parse failures raised inside
the `with` block into a typed [`BioptimusValueError`](/sdk-reference/observability/errors#bioptimusvalueerror) with code
[`ErrorCode.INVALID_CONFIG`](/sdk-reference/observability/errors#invalid_config).
***
#### fs\_write
```python theme={null}
@contextmanager
def fs_write(path: str | Path, *, description: str) -> Iterator[None]
```
Translates write failures within the block into typed errors.
Wraps the boundary call that writes *path*. Raw `OSError` failures are converted into a [`BioptimusRuntimeError`](/sdk-reference/observability/errors#bioptimusruntimeerror): code [`ErrorCode.DISK_FULL`](/sdk-reference/observability/errors#disk_full) when the volume is out of space or over quota, or [`ErrorCode.OUTPUT_WRITE_FAILED`](/sdk-reference/observability/errors#output_write_failed) otherwise, attaching the path to the context. Already-typed [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) instances raised inside the block propagate unchanged.
The destination path being written, used for structured context.
A short noun phrase naming the artifact, for example `"cohort manifest"`. It is interpolated into the message.
**Returns**:
A context manager that translates write failures raised inside the
`with` block into a typed [`BioptimusRuntimeError`](/sdk-reference/observability/errors#bioptimusruntimeerror) with code
[`ErrorCode.DISK_FULL`](/sdk-reference/observability/errors#disk_full) or [`ErrorCode.OUTPUT_WRITE_FAILED`](/sdk-reference/observability/errors#output_write_failed).
# bioptimus.observability.healthcheck
Source: https://docs.bioptimus.com/sdk-reference/observability/healthcheck
Post-deployment endpoint verification with a synthetic tile.
Answers the first question a customer has after deploying a model and installing the SDK: *is the endpoint up, reachable, and returning the output I expect?* The check builds a synthetic tile of the exact geometry the model declares, sends it through whichever backend the customer deployed (`remote`, `aws`, `local`, or `huggingface`), and validates the response against the model's output contract.
Unlike the probes in `bioptimus.observability.probes`, this module performs real work: it constructs a client, opens a connection or loads a checkpoint, and issues a billable inference request. It is therefore never run implicitly as part of a diagnostic snapshot — the caller asks for it explicitly, through [`verify_endpoint`](/sdk-reference/observability/healthcheck#verify_endpoint), the `bioptimus verify` CLI subcommand, or the `verify` argument of `build_support_bundle`.
Like the probes, the entry point is fail-safe by default: every failure is captured as a [`Check`](/sdk-reference/observability/healthcheck#check) carrying its `ErrorCode` and remediation, so a broken deployment still yields a complete, shareable report instead of a traceback. Pass `raise_on_failure=True` to turn the first failure into a raised `BioptimusRuntimeError` instead.
**Example**:
```python theme={null}
from bioptimus.observability.healthcheck import verify_endpoint
report = verify_endpoint("h1", backend="remote", base_url="http://localhost:8080")
print(report.ok, [check.name for check in report.checks if not check.passed])
```
## Check
```python theme={null}
@dataclass(frozen=True)
class Check()
```
Outcome of a single verification step.
**Attributes**:
* `name` - Stable slug identifying the step, for example `"embedding_request"`.
* `passed` - Whether the step succeeded. A skipped step is reported as passed so that a model which does not offer a mode cannot fail the run.
* `duration_ms` - Wall-clock duration of the step in milliseconds.
* `skipped` - Whether the step did not run (for example a mode the model does not expose). `detail` carries the reason.
* `detail` - Human-readable outcome or failure detail.
* `error_code` - The `ErrorCode` value for a failed step, otherwise `None`.
* `remediation` - Actionable hint for a failed step, otherwise `None`.
* `context` - Structured fields describing the step, for example the observed and expected output lengths.
## HealthReport
```python theme={null}
@dataclass(frozen=True)
class HealthReport()
```
Result of verifying one model on one backend.
**Attributes**:
* `schema_version` - Version of this report's shape.
* `captured_at` - UTC ISO-8601 timestamp of the run.
* `model` - Model name that was verified, for example `"h1"`.
* `backend` - Backend that was verified, for example `"aws"`.
* `target` - Human-readable description of what was reached (a base URL, a SageMaker endpoint name, a checkpoint path, or a Hub repo), or `None` when the client could not be built.
* `modes` - Inference modes that were attempted.
* `ok` - Whether every non-skipped check passed.
* `checks` - The checks in execution order.
***
#### failures
```python theme={null}
@property
def failures() -> list[Check]
```
Returns the checks that ran and failed, in execution order.
***
#### to\_dict
```python theme={null}
def to_dict() -> dict[str, Any]
```
Returns the JSON-friendly representation of the report.
**Returns**:
A mapping suitable for `json.dumps`, for inclusion in a
support bundle, or for printing from the CLI.
***
#### summary
```python theme={null}
def summary() -> str
```
Returns a one-line, human-readable verdict.
**Returns**:
A string such as `"h1 on remote: ok (3 checks)"` or
`"h1 on remote: FAILED — embedding_request [BIO-NET-001]"`.
***
#### synthetic\_tile
```python theme={null}
def synthetic_tile(size: tuple[int, int] = _DEFAULT_TILE_SIZE,
*,
seed: int = _DEFAULT_SEED) -> Image.Image
```
Builds a deterministic synthetic RGB tile.
The tile combines a smooth two-axis gradient with low-amplitude seeded noise, so it is neither a constant image (which can mask a broken normalisation path) nor pure noise (which compresses poorly and is harder to eyeball in a server log). The same `seed` and `size` always produce byte-identical pixels, so repeated runs are directly comparable.
Tile `(width, height)` in pixels.
Seed for the noise component.
**Returns**:
An RGB `PIL.Image.Image` of the requested size.
***
#### synthetic\_expression
```python theme={null}
def synthetic_expression(num_genes: int) -> list[float]
```
Builds a synthetic bulk-RNA expression vector.
Number of genes the model expects, in the model's gene order.
**Returns**:
A list of `num_genes` identical, non-zero expression values.
***
#### verify\_endpoint
```python theme={null}
@log_failures
def verify_endpoint(model: str = "h1",
backend: str = "remote",
*,
modes: Sequence[str] | str = "auto",
tile_size: tuple[int, int] | None = None,
seed: int = _DEFAULT_SEED,
raise_on_failure: bool = False,
**backend_kwargs: Any) -> HealthReport
```
Verifies that a deployed model endpoint is up and correctly configured.
Builds a client for `backend`, fetches model metadata, sends one synthetic tile per requested inference mode, and validates each response against the model's declared output contract (length, finiteness, and echoed tile metadata). Every step is recorded as a [`Check`](/sdk-reference/observability/healthcheck#check); nothing raises unless `raise_on_failure` is set.
Model name, for example `"h1"` or `"m-optimus"`. Must match a registered SDK model config.
Backend to verify: `"remote"`, `"aws"`, `"local"`, or `"huggingface"`.
Inference modes to exercise. `"auto"` (the default) resolves the modes the model actually exposes — embedding only for the plain backbones, prediction only for `tissue-seg`, both for `m-optimus` — and narrows to embedding for the `huggingface` backend, which has no prediction head. Pass `"embedding"`, `"prediction"`, or a sequence of both to force a specific set.
Tile `(width, height)` to send. Defaults to the size in the model's tile spec, which is what a correctly configured endpoint expects; override only to probe how an endpoint handles another size.
Seed for the synthetic tile, so runs are reproducible.
When `True`, raises instead of returning a report whose `ok` is `False`.
Forwarded verbatim to `Backbone` — for example `base_url` for `remote`, `endpoint_name` and `region_name` for `aws`, `model_dir` or `checkpoint` for `local`, and `device` or `precision` for `huggingface`.
**Returns**:
A [`HealthReport`](/sdk-reference/observability/healthcheck#healthreport) describing every check that ran.
**Raises**:
* `BioptimusValueError` - When `modes` names a mode that cannot be verified, or when `tile_size` is not a pair of positive integers. Both are raised up front, before any client is built.
* `BioptimusRuntimeError` - When `raise_on_failure` is `True` and any check failed. The error carries the first failure's code and the full report in its `context`.
# bioptimus.observability.log_export
Source: https://docs.bioptimus.com/sdk-reference/observability/log_export
Redacting readers over the structured log files.
Reads the JSON log file `bioptimus.observability.logging` writes and returns its entries as mappings, anonymized by default so the result is suitable to export in a support bundle or the server `/diagnostics` response. This redaction is best-effort, not a guarantee (see `bioptimus.observability.anonymize`). Two views are provided: [`read_recent_logs`](/sdk-reference/observability/log_export#read_recent_logs) tails the active file, and [`read_usage_records`](/sdk-reference/observability/log_export#read_usage_records) selects model-request usage entries across the active file and all of its rotations. Both resolve the log file exactly as `bioptimus.observability.logging.configure_logging` does and reuse the same per-field anonymization policy as emission-time redaction.
***
#### USAGE\_EVENT
Structured-log `event` slug identifying a model-request usage record. The API server emits it on every inference, embedding, and metadata call, so [`read_usage_records`](/sdk-reference/observability/log_export#read_usage_records) can select these entries for usage reporting.
***
#### read\_recent\_logs
```python theme={null}
def read_recent_logs(log_file: str | Path | None = None,
*,
max_entries: int = _DEFAULT_LOG_TAIL,
anonymize: bool = True) -> list[dict[str, Any]]
```
Reads the most recent entries from the active log file.
Resolves the log file the same way [`configure_logging`](/sdk-reference/observability/logging#configure_logging) does: from `log_file` when given, otherwise the `BIOPTIMUS_LOG_FILE` environment variable, otherwise [`default_log_file`](/sdk-reference/observability/logging#default_log_file). Reads up to the last `max_entries` lines, parses each structured JSON line into a mapping, and (by default) anonymizes it exactly as it is anonymized at emission. A line that is not JSON is wrapped as a `{"message": ...}` entry with embedded slide paths and user names scrubbed.
Only the active log file is read; rotated files are left to the support bundle, which archives the full history. When file logging is disabled (`BIOPTIMUS_LOG_FILE` set empty) or the file does not yet exist, an empty list is returned.
Path to the log file to read. When omitted, the `BIOPTIMUS_LOG_FILE` environment variable supplies it and, if that is unset, [`default_log_file`](/sdk-reference/observability/logging#default_log_file).
Maximum number of trailing entries to return.
When `True` (the default), redacts PII from each entry so the result is safe to export.
**Returns**:
Recent log entries, oldest first, each as a mapping.
***
#### read\_usage\_records
```python theme={null}
def read_usage_records(log_file: str | Path | None = None,
*,
anonymize: bool = True) -> list[dict[str, Any]]
```
Reads every model-request usage record from the active and rotated logs.
Selects the structured log entries whose `event` is `USAGE_EVENT` across the active log file and all of its rotations, so the result is the complete usage history rather than the recent tail [`read_recent_logs`](/sdk-reference/observability/log_export#read_recent_logs) returns. Records are returned oldest first: the rotated files (ordered by their date suffix) precede the active file, and entries within each file keep their append order. Each record is anonymized the same way it was at emission unless `anonymize` is `False`.
The log file is resolved exactly as by [`configure_logging`](/sdk-reference/observability/logging#configure_logging): from `log_file` when given, otherwise the `BIOPTIMUS_LOG_FILE` environment variable, otherwise [`default_log_file`](/sdk-reference/observability/logging#default_log_file). When file logging is disabled (`BIOPTIMUS_LOG_FILE` set empty) or no log file exists yet, an empty list is returned.
Path to the active log file. When omitted, the `BIOPTIMUS_LOG_FILE` environment variable supplies it and, if that is unset, [`default_log_file`](/sdk-reference/observability/logging#default_log_file). Rotated files sharing the same base name are read automatically.
When `True` (the default), redacts PII from each record so the result is safe to export.
**Returns**:
Every usage record, oldest first, each as a mapping.
# bioptimus.observability.logging
Source: https://docs.bioptimus.com/sdk-reference/observability/logging
Structured JSON logging for the SDK.
Provides a JSON log formatter, a helper for emitting structured events with a machine-readable error code and arbitrary fields, and a [`configure_logging`](/sdk-reference/observability/logging#configure_logging) entry point for explicit setup.
Logging works with zero setup: importing the package has no side effects, but the first emitted event auto-installs a default file handler (see [`default_log_file`](/sdk-reference/observability/logging#default_log_file)) so a durable record always exists; the console stays silent unless `configure_logging(console=True)` (or an explicit `stream`) is passed, and records always propagate to the application's own logging hierarchy. Call [`configure_logging`](/sdk-reference/observability/logging#configure_logging) to override the destination, level, or anonymization, or set `BIOPTIMUS_LOG_FILE` to redirect the file. Setting `BIOPTIMUS_LOG_FILE` to an empty string disables the SDK's own **file** sink and auto-init — it does not silence logging, since records still propagate to any handlers the host application (or `configure_logging(console=True)`) has installed. Each emitted line carries a timestamp, the level, an `event` slug (the log message), the active `run -> slide -> request` correlation context, any structured fields passed via [`log_event`](/sdk-reference/observability/logging#log_event), and an optional `error_code`. PII is redacted by `bioptimus.observability.anonymize.AnonymizingFilter` when enabled.
***
#### STRUCTURED\_KEY
Custom LogRecord attribute holding the structured-field payload.
## JsonFormatter
```python theme={null}
class JsonFormatter(logging.Formatter)
```
Formats log records as single-line JSON objects.
The output object always includes `timestamp` (ISO 8601, UTC), `level`, `event` (the formatted message), and `logger`. Structured fields attached via [`log_event`](/sdk-reference/observability/logging#log_event), together with the active correlation context, are merged in at the top level. Exception and stack information, when present, are rendered under `exception` and `stack`.
***
#### format
```python theme={null}
def format(record: logging.LogRecord) -> str
```
Renders the record as a JSON string.
The log record to serialize.
**Returns**:
A single-line JSON document describing the record.
## ConsoleFormatter
```python theme={null}
class ConsoleFormatter(logging.Formatter)
```
Human-readable single-line formatter for console output.
Renders `timestamp level event [key=value ...]` so logs are easy to scan in a terminal or notebook, while the file handler retains full JSON.
***
#### format
```python theme={null}
def format(record: logging.LogRecord) -> str
```
Renders the record as a readable text line.
The log record to format.
**Returns**:
A human-readable single-line message, or a compact multiline
message when structured fields are present.
***
#### log\_event
```python theme={null}
def log_event(logger: logging.Logger,
level: int,
event: str,
*,
error_code: ErrorCode | str | None = None,
exc_info: bool | BaseException = False,
fields: dict[str, Any] | None = None,
**extra: Any) -> None
```
Emits a structured log event.
The logger to emit on.
The logging level, for example `logging.WARNING`.
A short, stable event slug used as the log message, for example `"wsi_backend_unavailable"`.
Optional machine-readable error code recorded under `error_code`.
Exception information forwarded to the logging call, enabling an `exception` field in the output.
Structured fields supplied as a single dict, merged into the log object. Callers with untrusted keys (for example an error's `context`) pass them here so a key such as `error_code` cannot collide with this function's named parameters. Explicit `extra` keyword arguments win on collision.
Structured fields supplied as keyword arguments, for example `backend="cucim"` or `reason="missing library"`. Convenient for hardcoded literals at the call site.
***
#### log\_error
```python theme={null}
def log_error(logger: logging.Logger,
error: BioptimusError,
*,
level: int = logging.ERROR,
exc_info: bool | BaseException = False,
**fields: Any) -> None
```
Logs a [`BioptimusError`](/sdk-reference/observability/errors#bioptimuserror) once, with its code, context, and fix.
Flattens the error's structured `context` and attaches its machine-readable `error_code`, `message`, and `remediation` so a single boundary log line fully explains the failure and how to resolve it. Extra `fields` override context entries on collision.
The logger to emit on.
The typed error to record.
The logging level. Defaults to `logging.ERROR`.
Exception information forwarded to the logging call, enabling an `exception` field in the output.
Extra structured fields merged on top of the error context.
***
#### is\_logged
```python theme={null}
def is_logged(exc: BaseException) -> bool
```
Reports whether a boundary has already logged an exception.
The exception to inspect.
**Returns**:
`True` when a [`log_failures`](/sdk-reference/observability/logging#log_failures) boundary has already recorded the
exception, so an outer boundary can avoid logging it a second time.
***
#### log\_failures
```python theme={null}
def log_failures(func: F | None = None,
*,
logger: logging.Logger | None = None,
level: int = logging.ERROR) -> F | Callable[[F], F]
```
Logs any failure from a public SDK call once, then re-raises it unchanged.
Wraps a public function or method so that any `Exception` it raises is recorded exactly once before propagating: a `BioptimusError` is logged with its code, context, and remediation, and any other exception with its type and message, each with a full traceback. The original exception is re-raised unchanged, so existing `except` handlers and return contracts are preserved.
Recording the failure at the boundary the user called guarantees a durable record reaches the configured sink (see [`configure_logging`](/sdk-reference/observability/logging#configure_logging)) even if the process or its console is killed afterwards. Internal helpers do not log; they raise, and the outermost boundary records the failure once. A marker on the exception keeps an outer boundary from logging it again. Both synchronous and asynchronous callables are supported; `KeyboardInterrupt`, `SystemExit`, and `asyncio.CancelledError` do not derive from `Exception` and are never caught.
The callable to wrap, supplied automatically when used as a bare `@log_failures` decorator.
Logger to record on. Defaults to the logger named after the wrapped callable's module.
Level for the boundary log line. Defaults to `logging.ERROR`.
**Returns**:
The wrapped callable when *func* is given, otherwise a decorator that
wraps a callable.
***
#### default\_log\_file
```python theme={null}
def default_log_file() -> Path
```
Returns the default log-file path under the system temp directory.
This cross-platform location is `bioptimus.log` inside the directory reported by `tempfile.gettempdir` (honoring `TMPDIR`/`TEMP`/`TMP` on the respective platforms). It is where [`configure_logging`](/sdk-reference/observability/logging#configure_logging) falls back to when neither an explicit `log_file` argument nor the `BIOPTIMUS_LOG_FILE` environment variable is set, so logging works with no setup while remaining easy to discover and override.
**Returns**:
The default log-file path.
***
#### get\_log\_file
```python theme={null}
def get_log_file() -> Path | None
```
Returns the log-file path the SDK is currently writing to.
Resolution order:
1. Path set via `configure_logging(log_file=...)` — highest priority.
2. `BIOPTIMUS_LOG_FILE` environment variable (empty string disables).
3. Default temp path (e.g. `/tmp/bioptimus.log`).
**Returns**:
The active log-file path, or `None` when file logging is disabled.
***
#### configure\_logging
```python theme={null}
def configure_logging(
*,
level: int | str = logging.INFO,
stream: TextIO | None = None,
log_file: str | Path | None = None,
anonymize: bool = False,
logger_name: str | None = _DEFAULT_LOGGER,
when: str = _DEFAULT_WHEN,
backup_count: int = _DEFAULT_BACKUP_COUNT,
console: bool = False,
file_handler_factory: Callable[[Path], logging.Handler] | None = None
) -> logging.Logger
```
Configures structured JSON logging for the SDK.
Installs a time-rotating file handler on the target logger, attaching correlation-context injection and optional PII anonymization. Importing the package has no logging side effects; the first emitted event auto-installs this same default configuration, and calling this function explicitly overrides it (for a custom path, level, or anonymization). Repeated calls replace the handlers a previous call installed, so the helper is safe to call more than once.
Rotated files are suffixed with the UTC date (for example `sdk.log.2026-06-24`), which keeps them easy to match to a support window.
By default the SDK is **silent on the console** — log records go only to the file sink (for support bundles) and propagate up Python's logger hierarchy. If the application has its own root-logger handlers, SDK logs appear there automatically. To additionally install the SDK's own console handler, pass `console=True`; passing an explicit `stream` implies `console=True`.
The file sink works with no setup: when `log_file` is omitted, the path is resolved from the `BIOPTIMUS_LOG_FILE` environment variable, and when that is unset it defaults to [`default_log_file`](/sdk-reference/observability/logging#default_log_file) (`bioptimus.log` under the system temp directory). To send logs elsewhere, pass `log_file` or set the variable; to disable file logging entirely, set the variable to an empty string.
Time-based rotation is the default, but the retention strategy is overridable: pass `file_handler_factory` to supply any handler (for example a size-based `logging.handlers.RotatingFileHandler`). The SDK still resolves the path, applies the JSON formatter, injects correlation context, and manages the handler's lifecycle, so `when` and `backup_count` are ignored when a factory is given.
Minimum level to emit, as a level name or numeric level.
Stream for the console handler. When provided, implies `console=True`. Defaults to `sys.stderr` when a console handler is installed.
Optional path enabling a time-rotating file handler. When omitted, the `BIOPTIMUS_LOG_FILE` environment variable is used and, if that is unset, [`default_log_file`](/sdk-reference/observability/logging#default_log_file); an explicit argument always takes precedence. Setting the variable to an empty string disables file logging.
When `True`, redacts PII from every emitted record, including the on-disk file (redaction at rest). Defaults to `False`, so records retain full detail and anonymization happens only on export.
Logger to configure. `None` targets the root logger; the default targets the `"bioptimus"` package logger.
Rotation interval passed to `logging.handlers.TimedRotatingFileHandler`, for example `"midnight"` (the default), `"H"`, or `"D"`.
Number of rotated files to retain before the oldest is deleted. Defaults to 14, matching the recommended retention window.
When `True`, installs the SDK's own console handler with human-readable output on `sys.stderr`. Defaults to `False` so the SDK stays silent on the console; log records still propagate to the caller's logging hierarchy, appearing in whatever handlers the application has configured (e.g. on the root logger).
Optional callable that builds the file handler from the resolved log path, replacing the default `logging.handlers.TimedRotatingFileHandler`. Enables a different retention strategy (for example size-based rotation via `logging.handlers.RotatingFileHandler`) while the SDK retains ownership of the path, formatter, and context injection. When given, `when` and `backup_count` are ignored.
**Returns**:
The configured `logging.Logger`.
# bioptimus.observability.probes
Source: https://docs.bioptimus.com/sdk-reference/observability/probes
Fail-safe environment probes for diagnostic snapshots.
Captures a point-in-time picture of the runtime environment that matters for inference and reports it as a plain, JSON-friendly mapping that feeds both the support bundle (`diagnostics.json`) and the server's `/diagnostics` endpoint. The snapshot covers:
* installed SDK and dependency versions ([`probe_versions`](/sdk-reference/observability/probes#probe_versions));
* installed dependencies versus the SDK's declared constraints ([`probe_dependencies`](/sdk-reference/observability/probes#probe_dependencies));
* host platform and resources ([`probe_host`](/sdk-reference/observability/probes#probe_host));
* inference-relevant environment variables ([`probe_env`](/sdk-reference/observability/probes#probe_env));
* GPU, CUDA, and PyTorch runtime state ([`probe_gpu`](/sdk-reference/observability/probes#probe_gpu));
* importable whole-slide-image backends ([`probe_wsi_backends`](/sdk-reference/observability/probes#probe_wsi_backends));
* temporary-directory headroom and leaked AOTInductor extractions ([`probe_disk`](/sdk-reference/observability/probes#probe_disk));
* `.pt2` checkpoint architecture compatibility ([`probe_pt2_compat`](/sdk-reference/observability/probes#probe_pt2_compat)); and
* loaded models ([`probe_models`](/sdk-reference/observability/probes#probe_models)).
The assembled [`snapshot`](/sdk-reference/observability/probes#snapshot) is stamped with a schema version and a capture timestamp so it can be parsed and correlated with the logs over time.
Every probe is fail-safe: it never raises. Any failure is captured in the returned data (for example `{"available": false, "reason": ...}` or an `"error"` key) so a broken environment still yields a usable snapshot.
***
#### probe\_wsi\_backends
```python theme={null}
def probe_wsi_backends() -> dict[str, dict[str, Any]]
```
Probes whether each supported WSI backend can be imported.
Attempts to import each backend library, recording its version when the import succeeds and the failure reason (for example a missing native library) when it does not.
**Returns**:
Mapping of backend name to a status dict: `available` is `True` with
a `version` when the backend imports, otherwise `False` with a
`reason`.
***
#### probe\_disk
```python theme={null}
def probe_disk() -> dict[str, Any]
```
Probes free space and AOTInductor temp-dir usage in the system tempdir.
Reports the temporary directory used for `.pt2` extraction, its free space and percentage used, whether it is writable, and how many leftover AOTInductor extraction directories it contains (a count that grows when packages are not cleaned up). Sets `warning` to `"TMP_NOT_WRITABLE"` when the directory cannot be written (which breaks `.pt2` extraction) or `"TMP_NEARLY_FULL"` when usage is high.
**Returns**:
A status dict with `tmp_path`, `tmp_free_gb`, `tmp_pct_used`,
`aotinductor_temp_dirs`, `writable`, and `warning` (`None` when
healthy), or `{"error": ...}` if the directory cannot be inspected.
***
#### probe\_models
```python theme={null}
def probe_models(
models: Mapping[str, object | None] | None = None
) -> list[dict[str, Any]]
```
Summarizes the supplied models for the diagnostic snapshot.
Accepts a mapping of model name to the loaded model object, or `None` for a model that is configured but failed to load. Each model is reported with its load state and, when loaded, its device and precision. Reading a single model never breaks the probe.
Mapping of model name to loaded model object (or `None` when it failed to load). Omitted or empty yields an empty list.
**Returns**:
A list of per-model status dicts, each with `name` and `loaded` and,
for loaded models, `device` and `precision`.
***
#### probe\_versions
```python theme={null}
def probe_versions() -> dict[str, str | None]
```
Probes installed versions of the SDK and its key dependencies.
Reads versions from installed package metadata (not module attributes), so it works even when a package cannot be imported. Dependency drift and ABI mismatches (for example a `torch` that does not match the compiled `.pt2` artifacts) are a common, hard-to-spot cause of failures.
**Returns**:
Mapping of distribution package name to its installed version string,
or `None` when the package is not installed.
***
#### probe\_dependencies
```python theme={null}
def probe_dependencies() -> dict[str, Any]
```
Checks installed dependencies against the SDK's declared constraints.
Reads the version constraints the installed `bioptimus-sdk` distribution declares for its dependencies and verifies that each installed package satisfies them. Where [`probe_versions`](/sdk-reference/observability/probes#probe_versions) reports the raw installed numbers, this probe reports the verdict: upgrading a dependency past a supported ceiling (for example `zarr<3.0.0` or `torch<2.10.0`, which the `.pt2` artifacts are compiled against) is a common, silent cause of breakage that a bare version list does not reveal.
**Returns**:
A status dict with `distribution`, `satisfied` (`True` when every
applicable constraint holds), `conflicts` (a `{"name", "installed",
"required"}` entry per violation), and `checked` (the number of
constraints evaluated), or `{"error": ...}` when the constraints
cannot be evaluated.
***
#### probe\_host
```python theme={null}
def probe_host() -> dict[str, Any]
```
Probes host platform and resource information.
Reports the Python and OS platform, the interpreter running the SDK (a common "installed it but it is not found" culprit when it is the wrong one), CPU count and affinity, total physical RAM, any container memory limit, the open-file-descriptor limit (which concurrent tile reads can exhaust), and system load average. Each resource field degrades to `None` when it is unavailable on the host (for example load average on non-Unix platforms, or a container limit outside a container).
**Returns**:
A mapping with `python_version`, `platform`, `machine`,
`executable`, `in_venv`, `cpu_count`, `cpu_affinity`,
`total_ram_gb`, `container_memory_limit_gb`, `open_file_limit`,
and `load_avg`.
***
#### probe\_env
```python theme={null}
def probe_env() -> dict[str, str | None]
```
Probes a fixed allowlist of inference-relevant environment variables.
Only the allowlisted, non-secret names in `_TRACKED_ENV_VARS` are read; the full environment is never captured, so credentials and tokens are never exposed.
**Returns**:
Mapping of environment-variable name to its value, or `None` when the
variable is unset.
***
#### probe\_gpu
```python theme={null}
def probe_gpu() -> dict[str, Any]
```
Probes GPU, CUDA, and PyTorch runtime state.
Reports the PyTorch and CUDA build versions, the NVIDIA driver version, the CUDA architectures PyTorch was compiled for, and whether CUDA is available, plus per-device properties (name, compute capability, whether PyTorch's compiled kernels cover it, total and free memory). The driver version is captured even when CUDA is unavailable, since a driver too old for the CUDA build is a leading cause of `cuda_available` being false. A device whose compute capability is missing from PyTorch's build (for example after a wheel reinstall) loads but then fails at kernel launch, so it is flagged here. When CUDA is available it also verifies the context is responsive with a small device-to-host copy, since a present GPU with a corrupted context still fails inference.
**Returns**:
A mapping describing the GPU and runtime state. It always includes
`cuda_available`, and includes `reason` when PyTorch is missing or
CUDA cannot be queried.
***
#### probe\_pt2\_compat
```python theme={null}
def probe_pt2_compat(
checkpoints: Mapping[str, str | Path] | None = None
) -> list[dict[str, Any]]
```
Checks exported `.pt2` checkpoints against the host architecture.
The AOTInductor `.pt2` artifacts are compiled for a specific GPU (and CPU) architecture; running an artifact compiled for a different architecture fails at load or execution time. This probe reports, per checkpoint, the architecture it was compiled for and whether it matches the current host.
Mapping of model name to `.pt2` checkpoint path. Omitted or empty yields an empty list.
**Returns**:
A list of per-checkpoint status dicts, each with `name`,
`checkpoint`, `compiled_for`, `cpu_arch`, `compatible`, and
`reason` (the mismatch message, or `None` when compatible).
***
#### snapshot
```python theme={null}
def snapshot(
models: Mapping[str, object | None] | None = None,
checkpoints: Mapping[str, str | Path] | None = None) -> dict[str, Any]
```
Assembles a full diagnostic snapshot of the runtime environment.
Combines every probe into one JSON-friendly mapping. Never raises: each section is captured independently, so a failing probe degrades to an `{"error": ...}` section instead of breaking the snapshot.
Optional mapping of model name to loaded model object (or `None` when it failed to load), forwarded to [`probe_models`](/sdk-reference/observability/probes#probe_models).
Optional mapping of model name to `.pt2` checkpoint path, forwarded to [`probe_pt2_compat`](/sdk-reference/observability/probes#probe_pt2_compat).
**Returns**:
A mapping stamped with `schema_version` and `captured_at` (UTC
ISO-8601), plus the `versions`, `dependencies`, `host`, `env`,
`gpu`, `wsi_backends`, `disk`, `pt2_compat`, and `models`
sections.
# bioptimus.preprocess.omics
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics
Omics preprocessing transforms and pipelines.
# bioptimus.preprocess.omics.base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/base
Base classes for omics preprocessing.
## OmicsPreprocessor
```python theme={null}
class OmicsPreprocessor(ABC)
```
Base class for omics array transformations.
Each subclass implements [`process`](/sdk-reference/preprocess/omics/base#process) (forward) and optionally [`reverse`](/sdk-reference/preprocess/omics/base#reverse) (inverse). Calling the instance delegates to [`process`](/sdk-reference/preprocess/omics/base#process).
***
#### process
```python theme={null}
@abstractmethod
def process(matrix: np.ndarray) -> np.ndarray
```
Applies the transform (forward direction).
Input array to transform.
**Returns**:
Transformed array.
***
#### reverse
```python theme={null}
def reverse(matrix: np.ndarray) -> np.ndarray
```
Undo the transform (inverse direction).
Raises `NotImplementedError` for irreversible transforms.
Transformed array to invert.
**Returns**:
Array in the original space.
## OmicsPipeline
```python theme={null}
class OmicsPipeline(steps: list[OmicsPreprocessor])
```
Chains multiple [`OmicsPreprocessor`](/sdk-reference/preprocess/omics/base#omicspreprocessor) steps sequentially.
Forward: input → step\[0] → step\[1] → … → output Reverse: output → step\[-1].reverse → … → step\[0].reverse → input
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Applies all pipeline steps in sequence.
Input array to transform.
**Returns**:
Array after applying all steps.
***
#### reverse
```python theme={null}
def reverse(matrix: np.ndarray) -> np.ndarray
```
Applies all pipeline steps in reverse order.
Transformed array to invert.
**Returns**:
Array in the original space.
# bioptimus.preprocess.omics.factory
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/factory
Factory for constructing omics transforms by name.
***
#### create\_transform
```python theme={null}
def create_transform(name: str, **kwargs: Any) -> OmicsPreprocessor
```
Creates a single omics transform by name.
Transform identifier (case-insensitive). One of `"log1p"`, `"expm1"`, `"tpm"`, `"zscore"`, `"subset_columns"`.
Arguments forwarded to the transform constructor. For example `zscore` requires `mean` and `std`; `tpm` accepts an optional `factor`.
**Returns**:
An instantiated [`OmicsPreprocessor`](/sdk-reference/preprocess/omics/base#omicspreprocessor).
**Raises**:
* `BioptimusValueError` - With code [`ErrorCode.INVALID_CONFIG`](/sdk-reference/observability/errors#invalid_config) if *name* is not in the registry, or if *kwargs* do not match the transform's constructor signature (e.g. required arguments are missing or unexpected ones are given). It subclasses `ValueError`, so `except ValueError` still catches it.
***
#### create\_pipeline
```python theme={null}
def create_pipeline(configs: list[dict[str, Any]]) -> OmicsPipeline
```
Builds an [`OmicsPipeline`](/sdk-reference/preprocess/omics/base#omicspipeline) from a list of step configs.
Each config dict must contain a `"name"` key; remaining entries are passed as keyword arguments to the constructor.
Ordered list of transform specifications, e.g.:
```python theme={null}
[
{"name": "tpm"},
{"name": "log1p"},
{"name": "zscore", "mean": [...], "std": [...]},
]
```
**Returns**:
A pipeline that chains the transforms in order. An empty
*configs* list yields an empty (identity) pipeline.
**Raises**:
* `BioptimusValueError` - With code [`ErrorCode.INVALID_CONFIG`](/sdk-reference/observability/errors#invalid_config) if a step config is missing the `"name"` key, or if a step's name is unknown or its arguments are invalid (see [`create_transform`](/sdk-reference/preprocess/omics/factory#create_transform)). It subclasses `ValueError`.
# bioptimus.preprocess.omics.transforms
Source: https://docs.bioptimus.com/sdk-reference/preprocess/omics/transforms
Concrete omics preprocessing transforms.
## Log1pTransform
```python theme={null}
class Log1pTransform(OmicsPreprocessor)
```
Element-wise `log(1 + x)` ↔ `expm1(x)`.
Entries equal to `NON_MEASURED_GENE` (the missing-gene sentinel) are preserved verbatim rather than transformed, since `log1p(-1) = -inf`. The model is trained to receive the sentinel unchanged in its input space, so it must survive the transform.
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Applies element-wise `log1p`, preserving the missing-gene sentinel.
Entries equal to `NON_MEASURED_GENE` are left unchanged; all other entries are `log1p`-transformed.
Input counts array.
**Returns**:
Log1p-transformed array with sentinels preserved.
***
#### reverse
```python theme={null}
def reverse(matrix: np.ndarray) -> np.ndarray
```
Applies element-wise `expm1` (inverse of log1p).
Entries equal to `NON_MEASURED_GENE` are preserved unchanged so the transform remains a true inverse of [`process`](/sdk-reference/preprocess/omics/transforms#process).
Log1p-transformed array.
**Returns**:
Array in original counts space.
## Expm1Transform
```python theme={null}
class Expm1Transform(OmicsPreprocessor)
```
Element-wise `expm1(x)` ↔ `log1p(x)`.
Inverse of [`Log1pTransform`](/sdk-reference/preprocess/omics/transforms#log1ptransform). Useful as a standalone post-processor to convert log1p-space model output back to raw counts.
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Applies element-wise `expm1`.
Log-space array.
**Returns**:
Array in counts space.
***
#### reverse
```python theme={null}
def reverse(matrix: np.ndarray) -> np.ndarray
```
Applies element-wise `log1p` (inverse of expm1).
Counts-space array.
**Returns**:
Log1p-transformed array.
## TPMNormaliser
```python theme={null}
class TPMNormaliser(factor: float = 1000000.0)
```
Transcripts Per Million normalisation (row-wise).
Forward: counts → (counts / row\_sum) \* 1e6 Not reversible without storing original totals.
Entries equal to `NON_MEASURED_GENE` (the missing-gene sentinel) are excluded from the row total and preserved verbatim in the output, so a partially or fully unmeasured sample is not scaled by a meaningless negative sum and the sentinel survives for downstream transforms.
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Normalises each row to sum to *factor* (default 1e6).
All-zero rows remain all zeros (division is skipped). Entries equal to `NON_MEASURED_GENE` are excluded from the row sum and left unchanged in the output.
Raw counts array.
**Returns**:
TPM-normalised array with missing-gene sentinels preserved.
## ZScoreNormaliser
```python theme={null}
class ZScoreNormaliser(mean: np.ndarray, std: np.ndarray, eps: float = 1e-08)
```
Gene-wise z-score normalisation.
Forward: `(x - mean) / std` Reverse: `x * std + mean`
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Applies gene-wise z-score normalisation.
Raw expression array.
**Returns**:
Z-scored array.
***
#### reverse
```python theme={null}
def reverse(matrix: np.ndarray) -> np.ndarray
```
Reverses z-score normalisation to recover original scale.
Z-scored array.
**Returns**:
Array in original expression scale.
## SubsetColumns
```python theme={null}
class SubsetColumns(indices: list[int])
```
Selects and reorders columns by index.
Not reversible (information is discarded).
***
#### process
```python theme={null}
def process(matrix: np.ndarray) -> np.ndarray
```
Selects columns at the stored indices.
Input array with columns to subset.
**Returns**:
Array with only the selected columns.
**Raises**:
* `ValueError` - If any index is out of range for the matrix's last-axis size.
# bioptimus.preprocess.wsi
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi
WSI preprocessing utilities.
Provides tissue-mask models and providers for per-slide mask generation.
# bioptimus.preprocess.wsi.models.base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/models/base
Abstract interface for tissue segmentation models.
Defines the [`TissueMaskModel`](/sdk-reference/preprocess/wsi/models/base#tissuemaskmodel) abstract base class used by tissue mask providers to process individual tile requests into binary segmentation masks.
## TissueMaskModel
```python theme={null}
class TissueMaskModel(ABC)
```
Abstract interface for tissue segmentation models.
Accepts a **batch** of fixed-size RGB tiles and returns a batch of binary masks. GPU-backed models can process the whole batch at once; CPU-only models (like Otsu) can iterate internally.
The batch dimension is always the first axis:
* Input: `(B, H, W, 3)` — `uint8` RGB.
* Output: `(B, H, W)` — `uint8`, values in `{0, 1}`.
**Example**:
```python theme={null}
class MyGPUModel(TissueMaskModel):
def __init__(self, net):
self.net = net
def process(self, batch):
tensor = torch.from_numpy(batch).permute(0, 3, 1, 2).float() / 255
with torch.no_grad():
logits = self.net(tensor.cuda())
return (logits.squeeze(1).cpu().numpy() > 0.5).astype(np.uint8)
```
***
#### tile\_spec
```python theme={null}
@property
@abstractmethod
def tile_spec() -> TileSpec | None
```
Returns the tile specification for this model, or `None`.
Subclasses must implement this property. Return `None` for models that operate on arbitrary tile sizes.
***
#### process
```python theme={null}
@abstractmethod
def process(request: ModelRequest) -> ModelResponse
```
Processes a single RGB tile into a binary mask.
Model request containing the RGB tile.
**Returns**:
* `ModelResponse` - Model response containing the binary mask.
# bioptimus.preprocess.wsi.models.bioptimus_mask
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/models/bioptimus_mask
Remote tissue segmentation mask model.
Wraps a tissue segmentation [`EndpointModel`](/sdk-reference/models/endpoint_model#endpointmodel) as a [`TissueMaskModel`](/sdk-reference/preprocess/wsi/models/base#tissuemaskmodel), forwarding each `ModelRequest` to the remote tissue segmentation endpoint.
## BioptimusTissueMaskModel
```python theme={null}
class BioptimusTissueMaskModel(backbone: EndpointModel)
```
Tissue mask model backed by a segmentation endpoint.
Forwards each [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) to the [`EndpointModel`](/sdk-reference/models/endpoint_model#endpointmodel) and returns the [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse).
An [`EndpointModel`](/sdk-reference/models/endpoint_model#endpointmodel) configured for tissue segmentation.
**Example**:
```python theme={null}
from bioptimus.models.backbones import Backbone
backbone = Backbone(
"tissue-seg", base_url="http://localhost:8080"
)
model = BioptimusTissueMaskModel(backbone)
response = model.process(request)
```
***
#### tile\_spec
```python theme={null}
@property
def tile_spec() -> TileSpec | None
```
Returns the tile spec from the backbone's model spec.
***
#### process
```python theme={null}
def process(request: ModelRequest) -> ModelResponse
```
Sends a request to the remote tissue segmentation endpoint.
A [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) containing the tile image and metadata.
**Returns**:
A [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) with the segmentation output.
***
#### process\_async
```python theme={null}
async def process_async(request: ModelRequest,
session: Any = None) -> ModelResponse
```
Sends a request asynchronously to the segmentation endpoint.
A [`ModelRequest`](/sdk-reference/inference/schemas#modelrequest) containing the tile image and metadata.
An `aiohttp.ClientSession` for connection pooling.
**Returns**:
A [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) with the segmentation output.
# bioptimus.preprocess.wsi.provider.base
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/base
Abstract interface for tissue mask providers.
Defines the [`TissueMaskProvider`](/sdk-reference/preprocess/wsi/provider/base#tissuemaskprovider) abstract base class that [`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) depends on for obtaining binary tissue masks.
## TissueMaskProvider
```python theme={null}
class TissueMaskProvider(ABC)
```
Abstract base class for tissue mask providers.
Every provider must implement [`generate`](/sdk-reference/preprocess/wsi/provider/base#generate), which receives an **open** [`WSIReader`](/sdk-reference/io/wsi/interface#wsireader) and returns a 2-D `uint8` binary mask where `1` = tissue.
[`TileExtractor`](/sdk-reference/extraction/wsi/tile_extraction#tileextractor) depends only on this interface, so any concrete provider — algorithmic, model-based, or pre-computed — can be plugged in.
**Example**:
```python theme={null}
class MyCustomMask(TissueMaskProvider):
def generate(self, reader):
return load_from_database(reader.path)
```
***
#### generate
```python theme={null}
@abstractmethod
def generate(reader: WSIReader, *, show_progress: bool = True) -> np.ndarray
```
Generates a binary tissue mask for the given slide.
An **open** WSI reader.
Whether to display a progress bar. Defaults to `True`.
**Returns**:
* `np.ndarray` - A `uint8` mask of shape `(H, W)` with values in `{0, 1}` where `1` = tissue.
# bioptimus.preprocess.wsi.provider.precomputed
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/precomputed
Pre-computed tissue mask provider.
Loads a pre-existing mask from a directory on disk, matched by slide name. Useful for clinician annotations or masks produced by an external pipeline.
## PrecomputedTissueMask
```python theme={null}
class PrecomputedTissueMask(mask_dir: Union[str, Path], suffix: str | None = None)
```
Loads a pre-existing mask from a directory, matched by slide name.
Clinicians or annotation pipelines often produce per-slide masks (e.g. tissue regions, tumour annotations) and store them alongside the slides. This provider looks up the mask file whose **stem** matches the slide name and loads it.
Supported formats: PNG, JPEG, TIFF, BMP (loaded via OpenCV) and `.npy` (loaded via `numpy.load`).
The loaded mask is binarised (non-zero → 1) and returned as `uint8 (H, W)`.
Directory containing mask files.
Expected file extension **including the dot** (e.g. `".png"`). When `None` (default), the provider searches for the first file in *mask\_dir* whose stem matches the slide name, trying all supported extensions.
**Raises**:
* `FileNotFoundError` - At [`generate`](/sdk-reference/preprocess/wsi/provider/precomputed#generate) time if no matching mask file is found for a slide.
**Example**:
```python theme={null}
from bioptimus.preprocess.wsi.provider.precomputed import (
PrecomputedTissueMask,
)
# Directory layout:
# /data/masks/slide_001.png
# /data/masks/slide_002.png
provider = PrecomputedTissueMask(mask_dir="/data/masks/")
mask = provider.generate(reader) # reader.path → …/slide_001.svs
# With explicit suffix:
provider = PrecomputedTissueMask(
mask_dir="/data/masks/", suffix=".tiff"
)
```
***
#### generate
```python theme={null}
def generate(reader: WSIReader, *, show_progress: bool = True) -> np.ndarray
```
Loads the pre-computed mask that matches the slide name.
Extracts the slide stem from `reader.path`, finds the corresponding file in `mask_dir`, loads and binarises it.
An **open** WSI reader.
Ignored for pre-computed masks (kept for interface compatibility).
**Returns**:
* `np.ndarray` - A `uint8` mask of shape `(H, W)` with values in `{0, 1}` where `1` = tissue / region of interest.
**Raises**:
* `FileNotFoundError` - If no mask file is found for the slide.
# bioptimus.preprocess.wsi.provider.tiled
Source: https://docs.bioptimus.com/sdk-reference/preprocess/wsi/provider/tiled
Tiled tissue mask generation using a batched segmentation model.
Implements [`TiledTissueMask`](/sdk-reference/preprocess/wsi/provider/tiled#tiledtissuemask), which walks a WSI in fixed-size patches, processes them through a [`TissueMaskModel`](/sdk-reference/preprocess/wsi/models/base#tissuemaskmodel), and stitches the per-tile predictions into a full-slide binary mask.
## TiledTissueMask
```python theme={null}
class TiledTissueMask(model: TissueMaskModel)
```
Tile-based, batch-aware tissue mask generation.
Combines a [`TissueMaskModel`](/sdk-reference/preprocess/wsi/models/base#tissuemaskmodel) with a tiling strategy to produce a full-slide binary mask. The tile size and resolution are taken from the model's tile spec. The workflow (see [`generate`](/sdk-reference/preprocess/wsi/provider/tiled#generate)) is:
1. Compute the tissue bounding box at the model's resolution.
2. Walk the bounding box in non-overlapping, tile-spec-sized patches.
3. Read patches sequentially and dispatch them concurrently to the model
endpoint for GPU-efficient batched inference.
4. Stitch the per-tile masks back into the output array.
Boundary tiles smaller than the tile size are zero-padded before processing and the padding is stripped from the output.
The segmentation model. Its tile spec supplies the tile size and resolution used for tiling.
**Example**:
```python theme={null}
provider = TiledTissueMask(model=my_model)
mask = provider.generate(reader)
```
***
#### generate
```python theme={null}
def generate(reader: WSIReader,
*,
show_progress: bool = True,
max_concurrency: int = _DEFAULT_MAX_CONCURRENCY) -> np.ndarray
```
Generates a tissue mask by tiling over the WSI.
Tile images are read sequentially from the WSI (readers are not thread-safe), then dispatched concurrently to the remote model endpoint for GPU-efficient batched inference. Results are stitched back into a single binary mask.
An **open** WSI reader.
Whether to display a per-tile progress bar. Defaults to `True`.
Maximum concurrent HTTP requests. Defaults to 64.
**Returns**:
A `uint8` mask of shape `(H, W)` covering the tissue
bounding box at `resolution`, with values in
`{0, 1}` where `1` = tissue.
# bioptimus.runtime
Source: https://docs.bioptimus.com/sdk-reference/runtime
In-process model runtime.
Loads exported `.pt2` checkpoints and runs inference in the current process. This package was extracted from the FastAPI server (`api/`) so that both the server and the SDK in-process backend share a single implementation of model loading, preprocessing, GPU forward passes, and dynamic request batching.
Public surface:
* `BaseServerModel`, `PreparedSample`.
* `BatchScheduler`.
* `build_server_model`.
* `.pt2` utilities from `bioptimus.runtime.pt2`.
# bioptimus.runtime.batch_scheduler
Source: https://docs.bioptimus.com/sdk-reference/runtime/batch_scheduler
Dynamic request batching for GPU inference.
Implements a two-stage pipeline:
1. **Deserialization** (parallel, per-request): Each incoming request is decoded, transformed, and validated in a thread pool. Failures are returned immediately as per-request errors (HTTP 400) without entering the batch queue.
2. **Batching** (GPU, shared): Successfully prepared samples are accumulated into batches and dispatched to the GPU. A GPU-level failure (OOM, CUDA error) is shared by all requests in the batch and mapped to HTTP 503 via `_ForwardError`.
This separation guarantees that one user's malformed payload can never cause failures for other users' valid requests.
## BatchScheduler
```python theme={null}
class BatchScheduler(model: BaseServerModel,
max_batch_size: int = _DEFAULT_MAX_BATCH_SIZE,
max_wait_ms: float = _DEFAULT_MAX_WAIT_MS,
preprocess_workers: int = _DEFAULT_PREPROCESS_WORKERS,
mode: InferenceMode = InferenceMode.PREDICTION)
```
Two-stage pipeline: parallel deserialization then batched GPU inference.
Stage 1 — Deserialization (per-request, parallel): Each [`predict`](/sdk-reference/runtime/batch_scheduler#predict) call submits the request to a thread pool for image decoding, transform, shape validation, and (for M-Optimus) bulk RNA preprocessing. If any step fails, the request's future is immediately resolved with a `ValueError` (→ HTTP 400) and the request never enters the batch queue.
Stage 2 — Batching (shared GPU): Valid [`PreparedSample`](/sdk-reference/runtime/server_model#preparedsample) objects are collected into batches of up to *max\_batch\_size* and dispatched to the GPU. A forward failure raises `_ForwardError` (a `RuntimeError` subclass → HTTP 503) to all requests in that batch.
The server-side model to batch requests for.
Maximum tiles per GPU forward pass.
Maximum milliseconds to wait before dispatching a partial batch.
Thread pool size for parallel deserialization.
`"prediction"`, `"embedding"`, or `"prediction_with_embedding"`.
***
#### shutdown
```python theme={null}
def shutdown() -> None
```
Releases resources owned by the scheduler.
Shuts down the preprocessing thread pool (without waiting) and cancels the batch-loop task if it is still pending. Safe to call multiple times and from outside the scheduler's event loop (e.g. when pruning a scheduler whose loop has already closed).
***
#### predict
```python theme={null}
async def predict(request: ModelRequest) -> ModelResponse
```
Deserializes, validates, and enqueues a request for batched inference.
Deserialization runs in a thread pool so the event loop is not blocked. If deserialization fails, the error is raised directly to the caller (HTTP 400). Otherwise the prepared sample enters the batch queue and awaits GPU execution.
The tile request to process.
**Returns**:
The model response for this tile.
# bioptimus.runtime.loader
Source: https://docs.bioptimus.com/sdk-reference/runtime/loader
Factory for server-side model handlers.
Maps a model name to the correct handler class and loads its exported `.pt2` checkpoint. Models with custom forward logic (M-Optimus, tissue segmentation) use a dedicated subclass; all others load as a plain `BaseServerModel` with the spec resolved from the SDK YAML configs.
This factory is the single source of truth for the model registry, shared by the FastAPI server (`api`) and the SDK in-process backend (`InProcessClient`).
***
#### custom\_model\_classes
```python theme={null}
def custom_model_classes() -> dict[str, type[BaseServerModel]]
```
Returns a copy of the model-name to handler-class registry.
**Returns**:
Mapping of model name to its handler class.
***
#### build\_server\_model
```python theme={null}
def build_server_model(
model_name: str,
checkpoint: str | Path,
device: str = "cuda",
*,
assets_root: str | Path | None = None) -> BaseServerModel
```
Instantiates the model handler for an exported checkpoint.
Registry key (e.g. `"h1"`, `"m-optimus"`, `"tissue-seg"`).
Path to the `.pt2` checkpoint file. Its parent directory is used as the model root and its filename as the checkpoint name.
Torch device string (`"cuda"`, `"cuda:0"`, `"cpu"`).
Directory containing gene-set CSVs for M-Optimus. Ignored by models that do not require assets. Defaults to the baked-in container path when omitted.
**Returns**:
A ready-to-use [`BaseServerModel`](/sdk-reference/runtime/server_model#baseservermodel) (or subclass).
**Raises**:
* `FileNotFoundError` - If the checkpoint file does not exist.
* `KeyError` - If the model has no custom handler and no SDK config.
# bioptimus.runtime.models
Source: https://docs.bioptimus.com/sdk-reference/runtime/models
Server-side model classes that load exported .pt2 checkpoints.
# bioptimus.runtime.models.m_optimus
Source: https://docs.bioptimus.com/sdk-reference/runtime/models/m_optimus
M-Optimus prediction server-side model.
Loads the exported M-Optimus spatial transcriptomics model. Accepts optional bulk RNA counts alongside the tile image.
In the two-stage pipeline, bulk RNA preprocessing and gene count validation happen during `prepare_sample` (Stage 1), so a malformed `bulk_rna` field fails only the individual request without affecting the batch.
## MOptimusModel
```python theme={null}
class MOptimusModel(checkpoint_name: str,
device: str = 'cuda',
model_root: Path = MODEL_ROOT,
assets_root: Path = ASSETS_ROOT)
```
M-Optimus prediction model (224x224, MPP 0.5).
Produces per-tile gene expression predictions. Optionally incorporates bulk RNA counts when provided in the request.
Filename of the `.pt2` checkpoint.
Torch device string.
Directory containing the checkpoint.
Directory containing gene set CSVs.
***
#### input\_gene\_names
```python theme={null}
@property
def input_gene_names() -> List[str]
```
Ordered list of input (bulk RNA) Ensembl gene IDs.
***
#### output\_gene\_names
```python theme={null}
@property
def output_gene_names() -> List[str]
```
Ordered list of predicted output Ensembl gene IDs.
***
#### get\_metadata
```python theme={null}
def get_metadata() -> dict
```
Returns base metadata plus input and output gene sets.
**Returns**:
Dictionary with model info, gene sets, and I/O specs.
***
#### prepare\_sample
```python theme={null}
def prepare_sample(request: ModelRequest) -> PreparedSample
```
Deserializes tile image and validates bulk RNA per-sample.
Bulk RNA preprocessing and gene count validation happen here so that a single request with malformed RNA data fails with a per-request error (HTTP 400) and never enters the batch.
Raw model request.
**Returns**:
A [`PreparedSample`](/sdk-reference/runtime/server_model#preparedsample) with tile tensor and optional
bulk RNA tensor in `extra_tensors`.
**Raises**:
* `BioptimusValueError` - Tagged with `ErrorCode.RNA_SCHEMA_MISMATCH` if bulk RNA preprocessing fails or the gene count does not match the model panel. Subclasses `ValueError`, so it still maps to an HTTP 400 at the API boundary.
* `ValueError` - If the tile image is invalid (raised by the base class).
***
#### forward
```python theme={null}
def forward(requests: list[ModelRequest]) -> torch.Tensor
```
Runs M-Optimus inference with optional bulk RNA (legacy path).
List of tile requests with optional bulk RNA.
**Returns**:
Model output tensor of shape `(N, num_output_genes)`.
# bioptimus.runtime.models.tissue_seg
Source: https://docs.bioptimus.com/sdk-reference/runtime/models/tissue_seg
Tissue segmentation server-side model.
Loads the exported tissue segmentation checkpoint and produces per-pixel binary masks flattened to a 1-D output vector.
## TissueSegModel
```python theme={null}
class TissueSegModel(checkpoint_name: str,
device: str = 'cuda',
model_root: Path = MODEL_ROOT)
```
Tissue segmentation model (512x512, MPP 8.0).
Produces a flat `(H*W,)` binary mask output where each value is the probability of tissue at that pixel.
***
#### forward
```python theme={null}
def forward(requests: list[ModelRequest]) -> torch.Tensor
```
Runs segmentation returning flattened masks (legacy path).
List of tile requests.
**Returns**:
Flattened mask tensor of shape `(N, H*W)`.
# bioptimus.runtime.pt2
Source: https://docs.bioptimus.com/sdk-reference/runtime/pt2
Utilities for inspecting AOT-compiled .pt2 model packages.
Extracts target architecture information from the embedded ELF binaries (.cubin for CUDA, .so for CPU) without unpacking the full archive. Also reads model metadata (input/output shapes, dtypes) from the `model_metadata.json` embedded in the package.
## PackageArchInfo
```python theme={null}
@dataclass
class PackageArchInfo()
```
Architecture information extracted from a .pt2 package.
***
#### cuda\_compute\_capability
```python theme={null}
@property
def cuda_compute_capability() -> str | None
```
Returns the CUDA compute capability as 'X.Y' string.
***
#### cuda\_gpu\_description
```python theme={null}
@property
def cuda_gpu_description() -> str
```
Returns a human-readable description of the target GPU.
***
#### cpu\_description
```python theme={null}
@property
def cpu_description() -> str
```
Returns a human-readable description of the target CPU.
***
#### get\_package\_arch\_info
```python theme={null}
def get_package_arch_info(pt2_path: str | Path) -> PackageArchInfo
```
Extracts architecture info from a .pt2 package.
Reads only the ELF headers of the first .cubin and .so found in the archive. Does not unpack the full files.
Path to the .pt2 model package.
**Returns**:
Architecture information for the package.
**Raises**:
* `BioptimusRuntimeError` - If the package archive cannot be read.
***
#### check\_device\_compatibility
```python theme={null}
def check_device_compatibility(
pt2_path: str | Path,
device: torch.device | int | None = None) -> str | None
```
Checks if a .pt2 package is compatible with the target device.
Path to the .pt2 model package.
The CUDA device to validate against. Accepts a `torch.device`, an integer device index, or `None`. When `None` (or a bare `"cuda"` device with no index), the current CUDA device is used. This ensures the check inspects the GPU the worker was actually assigned rather than always device 0.
**Returns**:
None if compatible, or a descriptive error message if not.
## TensorSpec
```python theme={null}
@dataclass
class TensorSpec()
```
Specification for a single model input or output tensor.
**Attributes**:
* `name` - Name of the tensor (e.g. `"x"`, `"histology_tiles"`).
* `shape` - List of dimension sizes. Dynamic dimensions are represented as the string `_DYNAMIC`; fixed dimensions are integers.
* `dtype` - Expected dtype as a string (e.g. `"float32"`, `"int64"`).
* `description` - Human-readable description from the model metadata.
***
#### fixed\_shape
```python theme={null}
@property
def fixed_shape() -> dict[int, int]
```
Returns a mapping of axis index to fixed size for non-dynamic dims.
***
#### ndim
```python theme={null}
@property
def ndim() -> int
```
Returns the expected number of dimensions.
***
#### shape\_description
```python theme={null}
def shape_description() -> str
```
Returns a human-readable shape string like '(B, 3, 512, 512)'.
**Returns**:
Parenthesised comma-separated dimension description.
## PackageMetadata
```python theme={null}
@dataclass
class PackageMetadata()
```
Model metadata extracted from a .pt2 package.
**Attributes**:
* `model` - Model class name (e.g. `"EfficientUnet"`).
* `inputs` - Ordered list of input tensor specifications.
* `outputs` - Ordered list of output tensor specifications.
* `raw` - The full raw metadata dictionary.
***
#### load\_package\_metadata
```python theme={null}
def load_package_metadata(pt2_path: str | Path) -> PackageMetadata
```
Loads model metadata from a .pt2 package.
Reads the `model_metadata.json` file from the archive without extracting the full package.
Path to the .pt2 model package.
**Returns**:
Parsed metadata with input/output tensor specifications.
**Raises**:
* `BioptimusRuntimeError` - If the package archive or metadata JSON cannot be read.
***
#### validate\_input\_shapes
```python theme={null}
def validate_input_shapes(metadata: PackageMetadata,
tensors: dict[str, torch.Tensor]) -> str | None
```
Validates tensor shapes against the package metadata.
Checks that:
* All expected inputs are present.
* Each tensor has the correct number of dimensions.
* Fixed dimensions match exactly.
* Dynamic dimensions are consistent across inputs (same batch size).
The package metadata with input specifications.
Mapping of input name to tensor.
**Returns**:
None if all shapes are valid, or a descriptive error message.
# bioptimus.runtime.server_model
Source: https://docs.bioptimus.com/sdk-reference/runtime/server_model
Base class for server-side exported model loading and inference.
Server models load exported `.pt2` checkpoints and run inference. Weights are loaded from `/opt/ml/model` (SageMaker convention) via `torch._inductor.aoti_load_package`.
For models with no custom forward logic (e.g. embedding backbones), `BaseServerModel` is used directly. Models that need custom input handling (e.g. bulk RNA, segmentation masks) subclass it and override `prepare_sample` and `forward_batch`.
## Two-stage failure isolation
Stage 1 — Deserialization (per-request, parallel):
`prepare_sample` decodes, transforms, and validates each
request in a thread pool. Failures are returned immediately as
`ValueError` (HTTP 400) to the individual caller.
Stage 2 — GPU forward (shared batch):
`forward_batch` receives only validated tensors, runs the
model, synchronizes CUDA, and transfers results to CPU. Any
failure at this stage raises `_ForwardError`
(`RuntimeError` subclass → HTTP 503) shared by all
requests in the batch.
## PreparedSample
```python theme={null}
@dataclass
class PreparedSample()
```
A fully validated, preprocessed sample ready for GPU batching.
Created by [`BaseServerModel.prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample) during the per-request deserialization stage. Only samples that pass all validation enter the batch queue.
**Attributes**:
* `tile` - Preprocessed image tensor of shape `(C, H, W)`.
* `extra_tensors` - Additional model inputs (e.g. bulk RNA tensor) keyed by input name. Empty for standard embedding models.
## BaseServerModel
```python theme={null}
class BaseServerModel(model_spec: ModelSpec,
checkpoint_name: str,
device: str = 'cuda',
model_root: Path = MODEL_ROOT)
```
Loads an exported `.pt2` model and runs single-tile inference.
For embedding backbones (h0-mini, h0, h1, …) this class is used directly — no subclass needed. Override [`prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample) and [`forward_batch`](/sdk-reference/runtime/server_model#forward_batch) for models that require custom input handling (e.g. bulk RNA).
The model specification (tile size, normalization, etc.).
Filename of the `.pt2` checkpoint.
Torch device string (`"cuda"` or `"cpu"`).
Directory containing exported checkpoints.
***
#### model\_spec
```python theme={null}
@property
def model_spec() -> ModelSpec
```
The model's specification (tile size, normalization, etc.).
***
#### model\_name
```python theme={null}
@property
def model_name() -> str
```
Human-readable model identifier.
***
#### preprocess\_bulk\_rna
```python theme={null}
def preprocess_bulk_rna(raw: list[float]) -> list[float]
```
Applies the model-spec omics transform to raw counts.
Raw bulk RNA counts as a flat list of floats.
**Returns**:
Transformed counts. Returned unchanged when no
omics transform is configured.
***
#### prepare\_sample
```python theme={null}
def prepare_sample(request: ModelRequest) -> PreparedSample
```
Deserializes, transforms, and validates a single request.
Called in a thread pool by the batch scheduler. Any exception raised here is returned as a per-request error (HTTP 400) and the request never enters the batch queue.
Subclasses override this to add custom validation (e.g. bulk RNA preprocessing for models).
Raw model request with base64 image data.
**Returns**:
A [`PreparedSample`](/sdk-reference/runtime/server_model#preparedsample) with the preprocessed tile tensor.
**Raises**:
* `ValueError` - If the image cannot be decoded, transformed, or has the wrong shape.
***
#### forward\_batch
```python theme={null}
@torch.inference_mode()
def forward_batch(samples: list[PreparedSample],
requests: list[ModelRequest],
mode: str = "prediction") -> list[ModelResponse]
```
Runs batched GPU inference on pre-validated samples.
All samples have already passed [`prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample). The entire GPU execution path (forward + synchronize + CPU transfer) is wrapped in error handling so that CUDA errors cannot escape.
Pre-validated samples with preprocessed tensors.
Original requests (for response metadata).
`"prediction"`, `"embedding"`, or `"prediction_with_embedding"`. The last returns both the prediction and the embedding from the single forward pass.
**Returns**:
One [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) per sample, positionally aligned.
**Raises**:
* `_ForwardError` - If the GPU forward pass, CUDA synchronization, or CPU transfer fails.
***
#### get\_metadata
```python theme={null}
def get_metadata() -> dict[str, Any]
```
Returns model metadata.
**Returns**:
Dictionary with model name, input/output specs, and config.
***
#### forward
```python theme={null}
def forward(requests: list[ModelRequest]) -> torch.Tensor
```
Runs the model on a batch of tile requests (legacy path).
Used by subclasses with custom forward logic that haven't migrated to the two-stage pipeline yet.
List of tile requests.
**Returns**:
Model output tensor of shape `(N, D)`.
# bioptimus.utils
Source: https://docs.bioptimus.com/sdk-reference/utils
Bioptimus SDK utilities — helpers for data download, visualization, and output loading.
# bioptimus.utils.pytorch
Source: https://docs.bioptimus.com/sdk-reference/utils/pytorch
PyTorch adapter for Bioptimus endpoint models.
## TorchModel
```python theme={null}
class TorchModel(model_name: str | Models,
endpoint_name: str,
region_name: str | None = None,
mode: Literal[InferenceMode.EMBEDDING, InferenceMode.PREDICTION] = InferenceMode.EMBEDDING,
timeout: float = 60.0,
max_concurrency: int = 32)
```
Wraps a SageMaker backbone endpoint with a PyTorch `forward` API.
Pass a model name plus the AWS endpoint details and the wrapper builds the SageMaker client for you. On each `forward` call it serializes the incoming image batch into endpoint requests, calls the endpoint, and returns a stacked tensor of outputs. This keeps loops shaped like `features = model(images)` while the model runs behind a SageMaker endpoint.
Keep `ToTensor` in the dataloader but drop `Normalize` (and any other scaling/stain transforms): the endpoint applies the model's own preprocessing server-side, so tiles must be passed as un-normalized RGB. `forward` also accepts numpy arrays or PIL images if you prefer not to convert to tensors at all.
**Notes**:
This is a thin remote adapter, not a real local model. Inference runs
server-side behind the SageMaker endpoint, and the wrapper registers no
parameters or buffers. The usual `Module` controls are
therefore inert no-ops: `.to(device)`, `.cuda()`, `.half()`,
`.eval()`, `.train()`, and `.parameters()` do nothing meaningful,
and there is no local device or dtype to configure. `forward` always
returns a freshly built CPU `float32` tensor regardless of any device
the caller set. The `Module` base class exists only so
the wrapper drops into loops that expect a callable `model(images)`;
do not rely on device, dtype, or eval/train state to change where or how
the endpoint runs.
Model to build the endpoint for, as a [`Models`](/sdk-reference/models/types#models) member (e.g. `Models.M_OPTIMUS`) or its string name (e.g. `"m-optimus"`).
Name of the deployed SageMaker endpoint to invoke.
AWS region of the endpoint. Falls back to the default boto3 resolution (env/config) when omitted.
`"embedding"` to call [`EndpointModel.embed`](/sdk-reference/models/endpoint_model#embed) (tile features), or `"prediction"` to call [`EndpointModel.predict`](/sdk-reference/models/endpoint_model#predict) (e.g. M-Optimus gene expression).
Per-request timeout in seconds.
Maximum number of tiles sent to the endpoint concurrently per `forward` call. Each tile is one request, so this caps how many in-flight requests a single batch produces. Lower it for single-GPU endpoints that cannot absorb many concurrent requests.
**Example**:
```python theme={null}
from bioptimus.utils import TorchModel
from bioptimus.models.types import Models
# AWS SageMaker endpoint.
model = TorchModel(
model_name=Models.M_OPTIMUS,
endpoint_name="m-optimus",
region_name="us-east-1",
)
features = model(images) # (N, embedding_dim)
```
***
#### forward
```python theme={null}
def forward(images: ImageBatch) -> torch.Tensor
```
Runs a batch of raw tiles through the wrapped endpoint.
Inference runs server-side, so the result is always a CPU `float32` tensor regardless of any device set on the wrapper (see the class note on the inert `Module` controls).
Un-normalized tiles, passed as-is without transforms. May be a tensor batch shaped `(N, C, H, W)` or `(N, H, W, C)`, a numpy array, a single PIL image, or a list of PIL images / tensors / numpy arrays.
**Returns**:
CPU tensor shaped `(N, output_dim)` with the endpoint outputs.
# bioptimus.utils.utils
Source: https://docs.bioptimus.com/sdk-reference/utils/utils
Shared utilities for Bioptimus SDK tutorial notebooks.
Provides common helpers for data download, WSI inspection, output loading, and visualization — used by both the H-Optimus-1 and M-Optimus tutorials.
Documentation: [https://docs.bioptimus.com/guides/get-started/sdk](https://docs.bioptimus.com/guides/get-started/sdk)
***
#### download\_if\_needed
```python theme={null}
def download_if_needed(url: str, output_path: Path) -> None
```
Downloads a file from *url* if it does not already exist at *output\_path*.
URL to download from.
Local destination path.
***
#### check\_server
```python theme={null}
def check_server(api_url: str, *, timeout: float = 5.0) -> dict[str, Any]
```
Verifies that the inference server at *api\_url* is healthy.
Raises on connection error or non-200 status.
Base URL of the inference server (e.g. `http://localhost:8080`).
HTTP request timeout in seconds.
**Returns**:
The JSON body of `GET /ping`.
***
#### display\_slide\_info
```python theme={null}
def display_slide_info(
wsi_path: str | Path, thumbnail_size: tuple[int, int] = (512, 512)
) -> PIL.Image.Image
```
Prints slide metadata and returns a PIL thumbnail image.
Uses the SDK's WSI reader; see [https://docs.bioptimus.com/guides/reference/wsi-reader](https://docs.bioptimus.com/guides/reference/wsi-reader)
Path to the whole-slide image file.
Maximum `(width, height)` of the returned thumbnail.
**Returns**:
A PIL thumbnail of the slide.
***
#### display\_rna\_summary
```python theme={null}
def display_rna_summary(tsv_path: str | Path) -> pd.DataFrame
```
Prints summary statistics for a bulk RNA-seq TSV file.
Path to a tab-separated RNA-seq counts file.
**Returns**:
The loaded DataFrame for further inspection.
***
#### read\_gene\_mapping
```python theme={null}
def read_gene_mapping(path: str | Path,
symbol_column: str = "gene_name",
id_column: str = "gene_id",
*,
separator: str | None = None,
strip_version: bool = False) -> dict[str, str]
```
Reads a gene symbol → ID mapping from a bulk RNA file.
Parses a long-format bulk RNA file (one row per gene) and returns a dict mapping gene symbols to identifiers (typically Ensembl IDs). Useful for converting human-readable gene names into the Ensembl IDs used by M-Optimus prediction outputs.
Path to the CSV or TSV file.
Column containing gene symbols (e.g. `"gene_name"`).
Column containing gene identifiers (e.g. `"gene_id"`).
Column delimiter. Inferred from file extension when `None` (`.tsv` → tab, otherwise comma).
When `True`, strips version suffixes from the *id\_column* (e.g. `ENSG00000119888.12` → `ENSG00000119888`).
**Returns**:
A dict mapping each symbol to the corresponding identifier.
***
#### load\_zarr\_output
```python theme={null}
def load_zarr_output(path: str | Path) -> dict
```
Loads a Zarr inference output into a dictionary.
Path to the Zarr store directory.
**Returns**:
A dict with keys `outputs`, `coords`, `metadata`, and
optionally `gene_names` (M-Optimus predictions), `thumbnail`,
`tissue_mask`, and `tissue_ratios`.
See [https://docs.bioptimus.com/guides/get-started/visualizing-results](https://docs.bioptimus.com/guides/get-started/visualizing-results)
***
#### convert\_spot\_to\_coord
```python theme={null}
def convert_spot_to_coord(
tissue_positions: str | Path,
*,
native_mpp: float,
tile_px: int,
tile_mpp: float,
x_column: str = "pxl_col_in_fullres",
y_column: str = "pxl_row_in_fullres"
) -> tuple[list[tuple[int, int]], pd.DataFrame]
```
Converts spatial-transcriptomics spot centres to tile coordinates.
Turns a 10x-style `tissue_positions` table (spot **centres** in full-resolution pixels) into the `(x, y)` top-left tile positions that `Inference` expects via its `coordinates` argument. Two conversions are applied:
1. **Resolution scaling.** Spot pixels are at the slide's native
resolution (*native\_mpp*), but the model reads tiles at *tile\_mpp*, so centres are scaled by `native_mpp / tile_mpp`.
2. **Centre → top-left.** The model always reads a fixed
`tile_px x tile_px` window, so the top-left is the scaled centre minus **half the tile size** (`tile_px // 2`). Anchoring on the tile size (rather than the spot's own footprint) keeps the model's window centred on each spot regardless of the spot/bin size.
Path to a 10x `tissue_positions` CSV with spot centres in full-resolution pixels.
Microns-per-pixel of the slide's full resolution (read from the WSI reader, e.g. `reader.mpp`).
Tile edge in pixels the model reads (e.g. `224`).
Microns-per-pixel the model expects (e.g. `0.5`).
Column holding the spot centre x in full-resolution pixels.
Column holding the spot centre y in full-resolution pixels.
**Returns**:
A tuple `(coordinates, spots)` where *coordinates* is a list of
`(x, y)` top-left, slide-origin pixel pairs at the tile resolution
(ready to pass as `Inference(..., coordinates=...)`), and *spots* is
the loaded table augmented with integer `tile_x`/`tile_y` columns
(row-aligned with *coordinates*) for downstream benchmark alignment.
**Example**:
```python theme={null}
>>> coords, spots = convert_spot_to_coord(
... "tissue_positions.csv", native_mpp=0.2738, tile_px=224, tile_mpp=0.5
... )
>>> infer = Inference(model_name="m-optimus", coordinates=coords, ...)
```
***
#### load\_spot\_size\_um
```python theme={null}
def load_spot_size_um(spot_metadata: str | Path, *,
native_mpp: float) -> tuple[float, float]
```
Returns the spot footprint `(width, height)` in microns from metadata.
Reads a platform `metadata.json` (10x-style) and converts the spot footprint to microns. Informational only — the model reads a fixed tile window regardless of spot size — but useful to report the spot footprint against the model's field of view.
Path to a `metadata.json` with `tile_shape` (`{"width", "height"}`) or a scalar `spot_diameter` in full-resolution pixels.
Microns-per-pixel of the slide's full resolution.
**Returns**:
The spot `(width_um, height_um)`.
***
#### plot\_slide\_and\_mask
```python theme={null}
def plot_slide_and_mask(
wsi_path: str | Path,
mask_path: str | Path,
*,
threshold: float = 0.5,
figsize: tuple[int, int] = (14, 5)) -> None
```
Shows a slide thumbnail alongside its tissue mask.
Path to the whole-slide image file.
Path to the tissue mask image (PNG).
Binarization threshold for the mask.
Matplotlib figure size `(width, height)`.
***
#### plot\_pca\_scatter
```python theme={null}
def plot_pca_scatter(embeddings: np.ndarray,
*,
n_components: int = 2,
title: str = "Embedding PCA",
cmap: str = "viridis",
figsize: tuple[int, int] = (8, 6),
point_size: float = 10) -> tuple[np.ndarray, Any]
```
Fits PCA on *embeddings* and plots the first two components.
Tile embeddings array of shape `(n_tiles, dim)`.
Number of PCA components to compute.
Plot title.
Matplotlib colormap name.
Figure size `(width, height)`.
Scatter point size.
**Returns**:
A `(scores, pca)` tuple — the transformed array and the fitted
PCA object.
***
#### plot\_spatial\_pca
```python theme={null}
def plot_spatial_pca(
scores: np.ndarray,
coords: np.ndarray,
*,
components: list[int] | None = None,
title_prefix: str = "",
figsize: tuple[int, int] = (18, 5)) -> None
```
Maps PCA scores back to tile coordinates as spatial heatmaps.
*components* selects which PCs to plot (0-indexed); defaults to the first 3 (or 2 if fewer are available).
PCA-transformed array of shape `(n_tiles, n_components)`.
Tile coordinates of shape `(n_tiles, 2)`.
Indices of PCs to plot (0-indexed).
Prefix prepended to each subplot title.
Figure size `(width, height)`.
***
#### plot\_nearest\_neighbor\_tiles
```python theme={null}
def plot_nearest_neighbor_tiles(embeddings: np.ndarray,
coords: np.ndarray,
wsi_path: str | Path,
*,
query_indices: list[int] | None = None,
n_queries: int = 4,
n_neighbors: int = 5,
tile_size: int = 224,
mpp: float = 0.5,
seed: int = 42,
figsize_per_tile: float = 1.8) -> None
```
Shows that tiles with similar embeddings look visually similar.
For each query tile, finds its nearest neighbors in embedding space and displays them side-by-side. This is a visual proof that the model captures meaningful morphology.
Tile embeddings array of shape `(n_tiles, dim)`.
Tile coordinates array of shape `(n_tiles, 2)`.
Path to the WSI file for reading tile images.
Specific tile indices to use as queries. If *None*, `n_queries` tiles are sampled from distinct PCA regions.
Number of query tiles (used when *query\_indices* is None).
Number of nearest neighbors to show per query.
Tile width/height in pixels at the target resolution.
Microns per pixel for tile extraction.
Random seed for reproducible query selection.
Figure size per tile in inches.
***
#### plot\_gene\_overlay
```python theme={null}
def plot_gene_overlay(
values: np.ndarray,
coords: np.ndarray,
wsi_path: str | Path,
*,
gene_name: str = "",
title: str = "",
cmap: str = "inferno",
alpha: float = 0.55,
vmin: float | None = 0.0,
vmax: float | None = None,
ax: plt.Axes | None = None,
figsize: tuple[int, int] = (8, 7),
thumbnail_size: tuple[int, int] = (1024, 1024)
) -> None
```
Overlays a gene-expression heatmap on the slide thumbnail.
Per-tile expression values of shape `(n_tiles,)`.
Tile coordinates of shape `(n_tiles, 2)`.
Path to the whole-slide image file.
Gene symbol for the title.
Custom plot title (overrides *gene\_name*).
Matplotlib colormap name.
Heatmap overlay opacity.
Colormap minimum.
Colormap maximum.
Existing axes to draw on. Creates a new figure when `None`.
Figure size when creating a new figure.
Maximum `(width, height)` of the slide thumbnail.
***
#### plot\_gene\_overlay\_comparison
```python theme={null}
def plot_gene_overlay_comparison(
preds_a: np.ndarray,
coords_a: np.ndarray,
preds_b: np.ndarray,
coords_b: np.ndarray,
gene_idx: int,
wsi_path: str | Path,
*,
gene_name: str = "",
label_a: str = "Condition A",
label_b: str = "Condition B",
slide_id: str = "",
cmap: str = "inferno",
alpha: float = 0.55,
figsize: tuple[int, int] = (18, 8),
thumbnail_size: tuple[int, int] = (1024, 1024)
) -> None
```
Side-by-side gene-expression overlays on the slide thumbnail.
Predictions array for condition A, shape `(n_tiles, n_genes)`.
Tile coordinates for condition A.
Predictions array for condition B.
Tile coordinates for condition B.
Column index of the gene to plot.
Path to the whole-slide image file.
Gene symbol for the title.
Display label for condition A.
Display label for condition B.
Slide identifier shown in the title.
Matplotlib colormap name.
Heatmap overlay opacity.
Figure size `(width, height)`.
Maximum `(width, height)` of the slide thumbnail.
***
#### plot\_gene\_panel\_overlay
```python theme={null}
def plot_gene_panel_overlay(
predictions: np.ndarray,
coords: np.ndarray,
gene_names: list[str],
genes_to_plot: dict[str, str],
wsi_path: str | Path,
*,
cols: int = 3,
figsize_per_plot: tuple[int, int] = (6, 5),
cmap: str = "inferno",
alpha: float = 0.55,
thumbnail_size: tuple[int, int] = (1024, 1024)) -> None
```
Overlays spatial heatmaps for multiple genes on the slide thumbnail.
Predictions array of shape `(n_tiles, n_genes)`.
Tile coordinates of shape `(n_tiles, 2)`.
List of gene identifiers matching prediction columns.
Mapping of gene symbol → identifier to visualize.
Path to the whole-slide image file.
Number of columns in the subplot grid.
Size `(width, height)` per subplot.
Matplotlib colormap name.
Heatmap overlay opacity.
Maximum `(width, height)` of the slide thumbnail.
***
#### plot\_top\_gene\_tiles
```python theme={null}
def plot_top_gene_tiles(predictions: np.ndarray,
coords: np.ndarray,
gene_idx: int,
wsi_path: str | Path,
*,
gene_name: str = "",
n_top: int = 5,
n_bottom: int = 5,
tile_size: int = 224,
mpp: float = 0.5,
figsize_per_tile: float = 2.0) -> None
```
Shows tiles with the highest and lowest predicted expression for a gene.
Displays two rows: top-N highest-expression tiles and top-N lowest-expression tiles, with expression values as labels.
Predictions array of shape `(n_tiles, n_genes)`.
Tile coordinates of shape `(n_tiles, 2)`.
Column index of the gene to plot.
Path to the whole-slide image file.
Gene symbol for the plot title.
Number of highest-expression tiles to show.
Number of lowest-expression tiles to show.
Tile width/height in pixels.
Microns per pixel for tile extraction.
Figure size per tile in inches.
***
#### plot\_spatial\_gene
```python theme={null}
def plot_spatial_gene(
values: np.ndarray,
coords: np.ndarray,
*,
gene_name: str = "",
title: str = "",
cmap: str = "inferno",
vmin: float | None = None,
vmax: float | None = None,
ax: plt.Axes | None = None,
figsize: tuple[int, int] = (8, 7)) -> None
```
Plots a spatial gene-expression heatmap at tile coordinates.
Per-tile expression values of shape `(n_tiles,)`.
Tile coordinates of shape `(n_tiles, 2)`.
Gene symbol for the title.
Custom plot title (overrides *gene\_name*).
Matplotlib colormap name.
Colormap minimum.
Colormap maximum.
Existing axes to draw on. Creates a new figure when `None`.
Figure size when creating a new figure.
***
#### plot\_gene\_comparison
```python theme={null}
def plot_gene_comparison(
preds_a: np.ndarray,
coords_a: np.ndarray,
preds_b: np.ndarray,
coords_b: np.ndarray,
gene_idx: int,
*,
gene_name: str = "",
label_a: str = "Condition A",
label_b: str = "Condition B",
slide_id: str = "",
cmap: str = "inferno",
figsize: tuple[int, int] = (18, 8)) -> None
```
Side-by-side spatial heatmaps comparing a gene under two conditions.
Predictions array for condition A, shape `(n_tiles, n_genes)`.
Tile coordinates for condition A.
Predictions array for condition B.
Tile coordinates for condition B.
Column index of the gene to plot.
Gene symbol for the title.
Display label for condition A.
Display label for condition B.
Slide identifier shown in the title.
Matplotlib colormap name.
Figure size `(width, height)`.
***
#### plot\_gene\_panel
```python theme={null}
def plot_gene_panel(predictions: np.ndarray,
coords: np.ndarray,
gene_names: list[str],
genes_to_plot: dict[str, str],
*,
cols: int = 3,
figsize_per_plot: tuple[int, int] = (6, 5),
cmap: str = "inferno") -> None
```
Plots spatial heatmaps for multiple genes in a grid.
*genes\_to\_plot* maps gene symbol → Ensembl ID (or name in gene\_names).
Predictions array of shape `(n_tiles, n_genes)`.
Tile coordinates of shape `(n_tiles, 2)`.
List of gene identifiers matching prediction columns.
Mapping of gene symbol → identifier to visualize.
Number of columns in the subplot grid.
Size `(width, height)` per subplot.
Matplotlib colormap name.
***
#### print\_config
```python theme={null}
def print_config(**kwargs: object) -> None
```
Pretty-prints experiment configuration as an aligned key-value table.