> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bioptimus.com/llms.txt
> Use this file to discover all available pages before exploring further.

# bioptimus.runtime.server_model

Base class for server-side exported model loading and inference.

Server models load exported `.pt2` checkpoints and run inference. Weights are loaded from `/opt/ml/model` (SageMaker convention) via `torch._inductor.aoti_load_package`.

For models with no custom forward logic (e.g. embedding backbones), `BaseServerModel` is used directly. Models that need custom input handling (e.g. bulk RNA, segmentation masks) subclass it and override `prepare_sample` and `forward_batch`.

## Two-stage failure isolation

Stage 1 — Deserialization (per-request, parallel):
`prepare_sample` decodes, transforms, and validates each
request in a thread pool.  Failures are returned immediately as
`ValueError` (HTTP 400) to the individual caller.

Stage 2 — GPU forward (shared batch):
`forward_batch` receives only validated tensors, runs the
model, synchronizes CUDA, and transfers results to CPU.  Any
failure at this stage raises `_ForwardError`
(`RuntimeError` subclass → HTTP 503) shared by all
requests in the batch.

## PreparedSample

```python theme={null}
@dataclass
class PreparedSample()
```

A fully validated, preprocessed sample ready for GPU batching.

Created by [`BaseServerModel.prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample) during the per-request deserialization stage.  Only samples that pass all validation enter the batch queue.

**Attributes**:

* `tile` - Preprocessed image tensor of shape `(C, H, W)`.
* `extra_tensors` - Additional model inputs (e.g. bulk RNA tensor) keyed by input name.  Empty for standard embedding models.

## BaseServerModel

```python theme={null}
class BaseServerModel(model_spec: ModelSpec,
                      checkpoint_name: str,
                      device: str = 'cuda',
                      model_root: Path = MODEL_ROOT)
```

Loads an exported `.pt2` model and runs single-tile inference.

For embedding backbones (h0-mini, h0, h1, …) this class is used directly — no subclass needed. Override [`prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample) and [`forward_batch`](/sdk-reference/runtime/server_model#forward_batch) for models that require custom input handling (e.g. bulk RNA).

<ParamField body="model_spec">
  The model specification (tile size, normalization, etc.).
</ParamField>

<ParamField body="checkpoint_name">
  Filename of the `.pt2` checkpoint.
</ParamField>

<ParamField body="device">
  Torch device string (`"cuda"` or `"cpu"`).
</ParamField>

<ParamField body="model_root">
  Directory containing exported checkpoints.
</ParamField>

***

#### model\_spec

```python theme={null}
@property
def model_spec() -> ModelSpec
```

The model's specification (tile size, normalization, etc.).

***

#### model\_name

```python theme={null}
@property
def model_name() -> str
```

Human-readable model identifier.

***

#### preprocess\_bulk\_rna

```python theme={null}
def preprocess_bulk_rna(raw: list[float]) -> list[float]
```

Applies the model-spec omics transform to raw counts.

<ParamField body="raw">
  Raw bulk RNA counts as a flat list of floats.
</ParamField>

**Returns**:

Transformed counts. Returned unchanged when no
omics transform is configured.

***

#### prepare\_sample

```python theme={null}
def prepare_sample(request: ModelRequest) -> PreparedSample
```

Deserializes, transforms, and validates a single request.

Called in a thread pool by the batch scheduler.  Any exception raised here is returned as a per-request error (HTTP 400) and the request never enters the batch queue.

Subclasses override this to add custom validation (e.g. bulk RNA preprocessing for  models).

<ParamField body="request">
  Raw model request with base64 image data.
</ParamField>

**Returns**:

A [`PreparedSample`](/sdk-reference/runtime/server_model#preparedsample) with the preprocessed tile tensor.

**Raises**:

* `ValueError` - If the image cannot be decoded, transformed, or has the wrong shape.

***

#### forward\_batch

```python theme={null}
@torch.inference_mode()
def forward_batch(samples: list[PreparedSample],
                  requests: list[ModelRequest],
                  mode: str = "prediction") -> list[ModelResponse]
```

Runs batched GPU inference on pre-validated samples.

All samples have already passed [`prepare_sample`](/sdk-reference/runtime/server_model#prepare_sample).  The entire GPU execution path (forward + synchronize + CPU transfer) is wrapped in error handling so that CUDA errors cannot escape.

<ParamField body="samples">
  Pre-validated samples with preprocessed tensors.
</ParamField>

<ParamField body="requests">
  Original requests (for response metadata).
</ParamField>

<ParamField body="mode">
  `"prediction"`, `"embedding"`, or `"prediction_with_embedding"`. The last returns both the prediction and the embedding from the single forward pass.
</ParamField>

**Returns**:

One [`ModelResponse`](/sdk-reference/inference/schemas#modelresponse) per sample, positionally aligned.

**Raises**:

* `_ForwardError` - If the GPU forward pass, CUDA synchronization, or CPU transfer fails.

***

#### get\_metadata

```python theme={null}
def get_metadata() -> dict[str, Any]
```

Returns model metadata.

**Returns**:

Dictionary with model name, input/output specs, and config.

***

#### forward

```python theme={null}
def forward(requests: list[ModelRequest]) -> torch.Tensor
```

Runs the model on a batch of tile requests (legacy path).

Used by subclasses with custom forward logic that haven't migrated to the two-stage pipeline yet.

<ParamField body="requests">
  List of tile requests.
</ParamField>

**Returns**:

Model output tensor of shape `(N, D)`.