> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bioptimus.com/llms.txt
> Use this file to discover all available pages before exploring further.

# bioptimus.observability.probes

Fail-safe environment probes for diagnostic snapshots.

Captures a point-in-time picture of the runtime environment that matters for inference and reports it as a plain, JSON-friendly mapping that feeds both the support bundle (`diagnostics.json`) and the server's `/diagnostics` endpoint. The snapshot covers:

* installed SDK and dependency versions ([`probe_versions`](/sdk-reference/observability/probes#probe_versions));
* installed dependencies versus the SDK's declared constraints ([`probe_dependencies`](/sdk-reference/observability/probes#probe_dependencies));
* host platform and resources ([`probe_host`](/sdk-reference/observability/probes#probe_host));
* inference-relevant environment variables ([`probe_env`](/sdk-reference/observability/probes#probe_env));
* GPU, CUDA, and PyTorch runtime state ([`probe_gpu`](/sdk-reference/observability/probes#probe_gpu));
* importable whole-slide-image backends ([`probe_wsi_backends`](/sdk-reference/observability/probes#probe_wsi_backends));
* temporary-directory headroom and leaked AOTInductor extractions ([`probe_disk`](/sdk-reference/observability/probes#probe_disk));
* `.pt2` checkpoint architecture compatibility ([`probe_pt2_compat`](/sdk-reference/observability/probes#probe_pt2_compat)); and
* loaded models ([`probe_models`](/sdk-reference/observability/probes#probe_models)).

The assembled [`snapshot`](/sdk-reference/observability/probes#snapshot) is stamped with a schema version and a capture timestamp so it can be parsed and correlated with the logs over time.

Every probe is fail-safe: it never raises. Any failure is captured in the returned data (for example `{"available": false, "reason": ...}` or an `"error"` key) so a broken environment still yields a usable snapshot.

***

#### probe\_wsi\_backends

```python theme={null}
def probe_wsi_backends() -> dict[str, dict[str, Any]]
```

Probes whether each supported WSI backend can be imported.

Attempts to import each backend library, recording its version when the import succeeds and the failure reason (for example a missing native library) when it does not.

**Returns**:

Mapping of backend name to a status dict: `available` is `True` with
a `version` when the backend imports, otherwise `False` with a
`reason`.

***

#### probe\_disk

```python theme={null}
def probe_disk() -> dict[str, Any]
```

Probes free space and AOTInductor temp-dir usage in the system tempdir.

Reports the temporary directory used for `.pt2` extraction, its free space and percentage used, whether it is writable, and how many leftover AOTInductor extraction directories it contains (a count that grows when packages are not cleaned up). Sets `warning` to `"TMP_NOT_WRITABLE"` when the directory cannot be written (which breaks `.pt2` extraction) or `"TMP_NEARLY_FULL"` when usage is high.

**Returns**:

A status dict with `tmp_path`, `tmp_free_gb`, `tmp_pct_used`,
`aotinductor_temp_dirs`, `writable`, and `warning` (`None` when
healthy), or `{"error": ...}` if the directory cannot be inspected.

***

#### probe\_models

```python theme={null}
def probe_models(
        models: Mapping[str, object | None] | None = None
) -> list[dict[str, Any]]
```

Summarizes the supplied models for the diagnostic snapshot.

Accepts a mapping of model name to the loaded model object, or `None` for a model that is configured but failed to load. Each model is reported with its load state and, when loaded, its device and precision. Reading a single model never breaks the probe.

**models**

Mapping of model name to loaded model object (or `None` when it failed to load). Omitted or empty yields an empty list.

**Returns**:

A list of per-model status dicts, each with `name` and `loaded` and,
for loaded models, `device` and `precision`.

***

#### probe\_versions

```python theme={null}
def probe_versions() -> dict[str, str | None]
```

Probes installed versions of the SDK and its key dependencies.

Reads versions from installed package metadata (not module attributes), so it works even when a package cannot be imported. Dependency drift and ABI mismatches (for example a `torch` that does not match the compiled `.pt2` artifacts) are a common, hard-to-spot cause of failures.

**Returns**:

Mapping of distribution package name to its installed version string,
or `None` when the package is not installed.

***

#### probe\_dependencies

```python theme={null}
def probe_dependencies() -> dict[str, Any]
```

Checks installed dependencies against the SDK's declared constraints.

Reads the version constraints the installed `bioptimus-sdk` distribution declares for its dependencies and verifies that each installed package satisfies them. Where [`probe_versions`](/sdk-reference/observability/probes#probe_versions) reports the raw installed numbers, this probe reports the verdict: upgrading a dependency past a supported ceiling (for example `zarr<3.0.0` or `torch<2.10.0`, which the `.pt2` artifacts are compiled against) is a common, silent cause of breakage that a bare version list does not reveal.

**Returns**:

A status dict with `distribution`, `satisfied` (`True` when every
applicable constraint holds), `conflicts` (a `{"name", "installed",
  "required"}` entry per violation), and `checked` (the number of
constraints evaluated), or `{"error": ...}` when the constraints
cannot be evaluated.

***

#### probe\_host

```python theme={null}
def probe_host() -> dict[str, Any]
```

Probes host platform and resource information.

Reports the Python and OS platform, the interpreter running the SDK (a common "installed it but it is not found" culprit when it is the wrong one), CPU count and affinity, total physical RAM, any container memory limit, the open-file-descriptor limit (which concurrent tile reads can exhaust), and system load average. Each resource field degrades to `None` when it is unavailable on the host (for example load average on non-Unix platforms, or a container limit outside a container).

**Returns**:

A mapping with `python_version`, `platform`, `machine`,
`executable`, `in_venv`, `cpu_count`, `cpu_affinity`,
`total_ram_gb`, `container_memory_limit_gb`, `open_file_limit`,
and `load_avg`.

***

#### probe\_env

```python theme={null}
def probe_env() -> dict[str, str | None]
```

Probes a fixed allowlist of inference-relevant environment variables.

Only the allowlisted, non-secret names in `_TRACKED_ENV_VARS` are read; the full environment is never captured, so credentials and tokens are never exposed.

**Returns**:

Mapping of environment-variable name to its value, or `None` when the
variable is unset.

***

#### probe\_gpu

```python theme={null}
def probe_gpu() -> dict[str, Any]
```

Probes GPU, CUDA, and PyTorch runtime state.

Reports the PyTorch and CUDA build versions, the NVIDIA driver version, the CUDA architectures PyTorch was compiled for, and whether CUDA is available, plus per-device properties (name, compute capability, whether PyTorch's compiled kernels cover it, total and free memory). The driver version is captured even when CUDA is unavailable, since a driver too old for the CUDA build is a leading cause of `cuda_available` being false. A device whose compute capability is missing from PyTorch's build (for example after a wheel reinstall) loads but then fails at kernel launch, so it is flagged here. When CUDA is available it also verifies the context is responsive with a small device-to-host copy, since a present GPU with a corrupted context still fails inference.

**Returns**:

A mapping describing the GPU and runtime state. It always includes
`cuda_available`, and includes `reason` when PyTorch is missing or
CUDA cannot be queried.

***

#### probe\_pt2\_compat

```python theme={null}
def probe_pt2_compat(
    checkpoints: Mapping[str, str | Path] | None = None
) -> list[dict[str, Any]]
```

Checks exported `.pt2` checkpoints against the host architecture.

The AOTInductor `.pt2` artifacts are compiled for a specific GPU (and CPU) architecture; running an artifact compiled for a different architecture fails at load or execution time. This probe reports, per checkpoint, the architecture it was compiled for and whether it matches the current host.

**checkpoints**

Mapping of model name to `.pt2` checkpoint path. Omitted or empty yields an empty list.

**Returns**:

A list of per-checkpoint status dicts, each with `name`,
`checkpoint`, `compiled_for`, `cpu_arch`, `compatible`, and
`reason` (the mismatch message, or `None` when compatible).

***

#### snapshot

```python theme={null}
def snapshot(
        models: Mapping[str, object | None] | None = None,
        checkpoints: Mapping[str, str | Path] | None = None) -> dict[str, Any]
```

Assembles a full diagnostic snapshot of the runtime environment.

Combines every probe into one JSON-friendly mapping. Never raises: each section is captured independently, so a failing probe degrades to an `{"error": ...}` section instead of breaking the snapshot.

**models**

Optional mapping of model name to loaded model object (or `None` when it failed to load), forwarded to [`probe_models`](/sdk-reference/observability/probes#probe_models).

**checkpoints**

Optional mapping of model name to `.pt2` checkpoint path, forwarded to [`probe_pt2_compat`](/sdk-reference/observability/probes#probe_pt2_compat).

**Returns**:

A mapping stamped with `schema_version` and `captured_at` (UTC
ISO-8601), plus the `versions`, `dependencies`, `host`, `env`,
`gpu`, `wsi_backends`, `disk`, `pt2_compat`, and `models`
sections.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.