> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bioptimus.com/llms.txt
> Use this file to discover all available pages before exploring further.

# On-premise

> Load, run, and verify the Bioptimus Model Server container in your own environment.

The Bioptimus Model Server runs H-Optimus and M-Optimus entirely within your infrastructure — keeping data on your hardware for residency, compliance, or air-gapped requirements. It serves the same JSON API as the [SageMaker deployment](/deployment/platforms/aws-sagemaker), and the [Bioptimus SDK](/guides/get-started/sdk) works against both.

Two package variants are available. Each image contains the relevant model **plus** the tissue segmentation model:

| Package | Models                              | Key endpoints                                                                              |
| ------- | ----------------------------------- | ------------------------------------------------------------------------------------------ |
| **H1**  | H-Optimus (`h1`), tissue-seg        | `POST /api/embed/h1`, `POST /api/predict/tissue-seg`                                       |
| **M**   | M-Optimus (`m-optimus`), tissue-seg | `POST /api/embed/m-optimus`, `POST /api/predict/m-optimus`, `POST /api/predict/tissue-seg` |

## Prerequisites

| Requirement              | Detail                                                                                                                                                                           |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Docker                   | Version 20.10 or later                                                                                                                                                           |
| NVIDIA Container Toolkit | Required for GPU inference ([install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html))                                               |
| GPU                      | NVIDIA GPU with **CUDA Compute Capability 8.6** (e.g. A10G-class, equivalent to SageMaker `ml.g5.xlarge`). A CUDA GPU is **required** — the container exits if none is detected. |
| CPU architecture         | **x86-64** — CC 8.6 + x86-64 runs out of the box; other architectures require recompilation.                                                                                     |
| GPU access               | Direct passthrough (`--gpus all`)                                                                                                                                                |
| OS                       | Linux with an NVIDIA GPU                                                                                                                                                         |
| Disk                     | Enough to hold the image (weights are baked in)                                                                                                                                  |

<Note>
  The REST API currently requires **no authentication** — run the server on a private network and restrict access at the network layer. See the [API reference](/api-reference/introduction#authentication).
</Note>

## 1. Load the container image

You receive the container as a compressed archive plus a `sha256` checksum. Verify integrity, then load it into Docker.

<Tabs>
  <Tab title="H1 package">
    ```bash theme={null}
    sha256sum -c bioptimus-h1-onpremise-v1.0.1.sha256
    docker load < bioptimus-h1-onpremise-v1.0.1.tar.gz
    docker images bioptimus-h1-onpremise
    ```
  </Tab>

  <Tab title="M package">
    ```bash theme={null}
    sha256sum -c bioptimus-m-onpremise-v1.0.1.sha256
    docker load < bioptimus-m-onpremise-v1.0.1.tar.gz
    docker images bioptimus-m-onpremise
    ```
  </Tab>
</Tabs>

## 2. Start the container

The server is self-contained — all weights and assets are baked into the image, so no volume mounts are required.

<Tabs>
  <Tab title="H1 package">
    ```bash theme={null}
    docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
      bioptimus-h1-onpremise:v1.0.1 serve
    ```
  </Tab>

  <Tab title="M package">
    ```bash theme={null}
    docker run -d --name bioptimus-server --gpus all -p 8080:8080 \
      bioptimus-m-onpremise:v1.0.1 serve
    ```
  </Tab>
</Tabs>

Models load at startup. Follow the logs until the server is ready:

```bash theme={null}
docker logs -f bioptimus-server 2>&1 | grep -m1 "models ready"
```

## 3. Air-gapped install

For environments with no outbound network, transfer the `.tar.gz` archive via your approved process, then run `sha256sum -c`, `docker load`, and `docker run` exactly as above. No registry access is needed — the image is self-contained.

## 4. Verify the deployment

<Steps>
  <Step title="Health check">
    <CodeGroup>
      ```bash H-Optimus package theme={null}
      curl -s http://localhost:8080/ping | python3 -m json.tool
      # {"status": "ok", "models": ["h1", "tissue-seg"]}
      ```

      ```bash M-Optimus package theme={null}
      curl -s http://localhost:8080/ping | python3 -m json.tool
      # {"status": "ok", "models": ["m-optimus", "tissue-seg"]}
      ```
    </CodeGroup>

    A 503 `{"status": "loading"}` means models are still initialising — retry. See the [API reference](/api-reference/introduction) for all health states.
  </Step>

  <Step title="Interactive API docs">
    Open `http://localhost:8080/docs` for a Swagger UI to explore endpoints and try calls. Service discovery is at `http://localhost:8080/bioptimus/`.
  </Step>

  <Step title="Test with the Bioptimus SDK">
    <CodeGroup>
      ```python H-Optimus theme={null}
      from bioptimus.models.backbones import Backbone
      from bioptimus.models.types import Models

      print(Backbone.available_backbones())   # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
      model = Backbone(Models.H1, base_url="http://localhost:8080")
      ```

      ```python M-Optimus theme={null}
      from bioptimus.models.backbones import Backbone
      from bioptimus.models.types import Models

      print(Backbone.available_backbones())   # ['h1', 'm-optimus', 'tissue-seg'] — SDK-known models; /ping shows what's deployed
      model = Backbone(Models.M_OPTIMUS, base_url="http://localhost:8080")
      ```
    </CodeGroup>

    For whole-slide inference, see the [Bioptimus SDK](/guides/get-started/sdk).
  </Step>
</Steps>

## Managing the container

```bash theme={null}
docker stop bioptimus-server      # stop (preserves state)
docker start bioptimus-server     # restart
docker logs bioptimus-server      # view logs
docker rm -f bioptimus-server     # remove
```

## Environment variables

Set with `-e` at start:

| Variable       | Default         | Description                                     |
| -------------- | --------------- | ----------------------------------------------- |
| `PORT`         | `8080`          | Server port                                     |
| `LOG_LEVEL`    | `INFO`          | Log level (`DEBUG`, `INFO`, `WARNING`, `ERROR`) |
| `SM_MODEL_DIR` | `/opt/ml/model` | Model artefact directory (used by SageMaker)    |

## Troubleshooting

<AccordionGroup>
  <Accordion title="curl: (7) Failed to connect">
    Container not running or still starting. Check `docker ps` and `docker logs bioptimus-server`.
  </Accordion>

  <Accordion title="/ping returns {&#x22;status&#x22;: &#x22;loading&#x22;}">
    Models are still loading. Wait for the `models ready` log line.
  </Accordion>

  <Accordion title="/ping returns {&#x22;status&#x22;: &#x22;unhealthy&#x22;}">
    The GPU is no longer available, or the CUDA context was corrupted. Check `nvidia-smi` on the host and restart the container.
  </Accordion>

  <Accordion title="Container exits immediately">
    Common causes: no CUDA GPU detected (a GPU is required), port conflict, or insufficient memory. Run `docker logs bioptimus-server` for the error.
  </Accordion>

  <Accordion title="Out of GPU memory">
    The server auto-batches concurrent requests (max batch size 32). Reduce the number of concurrent SDK requests to lower peak memory use.
  </Accordion>
</AccordionGroup>