> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bioptimus.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Diagnosing slow throughput or errors from the Model Server.

Not everything slow is a tuning problem. Start here before changing any settings.

## Is the GPU actually the bottleneck?

Watch utilization while a run is in progress:

```bash theme={null}
watch nvidia-smi
```

* **GPU near 100%** → you're GPU-bound. Only precision, tile count, or a bigger/newer GPU will help further — see [Performance](/api-reference/performance).
* **GPU well below 100%** → the GPU is starved. Check the client's concurrency setting first, then disk/network I/O and CPU load.

At `INFO` level, the server also logs per-batch timing (`forward=... cpu_transfer=... total=...`). If `forward` ≈ `total` and batches fire back-to-back, you're GPU-bound (expected); a gap between consecutive "batch done" lines means the GPU is waiting on tile production instead.

## Failure modes that look like slowness

<AccordionGroup>
  <Accordion title="First request is much slower than the rest">
    Cold start — model loading and first-batch kernel warmup. Don't benchmark the first request; measure steady-state throughput instead.
  </Accordion>

  <Accordion title="Throughput drops when reading from a network share or object storage">
    Storage can't keep up with the GPU. Stage slides on fast local disk where possible.
  </Accordion>

  <Accordion title="Throughput is roughly halved">
    Another process may be sharing the GPU. Check `nvidia-smi` for other processes and ensure exclusive access.
  </Accordion>

  <Accordion title="Raising PREPROCESS_WORKERS makes things worse">
    Oversubscribing CPU cores causes thread thrashing. Don't set it above the host's core count; it rarely helps once the GPU is the bottleneck anyway.
  </Accordion>

  <Accordion title="Model fails to load with a zip/extraction error">
    `.pt2` artifacts extract to a temp directory (`TMPDIR`, default `/tmp`) on load; a failed load can leave stale extraction directories behind and fill the disk. Check `df -h /tmp`, clear stale extraction directories, or point `TMPDIR` at a larger volume.
  </Accordion>
</AccordionGroup>

## Errors

`503` and out-of-memory responses usually mean too many requests are arriving concurrently for the deployed batch size — see [Errors](/api-reference/introduction#errors) for the full status code reference.

<Note>
  Calling the server through the SDK? For connection issues (e.g. `/ping` not responding), see the SDK's [Common issues](/guides/get-started/troubleshooting) guide.
</Note>