Skip to main content
HTTP client for direct API endpoints. Sends JSON payloads to a model server via plain HTTP POST using requests (synchronous) and aiohttp (asynchronous).

HTTPClient

Sends JSON payloads over HTTP. Fully configured at construction — predict / embed only need the serialized body.
Root URL of the model server, e.g. "http://localhost:8080".
Mapping of endpoint type to path, e.g. {"prediction": "/api/predict/m-optimus", "embedding": "/api/embed/m-optimus"}. At least one key must be present.
HTTP request timeout in seconds.
Example:

close

Closes the underlying HTTP session and releases connections.

predict

POST to the prediction endpoint.
str
required
Serialized JSON request payload.
str
Response body as a JSON string.

embed

POST to the embedding endpoint.
str
required
Serialized JSON request payload.
str
Response body as a JSON string.

metadata

GET the metadata endpoint.
str
Response body as a JSON string.

predict_async

POST to the prediction endpoint asynchronously.
str
required
Serialized JSON request payload.
Any
An ClientSession for connection pooling.
str
Response body as a JSON string.

embed_async

POST to the embedding endpoint asynchronously.
str
required
Serialized JSON request payload.
Any
An ClientSession for connection pooling.
str
Response body as a JSON string.