Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
Client protocol that all client backends (HTTP, AWS SageMaker, GCP Vertex AI, Azure ML, …) must satisfy. Each client encapsulates connection details and knows how to send a JSON request body and return the JSON response body.
@runtime_checkable
class Client(Protocol)
predict / embed only need the serialized body.
def predict(body: str) -> str
def embed(body: str) -> str
def predict_with_embedding(body: str) -> str
NotImplementedError.
def metadata() -> str
async def predict_async(body: str, session: Any = None) -> str
async def embed_async(body: str, session: Any = None) -> str
async def predict_with_embedding_async(body: str, session: Any = None) -> str