soev_server

Make the server app from the SOEV_SERVER_* environment variables.

The soev-server command runs this function with uvicorn. It reads Settings, then builds the app with build from these parts:

• The agents of the plugins of Settings.plugin_allowlist, from the entry point group soev.agents. Each agent gets, for each request, the inference client, the Postgres pool, the model ids, the default model, the context windows, the embedding model, the indexing schema and the subject of the request.
• A Postgres trace store and a PostgresThreadIndex, both in Settings.postgres_schema.
• The model list and the task completions from the chat endpoints of Settings.inference_config_root.
• Collections that embed uploads with Settings.embed_model and store them in Settings.indexing_schema.

At startup, the app opens the connection pool, creates the schema, the thread index table and the collection tables if they do not exist, and opens the trace store. At shutdown, it closes them and the inference client.

import uvicorn

uvicorn.run("soev_server:create", factory=True, host="127.0.0.1", port=8000)

Raises:

• pydantic.ValidationError: A required environment variable is missing, or a value is not valid.
• ValueError: The default model is not a chat endpoint of the inference configuration, or a chat endpoint has no context_window fact.
• soev_agents.plugins.PluginError: A plugin of the allowlist does not load or is not valid.

The configuration of a server that create builds and the soev-server command runs.

Each field comes from the environment variable SOEV_SERVER_ plus the field name in upper case, for example SOEV_SERVER_API_KEY. A list field takes a JSON list, for example SOEV_SERVER_PLUGIN_ALLOWLIST='["tutorial"]'. Settings() raises pydantic.ValidationError when a required variable is missing or a value is not valid.

Make the server app from your agents, a trace store, a thread index and optional collections.

The app serves the threads API of the public profile of soev-agents: the threads under /v1/chat/threads, the tool list at /v1/chat/tools, the model list at /v1/models, task completions at /v1/completions/task, the capabilities at /v1/capabilities and the health route at /healthz. It also serves the routes of Collections.

Each HTTP request, also to the health route and the OpenAPI docs, must send Authorization: Bearer <api_key>. The header X-Soev-Subject names the subject of the request: a plain string, or a JWT whose sub claim the server reads without a check of its signature. So each holder of the key can act as each subject. Without the header, the subject is key-holder. A request without a valid key or subject gets 401.

The thread index records which subject owns each thread. A call on a thread of another subject gets 404, as for a thread that does not exist. The thread list at GET /v1/chat/threads comes from ThreadIndex.listing.

Error responses have the media type application/problem+json, and a JSON body has a code and a detail. A failed admission has the code unauthorized. In the threads API, the code follows the status: not_found (404), conflict (409), invalid_request (422), service_unavailable (503) and refused (each other status). Each error of the collection routes has the code refused. A thread event stream that fails ends with an error event whose code is service_unavailable.

from contextlib import asynccontextmanager

from fastapi import HTTPException

from soev_agents.adapters import InMemoryTraceStore
from soev_agents.entrypoints.http import ModelList, RequestContext, TaskRequest, ToolList
from soev_server import MemoryThreadIndex, build


async def list_models(ctx: RequestContext) -> ModelList:
    return ModelList(data=[])


async def list_tools(ctx: RequestContext) -> ToolList:
    return ToolList(data=[])


@asynccontextmanager
async def complete_task(ctx: RequestContext, body: TaskRequest):
    raise HTTPException(501, "This server completes no tasks")
    yield


app = build(
    api_key="change-me",
    store=InMemoryTraceStore(),
    index=MemoryThreadIndex(),
    agents={"assistant": agent},  # your soev_agents.runtime.Agent
    models=["my-model"],
    default_model="my-model",
    model_provider=list_models,
    task_provider=complete_task,
    tools_provider=list_tools,
    version="1.0.0",
)

Raises:

• ValueError: api_key is empty or only white space.

A store of the threads of each subject, which the server reads to list threads and check access.

The trace store keeps the events of a thread, but not who owns it. The server records each thread in the index when a subject opens or forks it, and removes it when the subject deletes it. Before a call on an existing thread, the server looks the thread up for the subject of the request: if the index does not have it, the call gets a 404 response. build takes an index, and create uses a PostgresThreadIndex.

An implementation can raise fastapi.HTTPException. The server sends its status code to the client.

A thread index that keeps its rows in a dictionary of the process.

The rows are lost when the process stops, and each process has its own rows. Use it for tests and for a small server with one process. It implements ThreadIndex.

Make an empty index.

A thread index that keeps its rows in the table subject_threads of a Postgres schema.

It implements ThreadIndex, and create uses it. Several server processes can share one table. The index does not create the schema: call setup after the schema exists, before the first request.

Make an index on the table subject_threads of schema.

The document collections of a server: the indexer that adds documents and the catalog that lists them.

Pass it to build as collections. The server then serves two routes for the subject of each request:

• GET /v1/collections lists the subject's collections from catalog.
• POST /v1/collections/{name}/documents takes one multipart file, indexes it into collection name of the subject with indexer, and returns 201 with document_id, collection and chunks. A file over max_upload_bytes gets 413, a format that the indexer does not support gets 415, and a document that the indexer refuses gets 422.

Without collections, both routes return 503. They also return 503 when the database, the connection pool, the file system or the embedding model fails.

A source of the document collections of each subject, which GET /v1/collections lists.

A soev_indexing.PgSink is a catalog.