Tutorial 3: one agent for many users

In tutorial 2, all users of the assistant searched the same library. In a real town, employees can read internal documents that residents cannot read. In this tutorial, one agent serves both: each request searches only the documents that its user can read. This is access control. You also move the prompt to a prompt file, with a different version of the rules for one model.

Do tutorial 2 first: this agent uses its tools.

Set upCopy Anchor Link

Make a folder for this tutorial, and copy the library and the tools of tutorial 2 into it:

mkdir -p my_tutorials/03-many-users
cp my_tutorials/02-citations/library.py my_tutorials/02-citations/tools.py my_tutorials/03-many-users/

This tutorial does not change library.py or tools.py. The tools already take the library as a parameter, so they work with the library of any user.

Step 1: write the prompt fileCopy Anchor Link

A prompt file, which PromptFile loads, is a YAML file that holds each text that the agent gives to its model. You need one for three reasons:

1. A version for each model. Models follow instructions differently. A layer in the file replaces part of the prompt for the models that it names. The code stays the same.
2. One place for each text. A change of the prompt is a change of one file, not of the code. A tool description can also go in the file, under tools.
3. A name for each set of texts. The framework computes a digest of the texts that a model reads. The digest tells two versions of the texts apart, for example in an evaluation.

Make my_tutorials/03-many-users/prompt.yaml:

system: |
  You are the assistant of the town of Springfield.
  {% block rules %}
  Answer questions from the town documents. If the documents do not answer the question, say so. Never guess.
  After each fact, cite the passage that gives it with its number, like {{ citation }}.
  {% endblock %}

models:
  - endpoints: [gpt-4.1-nano]
    blocks:
      rules: |
        Use only the town documents. If they do not answer the question, write: "The documents do not say."
        Write one short sentence for each fact, and end it with the number of its passage, like {{ citation }}.

system is the system prompt. It is a Jinja template: a text with parts that the agent fills when it builds the prompt.

• {% block rules %} ... {% endblock %} names a part of the prompt. A layer can replace that part.
• The layer under models names, in endpoints, the models that it is for: here, gpt-4.1-nano. Each name must be the same as the model of the Input, here the value of OPENAI_MODEL. For that model, the agent uses the rules of the layer. Each other model gets the base text.
• {{ citation }} is a variable. The agent fills it with an example of a citation, such as [1], from its citation strategy. In tutorial 2, the prompt wrote [1] itself. Now the prompt and the parser of the citations always use the same form, also when you change the strategy.

citation is the only variable that a ReActAgent gives to the template. The code and the file must agree on that, so ReActConfig checks the file. If the file reads another variable, for example the typo {{ citaton }}, ReActConfig raises PromptFileError. In this tutorial, the agent makes its ReActConfig at the start of each turn (step 3), so the first turn fails before the model runs, and records nothing. To check the file when the program starts, call PromptFile.check_variables after you load the file, here PROMPTS.check_variables({"citation"}).

The system prompt does not change within a thread. The agent renders the template again at each turn, but with the same model it gets the same text. It changes only when a turn names another model.

To see the texts that the file gives one model, print them: the digest, the base template, and each block that the layer of that model replaces:

uv run python -m soev_agents.toolkit.prompts my_tutorials/03-many-users/prompt.yaml --model gpt-4.1-nano

The output, shortened:

# model: gpt-4.1-nano
# digest: 052c1f27f6dcf6b9

## system
You are the assistant of the town of Springfield.
{% block rules %}
...
{% endblock %}


## system block rules (this model)
Use only the town documents. If they do not answer the question, write: "The documents do not say."
...

The command does not render the template. For gpt-4.1-nano, the block of the layer replaces the base rules block.

Step 2: give each user a libraryCopy Anchor Link

Make my_tutorials/03-many-users/agent.py. These are all its imports, and the line that loads the prompt file:

import asyncio
import os
from pathlib import Path

from library import LIBRARY, Library
from openai import AsyncOpenAI
from tools import Document, Passage, read_document, search

from soev_agents.adapters import InMemoryTraceStore, OpenAIChatModel
from soev_agents.domain import Input, ModelOutput, inserted
from soev_agents.ports import TraceStore
from soev_agents.runtime import Agent, AgentCatalog
from soev_agents.toolkit import BRACKETS, PromptFile, ReActAgent, ReActConfig, Text

PROMPTS = PromptFile.load(Path(__file__).with_name("prompt.yaml"))

soev_agents.ports is a fifth part: the interfaces that the adapters implement. TraceStore is the interface of a store, and Agent is the interface of an agent. Here, they are only type annotations.

Add an internal document, and a library for each kind of user:

INTERNAL = Library.of(
    {
        "dog-tax-exemptions": (
            "Dog tax exemptions (internal)",
            (
                "The owner of an assistance dog pays no dog tax for that dog.",
                "Before you approve an exemption, check the certificate of the training school.",
            ),
        ),
    }
)

LIBRARIES = {
    "resident": LIBRARY,
    "employee": Library(LIBRARY.sections + INTERNAL.sections),
}

A resident searches the public documents. An employee searches the public documents and the internal one.

Step 3: build the settings for each turnCopy Anchor Link

In tutorial 2, you gave ReActAgent one ReActConfig. Here, the tools need the library of the user who asks, and that changes for each request. So you give a function that builds the ReActConfig for each turn:

def assistant(model: OpenAIChatModel) -> ReActAgent[Library, Text]:
    def config(library: Library, turn: Text, model_name: str) -> ReActConfig[Text]:
        return ReActConfig(
            model=model,
            prompt=PROMPTS,
            tools=[search.bind(library=library), read_document.bind(library=library)],
            citations=BRACKETS,
        )

    return ReActAgent(config, element_types=(Document, Passage))

The agent calls config at the start of each turn, with three values:

• library: the resources of this request. Resources are the adapters that one request gets, such as the document store of its user. Step 5 shows where they come from.
• turn: the input of the turn, parsed into Text.
• model_name: the model that the input names.

This config reads only library. A config can also read the turn or the model, for example to switch off a tool for a model that cannot call tools.

When to give a function, and when to give a ReActConfig:

• If each setting is the same for all requests, give a ReActConfig, as in tutorials 1 and 2.
• If a setting depends on the request, the turn or the model, give a function. Here, the tools depend on the user.

prompt=PROMPTS gives the whole prompt file. The agent selects the texts for the model of the turn, and fills {{ citation }}.

ReActAgent[Library, Text] gives the types of the resources and of the input. Pyright then checks that config takes a Library.

Step 4: understand the agent and the graphCopy Anchor Link

A function that runs at each turn shows how the framework runs an agent. Two objects are involved:

• The agent is the object that you made with ReActAgent. It exists once, for the whole process. All threads and all users share it.
• The graph is the program for one turn: the steps, and the order in which they run. A ReAct graph has two steps: llm calls the model, and tools runs the tool calls of the model. You saw their names in the trace of tutorial 1.

This is one turn, for one user:

1. A request comes in. The catalog gets the resources of this request: the library of the user.
2. The agent builds the graph for the turn. It calls config with the library, and puts the bound tools in the step tools. The search tool of this graph now holds the library of this user. The framework also checks the turn here. For example, it refuses a tool with a parameter that nothing binds. A refused turn records nothing.
3. The runner runs the graph. It calls llm, then tools, then llm again. Each step gets only the trace. A step does not get the library: it already holds it from item 2.
4. The turn ends. The graph is dropped. The next turn starts again at item 1. After a crash, a thread also starts again at item 1, from the input that the trace recorded.

This is why the agent and the graph are two objects. The agent is shared by all users, so it must not hold the library of one user. If it did, the turn of an employee could put internal documents in the agent, and the next resident could search them. Each turn gets its own graph, with the library of its own user.

Tutorial 5 gives more reasons, when you build a graph yourself.

Step 5: ask as two usersCopy Anchor Link

Add a function that asks a question as one user:

async def ask(agent: ReActAgent[Library, Text], store: TraceStore, user: str, question: Input) -> None:
    async def resources(agent: Agent[Library, Text]) -> Library:
        return LIBRARIES[user]

    agents = AgentCatalog(store, {"assistant": agent}, version="tutorial-3", resources=resources)
    thread, _ = await agents.open("assistant", question)
    answer = (await thread.resume()).by_type(ModelOutput)[-1]
    print(f"{user}:", inserted(answer.content, [(marker.at, marker.text) for marker in answer.markers]))

resources tells the catalog how to get the resources for each turn. Here, it returns the library of user. This script makes a catalog for each user, so that you can see both users in one run. Both catalogs share the agent and the store. A server has one catalog, and reads the user from each HTTP request (tutorial 6).

Now add main, and the lines that run it:

async def main() -> None:
    model = OpenAIChatModel(AsyncOpenAI(), model=os.environ["OPENAI_MODEL"])
    agent = assistant(model)
    store = InMemoryTraceStore()
    question = Input(payload="Is there an exemption from the dog tax?", model=model.model)
    await ask(agent, store, "resident", question)
    await ask(agent, store, "employee", question)


if __name__ == "__main__":
    asyncio.run(main())

Run the agent:

uv run python my_tutorials/03-many-users/agent.py

The output is similar to this:

resident: The documents do not say.
employee: The owner of an assistance dog pays no dog tax for that dog [4].

One agent gave two answers, because the search of each user found different passages. The resident never saw the internal document. The tools did not change: only the library that the agent bound to them changed.

What you builtCopy Anchor Link

• A prompt file with a layer for one model, and a citation example from the citation strategy.
• A config function that builds the settings of each turn from the resources of the request.
• One agent that serves two users, each with the documents that the user can read.

You also saw how a turn runs: the agent builds a graph for each turn, with the resources of its request, and the runner runs that graph.

The complete code is in examples/tutorials/03-many-users.

A long conversation, or a long document, sends the model more text at each call. In tutorial 4, you look at exactly what the model receives, and control it.