A VectorStore over pgvector tables the host owns: a table per target, fields wherever the host keeps them.

Install the soev-retrieval[pgvector] extra to use this package. The store reads tables that the host created, and never creates or changes one.

Store

The store maps each target to a table, and reads it through a psycopg async pool.

`PgVectorStore`

dataclassPgVectorStore

(pool, *, resolve, fields_of, extension_schema=None, id_of=None, extra_filter=None, alpha=0.7, ef_search=100, candidates=4)

A VectorStore over a caller-owned async pool, configured by arguments and never subclassed.

The store needs pgvector 0.8 or later. Reads never create tables or embed queries.

Each argument after pool is keyword-only. Each read runs in a transaction with HNSW iterative scan, with ef_search, and with a custom plan for each query. Each method raises StoreUnavailable when Postgres cannot be reached or the pool gives no connection, Unconfigured when a table, column, schema, function, operator or type does not exist, and InvalidQueryError when Postgres refuses the data or the SQL of the call.

Example

from psycopg_pool import AsyncConnectionPool

from soev_retrieval.adapters.pgvector import PgTable, PgVectorStore

embedding = [0.1, 0.2, 0.3]  # the query, embedded with the model of the table
tables = {"documents": PgTable("documents", text="content")}
fields = {"documents": {"source": ("keyword", True), "year": ("number", True)}}
async with AsyncConnectionPool("postgresql://localhost/app", open=False) as pool:
    store = PgVectorStore(pool, resolve=tables.__getitem__, fields_of=fields)
    hits = await store.near_vector("documents", vector=embedding, limit=5)

parampoolAsyncConnectionPool[Any]

The psycopg async connection pool. The caller opens and closes it.

paramresolveCallable[[Target], PgTable]

Target to the PgTable it reads. It can raise InvalidQueryError for a target it will not serve. A KeyError from it counts as an unknown target, so a dict.__getitem__ works.

paramfields_ofMapping[Target, FieldSpec]

The field spec of each target. There is no schema introspection.

paramextension_schemastr | None

= None

Qualify pgvector names with this schema. None relies on search_path.

paramid_ofCallable[[Mapping[str, Any]], str] | None

= None

Derive a chunk id from the row mapping. The default joins the key columns with #.

paramextra_filterExtraFilter | None

= None

A function that returns an SQL fragment over CHUNK and its named parameters for a target, or None. The store adds it with AND to each side of a search and to each fetch. A parameter name must not start with pgv_.

paramalphafloat

= 0.7

The hybrid blend when a call passes alpha=None, from 0 to 1. 1.0 is pure vector, 0.0 pure keyword.

paramef_searchint

= 100

The hnsw.ef_search setting of each read, a positive integer.

paramcandidatesint

= 4

Each side of hybrid reads limit * candidates rows. The default of 4 is a value to tune on the host's corpus.

Raises:

InvalidQueryError: alpha is not between 0 and 1, or ef_search or candidates is not a positive integer.

methodhybridasync (target, *, query, vector=None, limit, where=None, alpha=None) -> Sequence[Hit]

Search target with a vector match and a keyword match, blended by alpha.

Each side reads limit * candidates rows. The store min-max normalises the scores of each side, weights the vector side by alpha and the keyword side by 1 - alpha, and sums the weights of each row. With alpha 1.0 only the vector side runs, and with 0.0 only the keyword side runs.

paramtargetTarget

The target to search. resolve maps it to a table.

paramquerystr

The text of the keyword side. Each word of it is a term, and a row matches when it holds one term in any of the ts_configs.

paramvectorSequence[float] | None

= None

The query embedding. The store does not embed, so None raises Unconfigured.

paramlimitint

The maximum number of hits, zero or more.

paramwhereFilter | None

= None

The filter on the candidates, or None for no filter.

paramalphafloat | None

= None

The weight of the vector side, from 0.0 to 1.0. None uses the alpha of the store.

Returns

Sequence[Hit]

At most limit hits, best first, with the blended score. The list is empty without a query when where matches nothing.

Raises:

InvalidQueryError: The target is unknown, where names a field, an operator or an operand that the store does not serve, alpha is not between 0 and 1, limit is negative, or vector holds a value that is not a finite number.
Unconfigured: vector is None, or the keyword side runs on a table without tsvector or text.
StoreUnavailable: Postgres gave no answer.

methodnear_vectorasync (target, *, vector, limit, where=None) -> Sequence[Hit]

Search target by distance to vector alone, through the HNSW index of the embedding.

paramtargetTarget

The target to search. resolve maps it to a table.

paramvectorSequence[float]

The query embedding.

paramlimitint

The maximum number of hits, zero or more.

paramwhereFilter | None

= None

The filter on the candidates, or None for no filter.

Returns

Sequence[Hit]

At most limit hits, best first. The score is one minus the pgvector distance. The list is empty without a query when where matches nothing.

Raises:

InvalidQueryError: The target is unknown, where names a field, an operator or an operand that the store does not serve, limit is negative, or vector is empty or holds a value that is not a finite number.
Unconfigured: The table, a column or the schema does not exist, a row has no document id, or a stored date is not ISO 8601.
StoreUnavailable: Postgres gave no answer.

methodbm25async (target, *, query, limit, where=None) -> Sequence[Hit]

Search target by keyword match on query alone, ranked by ts_rank_cd.

paramtargetTarget

The target to search. resolve maps it to a table.

paramquerystr

The keyword query. Each word of it is a term, and a row matches when it holds one term in any of the ts_configs.

paramlimitint

The maximum number of hits, zero or more.

paramwhereFilter | None

= None

The filter on the candidates, or None for no filter.

Returns

Sequence[Hit]

At most limit hits, best first, with the ts_rank_cd score. A chunk that matches no term is not returned. The list is empty without a query when where matches nothing.

Raises:

InvalidQueryError: The target is unknown, where names a field, an operator or an operand that the store does not serve, or limit is negative.
Unconfigured: The table has no tsvector and no text column.
StoreUnavailable: Postgres gave no answer.

methodfetchasync (target, *, where=None, limit, order_by=None, descending=False) -> Sequence[Chunk]

Return the chunks of target that match where, with nothing searched for.

paramtargetTarget

The target to read. resolve maps it to a table.

paramwhereFilter | None

= None

The filter on the chunks, or None for no filter.

paramlimitint

The maximum number of chunks, zero or more.

paramorder_bystr | None

= None

The field to sort on, ascending unless descending. The sort key comes from the FieldSource that serves the field. None sorts on the key columns only.

paramdescendingbool

= False

If true, sort on order_by from high to low.

Returns

Sequence[Chunk]

At most limit chunks. Rows with the same order_by value are in the order of the key columns. The list is empty without a query when where matches nothing.

Raises:

InvalidQueryError: The target is unknown, order_by is not a field of the target or its source refuses to sort on it, where names a field, an operator or an operand that the store does not serve, or limit is negative.
Unconfigured: The table, a column or the schema does not exist, a row has no document id, or a stored date is not ISO 8601.
StoreUnavailable: Postgres gave no answer.

methodfieldsasync (target) -> FieldSpec

Return the stored fields of target and which of them a filter can name.

The store reads no schema. The spec is the entry of fields_of for target.

paramtargetTarget

The target to describe.

Returns

FieldSpec

The field spec of target.

Raises:

InvalidQueryError: resolve or fields_of does not know target.

`ExtraFilter`

type aliasExtraFilter = Callable[[Target], tuple[sql.Composable, Mapping[str, object]] | None]

A function that returns the SQL to add to each read of a target, with its named parameters.

The store calls it with the target of each search and each fetch, but not of fields. It returns a pair of an SQL fragment over the alias CHUNK and the values of the named parameters (%(name)s) in the fragment, or None to add nothing. The store adds the whole fragment, in parentheses, with AND to the filter of the call. A parameter name must not start with pgv_, because that prefix is reserved for the store.

`CHUNK`

constantCHUNK = …

The table alias of the chunk row in each query of PgVectorStore.

Write each SQL fragment of a FieldSource and of an extra_filter over this alias, for example sql.SQL("{}.tenant = %(tenant)s").format(CHUNK).

Tables

A table description names the key, the document column, the embedding and the keyword columns.

`PgTable`

dataclassPgTable

(name, *, schema=None, key=('id',), document='id', embedding=Embedding(), tsvector=None, text=None, ts_configs=('english',), fields=(JsonbObject('metadata'),))

One table a target reads.

The host owns the table and its indexes. The adapter only reads.

Each argument after name is keyword-only.

paramnamestr

The name of the table.

paramschemastr | None

= None

The schema of the table. None uses the search_path of the connection.

paramkeytuple[str, ...]

= ('id',)

The primary key. Ties break on it and hybrid fusion joins on it.

paramdocumentstr

= 'id'

The column supplying document_id.

paramembeddingEmbedding

= Embedding()

The embedding column and the metric of its index.

paramtsvectorstr | None

= None

An indexed tsvector column for keyword search.

paramtextstr | None

= None

Fallback when there is no tsvector: this column is tokenised per query with every ts_configs entry, so keyword search scans the table. Give tsvector for anything beyond a small table.

paramts_configstuple[str, ...]

= ('english',)

Text search configurations for the query, and for text. Match the ones tsvector was built with.

paramfieldstuple[FieldSource, ...]

= (JsonbObject('metadata'),)

Where the fields of the field spec live. The first source that serves a name answers for it.

Raises:

Unconfigured: key or ts_configs is empty, or both tsvector and text are given.

attributeidentifiersql.Identifier

The table name for a psycopg.sql query, qualified with schema when it is set.

`Embedding`

dataclassEmbedding(column='embedding', type='vector', metric='cosine')

The embedding column, its pgvector type and the metric its HNSW index was built with.

The store ranks by the pgvector distance operator of the metric. A bit column compares with the query vector quantised by binary_quantize.

paramcolumnstr

= 'embedding'

The name of the embedding column.

paramtypeVectorType

= 'vector'

The pgvector type of the column.

parammetricMetric

= 'cosine'

The metric of the HNSW index on the column.

Raises:

Unconfigured: The metric does not fit the type, for example bit with cosine.

`VectorType`

type aliasVectorType = Literal['vector', 'halfvec', 'bit']

The pgvector type of an embedding column: vector, halfvec or bit.

`Metric`

type aliasMetric = Literal['cosine', 'dot', 'l2', 'hamming']

The distance metric of an HNSW index.

vector and halfvec take cosine, dot or l2. bit takes hamming.

Field storage

A field source says where some fields of a table live. Write your own for storage that the stock sources do not cover.

`FieldSource`

protocolFieldSource

Where some fields of a table live.

PgTable.fields asks its sources in order, and the first that serves wins.

The source that serves a field filters on it and sorts on it. Write each SQL fragment over the alias CHUNK.

methodserves(name) -> bool

Return whether this source stores the field name.

paramnamestr

A field name of the field spec.

Returns

bool

True if this source filters and sorts on name.

methodpredicate(name, kind, terms, bind) -> sql.Composable

Return SQL over CHUNK that holds when every term, all on name, holds.

NULL counts as not holding.

The terms that one AllOf holds on one field reach the source together, so the source can answer them in one lookup.

paramnamestr

The field that every term names.

paramkindFieldType

The type of the field in the field spec.

paramtermsSequence[DeclaredFilter]

The predicates on name. Each has an operator of OPERATORS and an operand of the right type.

parambindBind

Bind each operand through it.

Returns

sql.Composable

The SQL condition.

Raises:

InvalidQueryError: The source does not serve an operator of terms.

methodvalues(names, bind) -> sql.Composable

Return a jsonb object over CHUNK of the named fields this source has a value for.

The store asks every source of the table, and takes each field from the first source that has a value for it.

paramnamesSequence[str]

The fields of the field spec.

parambindBind

Bind each value of the fragment through it.

Returns

sql.Composable

An SQL expression of type jsonb, with one key for each field that the row has a value for.

methodsort_key(name, kind) -> sql.Composable | None

Return an expression to order fetch on, or None to refuse ordering on name.

paramnamestr

A field that this source serves.

paramkindFieldType

The type of the field in the field spec.

Returns

sql.Composable | None

The SQL expression to sort on, or None. The store refuses a sort on None as InvalidQueryError.

`Columns`

dataclassColumns(mapping)

Fields stored as plain columns of the table, named by mapping (field name to column).

Each column holds one value. Store a list field in a JsonbObject, or write a FieldSource for array columns.

parammappingMapping[str, str]

The column of each field, by field name. The source serves exactly the names of mapping.

methodserves(name) -> bool

Return whether mapping has a column for the field name.

paramnamestr

A field name of the field spec.

Returns

bool

True if name is a key of mapping.

methodpredicate(name, kind, terms, bind) -> sql.Composable

Return SQL that holds when the column of name satisfies every term, with compare.

paramnamestr

The field that every term names.

paramkindFieldType

The type of the field in the field spec.

paramtermsSequence[DeclaredFilter]

The predicates on name.

parambindBind

Bind each operand through it.

Returns

sql.Composable

The SQL condition.

Raises:

InvalidQueryError: A term has an operator that compare does not serve.

methodvalues(names, bind) -> sql.Composable

Return a jsonb object of the columns of the named fields that mapping holds.

paramnamesSequence[str]

The fields of the field spec.

parambindBind

Bind each key of the object through it.

Returns

sql.Composable

An SQL expression of type jsonb, with a key for each name of names that mapping holds.

methodsort_key(name, kind) -> sql.Composable | None

Return the column of name, to sort on.

paramnamestr

A field that mapping holds.

paramkindFieldType

The type of the field. The column sorts by its own type.

Returns

sql.Composable | None

The column of name over CHUNK.

`JsonbObject`

dataclassJsonbObject(column)

Fields as keys of a jsonb object column, for example a column metadata.

The source serves every name, so list it last.

Keyword and text equality matches a string value or an array containing it, through @>, which a GIN index on the column serves. Number, date and bool values must be scalars. They are cast per row, unindexed unless the table has an expression index on that key.

paramcolumnstr

The name of the jsonb column.

methodserves(name) -> bool

Return true: this source serves every field name.

paramnamestr

A field name of the field spec.

Returns

bool

Always true.

methodpredicate(name, kind, terms, bind) -> sql.Composable

Return SQL that holds when the key name of the column satisfies every term.

On a keyword or text field, eq, in and contains_any use @>. An in or contains_any with an empty list holds for no row. The other terms compare the value of the key with compare: as text, or cast to the Postgres type of a number, date or bool field.

paramnamestr

The field that every term names, and the key in the object.

paramkindFieldType

The type of the field in the field spec.

paramtermsSequence[DeclaredFilter]

The predicates on name.

parambindBind

Bind each operand through it.

Returns

sql.Composable

The SQL condition.

Raises:

InvalidQueryError: A term has an operator that compare does not serve.

methodvalues(names, bind) -> sql.Composable

Return a jsonb object of the keys of the column that names holds.

paramnamesSequence[str]

The fields of the field spec.

parambindBind

Bind the list of names through it.

Returns

sql.Composable

An SQL expression of type jsonb with the named keys that the row has. A row without them gives {}.

methodsort_key(name, kind) -> sql.Composable | None

Return the value of the key name, cast to the field type, to sort on.

paramnamestr

The key in the object.

paramkindFieldType

The type of the field. A number, date or bool value is cast to its Postgres type, and other values sort as text.

Returns

sql.Composable | None

The SQL expression to sort on.

`Bind`

protocolBind

A function that binds a value as a fresh adapter-owned parameter and returns its placeholder.

The store gives one to each method of a FieldSource that writes SQL. Bind each value of a fragment through it, so that the fragment holds no literal values.

method__call__(value) -> sql.Composable

Bind value as a new parameter of the query.

paramvalueobject

The value to bind. psycopg adapts it to a Postgres value.

Returns

sql.Composable

The placeholder of the parameter, to use in the fragment in place of the value.

`compare`

functioncompare(expression, kind, term, bind) -> sql.Composable

Return expression op value for one term, for sources that store scalars.

The function binds the value through bind. It casts the value to timestamptz for a date field. in and contains_any compare against each value of the list with = ANY.

paramexpressionsql.Composable

The SQL expression of the stored value, over CHUNK.

paramkindFieldType

The type of the field in the field spec.

paramtermDeclaredFilter

The predicate to compare.

parambindBind

Bind the operand through it.

Returns

sql.Composable

The SQL condition.

Raises:

InvalidQueryError: The operator is not eq, gt, gte, lt, lte, in or contains_any, or an in or contains_any operand is not a list.

`all_hold`

functionall_hold(parts) -> sql.Composable

Return the parts ANDed. The caller parenthesises the result.

Use it in FieldSource.predicate to join one condition for each term.

parampartsSequence[sql.Composable]

The SQL conditions, at least one. The function puts each in parentheses when there is more than one.

Returns

sql.Composable

The one part as it is, or the parts joined with AND.

soev_retrieval.adapters.pgvector

StoreCopy Anchor Link

PgVectorStoreCopy Anchor Link

ExtraFilterCopy Anchor Link

CHUNKCopy Anchor Link

TablesCopy Anchor Link

PgTableCopy Anchor Link

EmbeddingCopy Anchor Link

VectorTypeCopy Anchor Link

MetricCopy Anchor Link

Field storageCopy Anchor Link

FieldSourceCopy Anchor Link

ColumnsCopy Anchor Link

JsonbObjectCopy Anchor Link

BindCopy Anchor Link

compareCopy Anchor Link

all_holdCopy Anchor Link

On this page

Store

`PgVectorStore`

`ExtraFilter`

`CHUNK`

Tables

`PgTable`

`Embedding`

`VectorType`

`Metric`

Field storage

`FieldSource`

`Columns`

`JsonbObject`

`Bind`

`compare`

`all_hold`